Beef cattle body size measuring system based on image recognition

Abstract

The invention relates to the technical field of cattle body measurement, in particular to a beef cattle body size measuring system based on image recognition. According to the technical scheme, the system comprises a multi-modal data acquisition module, a model construction and measurement module, a dynamic attitude estimation and compensation module, a multi-source data fusion module and a visual interaction module. According to the invention, a data basis is provided for subsequent processing through multi-modal data fusion, the attitude estimation accuracy is improved through the attitude estimation sub-module, and the accuracy of a body size measurement result is further improved through an improved ICP registration algorithm. The system can estimate and compensate the gestures of the beef cattle when the beef cattle are in various natural gestures through the dynamic posture estimation and compensation module, so that the accuracy of body size measurement is ensured, in addition, the visualization effect and interactivity of data are enhanced through the visualization interaction module, a user can conveniently find the abnormal gestures of the beef cattle in time, and the user experience is improved. A scientific decision basis is provided for the user, and the user is helped to better manage the breeding process of beef cattle.

Description

Technical Field

[0001] The present invention relates to the technical field of cattle body measurement, and in particular to a beef cattle body size measurement system based on image recognition. Background Art

[0002] Beef cattle are sheep raised specifically for meat production, characterized by rapid growth and superior meat quality. Beef cattle farming is widespread worldwide, with particular importance in Australia, New Zealand, and China. Measuring beef cattle body dimensions is a crucial tool for assessing their growth, development, production performance, and breed characteristics. By measuring height, length, chest circumference, and other body dimensions, we can understand their growth, nutritional status, and genetic potential, providing a scientific basis for cattle breeding, feeding, and production efficiency improvement. Traditional methods for measuring beef cattle body dimensions typically rely on manual measurement, using tools such as tape measures and measuring rods to measure directly on the sheep. This method has the problems of high labor intensity and low measurement efficiency. With the development of science and technology, technologies such as image recognition have gradually been applied to the measurement of beef cattle body size. However, existing measurement systems usually require beef cattle to maintain a relatively standard standing posture, such as upright limbs and upright body, in order to accurately identify and measure body size indicators. However, it is difficult for beef cattle to maintain such a standard posture in their natural state. Their random movements and posture changes will lead to increased errors in image recognition. Therefore, we propose a beef cattle body size measurement system based on image recognition. Summary of the Invention

[0003] The purpose of the present invention is to address the problems existing in the background technology and to propose a beef cattle body size measurement system based on image recognition.

[0004] The technical solution of the present invention is a beef cattle body size measurement system based on image recognition, which includes a multimodal data acquisition module, a model building and measurement module, a dynamic posture estimation and compensation module, a multi-source data fusion module and a visualization interaction module, wherein:

[0005] The multimodal data collection module is used to collect multi-dimensional data;

[0006] The dynamic posture estimation and compensation module estimates the dynamic posture of beef cattle in real time based on multi-dimensional data and performs compensation calculation based on the posture compensation algorithm;

[0007] The model building and measurement module is used to build a three-dimensional point cloud model and measure the body size of beef cattle;

[0008] The multi-source data fusion module is used to integrate the multi-dimensional data and perform fusion processing;

[0009] The visual interaction module is used to provide a visual interaction platform.

[0010] Optionally, the multimodal data collection module includes an adjustable camera array and an embedded environmental perception unit, the adjustable camera array includes an RGB-D camera array and a liftable robotic arm, the camera array is fixedly mounted at the end of the liftable robotic arm, the embedded environmental perception unit is used to obtain the surface temperature distribution of beef cattle and the distance information between the beef cattle and the sensor, the embedded environmental perception unit includes a thermal imaging sensor and a laser ranging module.

[0011] Optionally, the dynamic posture estimation and compensation module includes a posture estimation submodule based on an improved HRNet neural network architecture. The improved HRNet neural network architecture introduces a hybrid attention mechanism module in the feature extraction stage, and its calculation formula is:

[0012]

[0013] Where Q, K, and V represent query, key, and value matrices respectively, ⊙ represents the Hadamard product, and d k Indicates the dimension of the key matrix K, used for QK T The results are normalized;

[0014] The improved HRNet neural network architecture introduces a breed feature constraint loss function in the network training stage. Assume that the key bone length ratio of a certain breed of beef cattle in the breed feature database is The corresponding bone length ratio predicted by the network is Then the variety characteristic constraint loss function formula is expressed as:

[0015]

[0016] Among them, j represents different bone ratios, m is the number of bone ratio relationships, and the network parameters are adjusted by the back propagation algorithm. At the same time, combined with the bone length constraint loss function, its formula is expressed as:

[0017]

[0018] in, represents the i-th predicted bone length value, Represents the pre-established beef cattle breed anatomical database, represents the prior bone length value of the i-th corresponding breed, and n represents the number of bone length data involved in the calculation.

[0019] Optionally, the dynamic posture estimation and compensation module also includes a posture compensation algorithm submodule, which includes constructing a kinematic model of beef cattle containing 16-18 joint points, wherein the joint points include key parts of the head, neck, shoulder, elbow, wrist, chest, waist, buttocks, hip, knee, and ankle, and using the Jacobian matrix inverse method to solve the joint angle compensation amount, assuming that the operating space position vector is x=[x, y, z, θ x ,θ y ,θ z ] T , the joint space angle vector is q=[q1,q2,…,q n ] T , the Jacobian matrix J satisfies the formula When the posture needs to be compensated, the desired change in the operating space position Δx is known, and the inverse of the Jacobian matrix J is solved. -1 Calculate the joint angle compensation Δq, the formula is expressed as:

[0020] Δq=J -1 (q)Δx

[0021] When the determinant in the Jacobian matrix is 0, the pseudo-inverse matrix The approximate solution is as follows:

[0022]

[0023] Among them, the pseudo-inverse matrix Calculation is performed using the singular value decomposition method;

[0024] The kinematic model of beef cattle is constructed using the DH parameter method. The rotation constraints of each joint are defined, and a coordinate system is established for each joint. The relative position and posture relationship between adjacent joint coordinate systems is described by four DH parameters. The four DH parameters include joint offset d, joint rotation angle θ, connecting rod length a, and connecting rod torsion angle α. For adjacent joints i and i+1, the homogeneous transformation matrix from the joint i coordinate system to the joint i+1 coordinate system is Expressed as:

[0025]

[0026] The kinematic model of the beef cattle is obtained by sequentially connecting the homogeneous transformation matrices of each joint. The total transformation matrix from the base coordinate system to the end effector is for:

[0027]

[0028] Where n is the total number of joints.

[0029] Optionally, the model building and measurement module uses a multi-view point cloud registration submodule based on an improved ICP algorithm to perform point cloud stitching, and its objective function is:

[0030]

[0031] Among them, R is the rotation matrix used to describe the rotation transformation of the point cloud in space, p i represents the i-th point in the source point cloud, t is the translation vector, q i Indicates the target point cloud with p i The corresponding i-th point, N represents the number of point pairs involved in the registration, λ is the weight coefficient, Ω(R) represents the rotation smoothness constraint term, and the beef cattle 3D point cloud model is constructed through multiple iterations.

[0032] Optionally, the model building and measurement module further includes a feature extraction submodule, which includes a key anatomical point positioning algorithm based on curvature analysis and thermal imaging feature fusion, fits the surface of the beef cattle by moving least squares method, calculates Gaussian curvature to identify the highest point of the buttocks, and the calculation formula is:

[0033]

[0034] Among them, L and N represent the coefficients of the second basic form, M is used to represent the degree of distortion of the surface, E and G represent the coefficients of the first basic form, F represents the angle between the two main directions on the tangent plane, and K is the Gaussian curvature;

[0035] The deep point of the beef brisket is located by the temperature gradient change of the thermal imaging, and the temperature gradient is calculated after the thermal imaging data collected by the thermal imaging sensor is denoised. At each pixel point (x, y) of the image, the gradient operator is used to calculate the horizontal direction G x and vertical direction G y The gradient component of the horizontal gradient G x The calculation formula is:

[0036]

[0037] The vertical gradient G y The calculation formula is:

[0038]

[0039] Among them, * represents convolution operation, I filtered (x,y) represents the pixel value of the denoised thermal image at coordinate (x,y);

[0040] Through the horizontal direction G x and vertical direction G yThe gradient component of the pixel point is calculated using the following formula:

[0041]

[0042] Based on the thermal imaging image of beef cattle, a rectangular area is delineated as the region of interest (ROI) with the center of the beef cattle chest as the reference. Within the region of interest, the local maximum point of the gradient amplitude is searched for the pixels that meet the requirements, and the point is used as the candidate chest depth point. The candidate chest depth point with the maximum gradient amplitude close to the symmetry axis of the beef cattle chest is selected as the chest depth point.

[0043] Optionally, the model building and measurement module further includes an intelligent measurement engine submodule, which includes body size calculation rules, including body height calculation, body length calculation, chest circumference calculation, and tube circumference calculation. The body height calculation includes plane fitting of ground point cloud data using a random sampling consistency algorithm. Assuming that the fitted ground equation is ax+by+cz+d=0, the calculation formula for body height H is:

[0044]

[0045] in, The highest point of the shoulder shoulder-max The z coordinate of

[0046] The calculation formula of body length L in the body length calculation is:

[0047]

[0048] in, and The nose tip point p of beef cattle nose and the posterior end point of the buttocks hip-end The three-dimensional coordinates of

[0049] The chest circumference calculation is performed by using a convex hull algorithm to calculate the convex hull of the beef cattle chest point cloud. The point set on the convex hull is C, and the convex hull perimeter is calculated using a discretization method. Suppose there are n points on the chest convex hull, which are p1, p2, ..., p n , beef cattle breast circumference C chest The calculation formula is:

[0050]

[0051] in, represents the three-dimensional coordinates of the i-th point on the chest convex hull;

[0052] The tube circumference calculation includes identifying the tube diameter point set T of the beef cattle tube bone by a local feature extraction algorithm, and calculating the beef cattle tube circumference by the convex hull algorithm. Assume that there are m points on the convex hull of the tube bone, which are q1, q2, ..., q n , beef tubular bone C pipe The calculation formula is:

[0053]

[0054] in, Represents the three-dimensional coordinates of the j-th point on the convex hull of the tubular bone.

[0055] Optionally, the multi-source data fusion analysis module includes a data processing framework submodule and a fusion algorithm submodule. The data processing framework submodule is used to extract features from image data and thermal imaging data. The fusion algorithm submodule performs data fusion on image data features, thermal imaging data features, and historical growth data features in different time and space dimensions based on a spatiotemporal attention fusion network.

[0056] Assume that the input data is image data X1, thermal imaging data X2, and historical growth data X3. The formula of the fused data X is: X=w1X1+w2X2+w3X3, where w1, w2, and w3 are weight coefficients. The weight coefficients are calculated by the spatiotemporal attention mechanism. The spatiotemporal attention weight w t,s The calculation formula is as follows:

[0057] w ts =σ(W t ·h t +W s ·h s +b)

[0058] Among them, h t is the time characteristic, h s is the spatial feature, W t and W s are the weight matrices corresponding to temporal features and spatial features respectively, b is the bias term, σ is the activation function, and the activation function is the Sigmoid function;

[0059] The multi-source data fusion analysis module also includes a variety characteristic knowledge graph, which is constructed based on the Neo4j graph database and includes a "has_standard" relationship between variety nodes and standard body size parameters, an "affects" relationship between environmental temperature and humidity nodes and growth rate, and an "indicates" relationship between abnormal body size measurements and disease types.

[0060] Optionally, the visualization interaction module includes an AR enhanced display submodule based on Hololens2, an abnormal posture warning submodule, and a growth trend prediction submodule. The AR enhanced display submodule is used to fuse the three-dimensional point cloud model with the real-time collected beef cattle image and display the beef cattle body size annotation information, including body height, body length and chest circumference, on a visualization interface;

[0061] The abnormal posture warning submodule calculates the degree of deviation between the current posture of the beef cattle and the standard posture based on the dynamic posture estimation submodule and the compensation module, and displays it in the form of a heat map on the visualization interface, marking it with different colors according to the degree of deviation;

[0062] The growth trend prediction submodule learns the historical body size of beef cattle through the long short-term memory network and outputs the prediction result of the body size change trend of beef cattle. Assume that the input sequence is X=[x1,x2,…,x T ], where x t =[H t ,L t ,C t ] T The data of height, length and chest depth at the time t are calculated by the long short-term memory network, and the output sequence is Y = [y1, y2, ..., y T ],in, Represents the predicted body height, body length and chest depth data at the tth moment.

[0063] Optionally, the AR enhanced display submodule includes performing coordinate transformation on the three-dimensional point cloud model, assuming that the point coordinates of the three-dimensional point cloud model in the original coordinate system are P t =[x cloud ,y cloud ,z cloud ] T , the coordinates of the point transferred to the Hololens2 coordinate system through the coordinate transformation matrix T are P aligned =TPcloud, where T is a 4×4 homogeneous transformation matrix containing rotation and translation information, and features are extracted from the real-time collected beef cattle images and matched with corresponding feature points in the three-dimensional point cloud model.

[0064] In summary, this application includes at least one of the following beneficial technical effects:

[0065] The present invention provides a data foundation for subsequent processing through multimodal data fusion and optimizes the network training process through a posture estimation submodule based on an improved HRNet neural network architecture. This makes the predicted posture more consistent with the actual physiological structure of beef cattle, improves the accuracy of posture estimation, and thus ensures the accuracy of body measurement. At the same time, the improved ICP registration algorithm ensures the accuracy of point cloud splicing, and the combination of the automatic feature extraction algorithm further improves the accuracy of body measurement results.

[0066] The present invention uses a dynamic posture estimation and compensation module that can tolerate ±30° pitch / yaw angle deviation and supports asymmetric standing posture measurement. This enables the system to estimate and compensate for the posture of beef cattle in various natural postures, thereby ensuring the accuracy of body measurement and adapting to the actual situation where beef cattle find it difficult to maintain a standard standing posture in their natural state.

[0067] The present invention uses an AR enhanced display submodule based on Hololens2 to intuitively display the body size annotation information of beef cattle on a visual interface, thereby enhancing the visualization effect and interactivity of the data, allowing users to more intuitively understand the growth status of beef cattle. At the same time, the abnormal posture warning submodule calculates the degree of deviation between the current posture of the beef cattle and the standard posture in real time, and displays it in the form of a heat map on the visual interface, making it convenient for users to promptly discover abnormal postures of beef cattle. In addition, the growth trend prediction submodule based on the long short-term memory network learns the historical body size data of beef cattle and outputs the prediction results of the body size change trend of beef cattle, providing users with a scientific decision-making basis and helping users better manage the breeding process of beef cattle. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 The structural block diagram of the beef cattle body size measurement system based on image recognition of the present invention is given. DETAILED DESCRIPTION

[0069] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0070] Example

[0071] The beef cattle body size measurement system based on image recognition proposed by the present invention is as follows: Figure 1 As shown, it includes multimodal data acquisition module, model building and measurement module, dynamic posture estimation and compensation module, multi-source data fusion module and visualization interaction module.

[0072] The multimodal data collection module is used to collect multi-dimensional data. It includes an adjustable camera array and an embedded environmental perception unit. A spacious area with relatively uniform lighting and frequent cattle activities is selected in the farm as the measurement point. The adjustable camera array is preferably 4 sets of RGB-D cameras and a liftable robotic arm. The 4 sets of RGB-D cameras are fixedly installed at the end of the liftable robotic arm. By adjusting the position and angle of the liftable robotic arm, the camera array forms a coverage area of 120°×60° to ensure that the body of the beef cattle can be fully photographed. At the same time, thermal imaging sensors and laser ranging modules are installed in this area to obtain the surface temperature distribution of the beef cattle and the distance information between the beef cattle and the sensor. The RGB-D camera array collects color images and depth information of the beef cattle at a speed of 30 frames per second, and the thermal imaging sensor obtains the surface temperature distribution of the beef cattle in real time at a speed of 10 frames per second. The laser ranging module measures the distance between the beef cattle and the sensor every 0.1 seconds.

[0073] During the installation process, the embedded environmental perception unit is debugged. At the same time, the camera array, thermal imaging sensor and laser ranging module are synchronized at the microsecond level through the PTP protocol to ensure the time consistency of the collected data. The present invention provides a data basis for subsequent processing through multimodal data fusion.

[0074] The dynamic posture estimation and compensation module estimates the dynamic posture of beef cattle in real time based on multi-dimensional data and performs compensation calculations based on the posture compensation algorithm. The dynamic posture estimation and compensation module includes a posture estimation submodule based on the improved HRNet neural network architecture.

[0075] The improved HRNet neural network architecture introduces a hybrid attention mechanism module in the feature extraction stage, and its calculation formula is:

[0076]

[0077] Where Q, K, and V represent query, key, and value matrices respectively, ⊙ represents the Hadamard product, and d k Indicates the dimension of the key matrix K, used for QK T The results are normalized;

[0078] The improved HRNet neural network architecture introduces a breed-constrained loss function during the network training phase. Before training, bone proportion data of 1,000 different beef cattle aged 1 to 3 years were collected from the beef cattle breed database. The bone proportion data were organized into a standard format and input into the breed-constrained loss function. The key bone length ratio of a certain breed of beef cattle in the breed database is set to The corresponding bone length ratio predicted by the network is The formula of the variety characteristic constraint loss function is expressed as:

[0079]

[0080] Among them, j represents different bone ratios, m is the number of bone ratio relationships, and the network parameters are adjusted by the back propagation algorithm. At the same time, combined with the bone length constraint loss function, its formula is expressed as:

[0081]

[0082] in, represents the i-th predicted bone length value, represents a pre-established anatomical database of beef cattle breeds, represents the prior bone length value of the i-th corresponding breed, and n represents the number of bone length data involved in the calculation. During the training process, a GPU-accelerated deep learning framework is used to set the training parameters, the learning rate is 0.001, the number of iterations is 5000, and the network parameters are adjusted through the backpropagation algorithm. The loss function value is monitored in real time during the training process until it is monitored that the loss function value tends to be stable and reaches a low level.

[0083] The present invention optimizes the network training process by introducing a posture estimation submodule with breed characteristic constraint loss function and bone length constraint loss function, so that the predicted posture is more consistent with the actual physiological structure of beef cattle, the accuracy of posture estimation is improved, and the accuracy of body size measurement is guaranteed.

[0084] The dynamic posture estimation and compensation module also includes a posture compensation algorithm submodule. The posture compensation algorithm submodule includes constructing a kinematic model of beef cattle containing 16-18 joint points. The joint points include the key parts of the head, neck, shoulder, elbow, wrist, chest, waist, buttocks, hip, knee, and ankle. Among them, the pitch angle range of the shoulder joint is set to [-30°, 45°], and the yaw angle range of the hip joint is set to [-40°, 35°]. The inverse Jacobi matrix method is used to solve the joint angle compensation. Suppose the operating space position vector is x = [x, y, z, θ x ,θ y ,θ z ] T , the joint space angle vector is q=[q1,q2,…,q n ] T , the Jacobian matrix J satisfies the formula When the posture needs to be compensated, the desired change in the operating space position Δx is known, and the inverse of the Jacobian matrix J is solved. -1 Calculate the joint angle compensation Δq, the formula is expressed as:

[0085] Δq=J -1 (q)Δx

[0086] When the determinant in the Jacobian matrix is 0, the pseudo-inverse matrix The approximate solution is as follows:

[0087]

[0088] Among them, the pseudo-inverse matrix The calculation is performed by singular value decomposition method, and the kinematic model of beef cattle is constructed by DH parameter method. The rotation constraints of each joint are defined, and a coordinate system is established for each joint. The relative position and posture relationship between adjacent joint coordinate systems are described by four DH parameters. The four DH parameters include joint offset d, joint rotation angle θ, connecting rod length a and connecting rod torsion angle α. For a certain adjacent joint i and i+1, the homogeneous transformation matrix from the joint i coordinate system to the joint i+1 coordinate system is Expressed as:

[0089]

[0090] The kinematic model of beef cattle is obtained by sequentially connecting the homogeneous transformation matrices of each joint. The total transformation matrix from the base coordinate system to the end effector is Here, n is the total number of joints. Before training, simulation calculations are performed on common posture changes of beef cattle, including walking and lying posture changes, to establish a posture change database so that posture compensation can be performed quickly and accurately in actual measurements.

[0091] The present invention can tolerate ±30° pitch / yaw angle deviation through a dynamic posture estimation and compensation module and supports asymmetric standing posture measurement, enabling the system to estimate and compensate for the posture of beef cattle in various natural postures, thereby ensuring the accuracy of body measurement and adapting to the actual situation where beef cattle find it difficult to maintain a standard standing posture in their natural state.

[0092] The model building and measurement module is used to build a three-dimensional point cloud model and measure the body size of beef cattle. The model building and measurement module uses a multi-view point cloud registration submodule based on the improved ICP algorithm to perform point cloud stitching. Its objective function is:

[0093]

[0094] Among them, R is the rotation matrix used to describe the rotation transformation of the point cloud in space, p i represents the i-th point in the source point cloud, t is the translation vector, q i Indicates the target point cloud with p iThe corresponding i-th point, N represents the number of point pairs involved in the registration, λ is the weight coefficient, Ω(R) represents the rotation smoothing constraint term, the point cloud data collected from different camera perspectives are used as input, the weight coefficient in the objective function is set to 0.8 and the rotation smoothing constraint term is set to 0.2, and a three-dimensional point cloud model of beef cattle is constructed through multiple iterations. The present invention ensures the accuracy of point cloud splicing through an improved ICP registration algorithm and further improves the accuracy of body size measurement results by combining the automatic feature extraction algorithm.

[0095] The model building and measurement module also includes a feature extraction submodule. The feature extraction submodule includes a key anatomical point positioning algorithm based on curvature analysis and thermal imaging feature fusion. The surface of the beef cattle is fitted using the moving least squares method. During the fitting process, a Gaussian kernel function is selected and the kernel function bandwidth is adjusted according to the point cloud density to improve the fitting accuracy. The Gaussian curvature is also calculated to identify the highest point of the buttocks. The calculation formula is:

[0096]

[0097] Among them, L and N represent the coefficients of the second basic form, M is used to represent the degree of distortion of the surface, E and G represent the coefficients of the first basic form, F represents the angle between the two main directions on the tangent plane, and K is the Gaussian curvature;

[0098] The deep point of the beef brisket is located by changing the temperature gradient of thermal imaging. The temperature gradient is calculated after denoising the thermal imaging data collected by the thermal imaging sensor. At each pixel (x, y) of the image, the Sobel operator is used to calculate the horizontal direction G x and vertical direction G y The gradient component, horizontal gradient G x The calculation formula is:

[0099]

[0100] Vertical gradient G y The calculation formula is:

[0101]

[0102] Among them, * represents convolution operation, I filtered (x, y) represents the pixel value of the denoised thermal image at the coordinate (x, y), and the horizontal direction G x and vertical direction G y The gradient component of the pixel point is calculated using the following formula:

[0103]

[0104] Based on the thermal imaging image of beef cattle, a rectangular area is delineated as the region of interest (ROI) with the center of the beef cattle chest as the reference. Within the region of interest, the local maximum point of the gradient amplitude is searched for the pixel points that meet the requirements, and it is used as the candidate chest depth point. The candidate chest depth point with the maximum gradient amplitude close to the symmetry axis of the beef cattle chest is selected as the chest depth point.

[0105] The model building and measurement module also includes an intelligent measurement engine submodule, which includes body size calculation rules. The body size calculation rules include body height calculation, body length calculation, chest circumference calculation, and tube circumference calculation. Body height calculation includes plane fitting of ground point cloud data using a random sampling consistency algorithm. During the fitting process, the number of samplings is set to 100 times, the internal point threshold is set to 0.05m, and the fitting ground equation is set to ax+by+cz+d=0. The calculation formula for body height H is:

[0106]

[0107] in, The highest point of the shoulder shoulder-max The z coordinate of

[0108] The calculation formula of body length L in body length calculation is:

[0109]

[0110] in, and The nose tip point p of beef cattle nose and the posterior end point of the buttocks hip-end The three-dimensional coordinates of .

[0111] The chest circumference is calculated by using the convex hull algorithm to calculate the convex hull of the beef cattle chest point cloud. Let the point set on the convex hull be C, and the discretization method is used to calculate the perimeter of the convex hull. Let there be n points on the chest convex hull, which are p1, p2, ..., p n , beef cattle breast circumference C chest The calculation formula is:

[0112]

[0113] in, Represents the three-dimensional coordinates of the i-th point on the chest convex hull.

[0114] The tube circumference calculation includes identifying the tube diameter point set T of the beef cattle tube bone through the local feature extraction algorithm, and calculating the beef cattle tube circumference through the convex hull algorithm. Assume that there are m points on the convex hull of the tube bone, which are q1, q2, ..., q n , beef tubular bone C pipe The calculation formula is:

[0115]

[0116] in, Represents the three-dimensional coordinates of the j-th point on the convex hull of the tubular bone.

[0117] The multi-source data fusion module is used to integrate multi-dimensional data and perform fusion processing. The multi-source data fusion analysis module includes a data processing framework submodule and a fusion algorithm submodule. The data processing framework submodule is used to extract features from image data and thermal imaging data. The SIFT algorithm is used to extract feature points from image data. During the extraction process, the number of Gaussian difference pyramid layers for scale space extreme value detection is set to 4, and the number of image groups in each layer is set to 5 to ensure that sufficient and accurate feature points can be extracted. The finite difference method is used to extract temperature gradient features from thermal imaging data, and the difference step size is set to 1 pixel. The fusion algorithm submodule performs data fusion on image data features, thermal imaging data features, and historical growth data features in different time and space dimensions based on the spatiotemporal attention fusion network.

[0118] Assume that the input data is image data X1, thermal imaging data X2, and historical growth data X3. The formula of the fused data X is: X = w1X1 + w2X2 + w3X3, where w1, w2, and w3 are weight coefficients. The weight coefficients are calculated by the spatiotemporal attention mechanism. The spatiotemporal attention weight w t,s The calculation formula is as follows:

[0119] w ts =σ(W t ·h t +W s ·h s +b)

[0120] Among them, h t is the time characteristic, h s is the spatial feature, W t and W s are the weight matrices corresponding to temporal features and spatial features respectively, b is the bias term, σ is the activation function, and the activation function is the Sigmoid function;

[0121] The multi-source data fusion analysis module also includes a variety characteristic knowledge graph, which is built based on the Neo4j graph database and includes the "has_standard" relationship between variety nodes and standard body size parameters, the "affects" relationship between environmental temperature and humidity nodes and growth rate, and the "indicates" relationship between abnormal body size measurements and disease types.

[0122] The visualization interaction module is used to provide a visualization interaction platform. The visualization interaction module includes an AR enhanced display submodule based on Hololens2, an abnormal posture warning submodule, and a growth trend prediction submodule. The AR enhanced display submodule is used to fuse the three-dimensional point cloud model with the real-time collected beef cattle image and display the beef cattle body size annotation information on the visualization interface, including body height, body length, and chest circumference. The AR enhanced display submodule includes coordinate conversion of the three-dimensional point cloud model. Let the point coordinates of the three-dimensional point cloud model in the original coordinate system be P t =[x cloud ,y cloud ,z cloud ] T , the coordinates of the point transferred to the Hololens2 coordinate system through the coordinate transformation matrix T are P aligned =TPcloud, where T is a 4×4 homogeneous transformation matrix containing rotation and translation information. Feature extraction is performed on the real-time acquired beef cattle image and matched with the corresponding feature points in the 3D point cloud model. The 3D point cloud model is fused with the real-time acquired beef cattle image using a matching algorithm based on a feature descriptor. The matching threshold is set to 0.8, and the beef cattle body size annotation information, including body height, body length, and chest circumference, is displayed on a visualization interface. The present invention intuitively displays the beef cattle body size annotation information on a visualization interface through an AR enhanced display submodule based on Hololens2, thereby enhancing the visualization effect and interactivity of the data and enabling users to more intuitively understand the growth status of the beef cattle.

[0123] The abnormal posture warning submodule calculates the degree of deviation between the current posture of the beef cattle and the standard posture based on the dynamic posture estimation submodule and the compensation module, and displays it in the form of a heat map on the visualization interface. Different colors are used for marking according to the degree of deviation. During the display process, the font size, color and display position of the body size annotation information can be adjusted according to user needs. The posture deviation threshold is set to 15°. When the deviation exceeds the threshold, red indicates severe abnormality, yellow indicates moderate abnormality, and green indicates normal. The present invention calculates the degree of deviation between the current posture of the beef cattle and the standard posture in real time through the abnormal posture warning submodule, and displays it in the form of a heat map on the visualization interface, so that users can find the abnormal posture of the beef cattle in time.

[0124] The growth trend prediction submodule learns the historical body size of beef cattle through the long short-term memory network and outputs the prediction results of the body size change trend of beef cattle. Suppose the input sequence is X=[x1,x2,…,x T ], where x t =[H t ,L t ,C t ] TRepresents the height, length and chest depth data at the tth moment. After calculation through the long short-term memory network, the output sequence is Y = [y1, y2, ..., y T ],in, The system represents the predicted height, length, and chest depth data at the tth moment. The present invention uses a growth trend prediction submodule based on a long short-term memory network to learn historical body size data of beef cattle and outputs predicted results of the changing trend of beef cattle body size. This provides users with a scientific decision-making basis and helps them better manage the beef cattle breeding process.

[0125] The above specific embodiments are merely several optional embodiments of the present invention. Based on the technical solutions of the present invention and the relevant inspirations of the above embodiments, those skilled in the art may make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A beef cattle body size measurement system based on image recognition, characterized in that: It includes multimodal data acquisition module, dynamic posture estimation and compensation module, model construction and measurement module, multi-source data fusion module and visualization interaction module, among which: The multimodal data collection module is used to collect multi-dimensional data; The model construction and measurement module is used to construct a three-dimensional point cloud model and measure the body size of beef cattle; the dynamic posture estimation and compensation module estimates the dynamic posture of beef cattle in real time based on multi-dimensional data and performs compensation calculation based on the posture compensation algorithm; The multi-source data fusion module is used to integrate the multi-dimensional data and perform fusion processing; The visual interaction module is used to provide a visual interaction platform.

2. The beef cattle body size measurement system based on image recognition according to claim 1, characterized in that: The multimodal data collection module includes an adjustable camera array and an embedded environmental perception unit. The adjustable camera array includes an RGB-D camera array and a liftable robotic arm. The camera array is fixedly installed at the end of the liftable robotic arm. The embedded environmental perception unit is used to obtain the surface temperature distribution of beef cattle and the distance information between the beef cattle and the sensor. The embedded environmental perception unit includes a thermal imaging sensor and a laser ranging module.

3. The beef cattle body size measurement system based on image recognition according to claim 1, characterized in that: The dynamic posture estimation and compensation module includes a posture estimation submodule based on the improved HRNet neural network architecture. The improved HRNet neural network architecture introduces a hybrid attention mechanism module in the feature extraction stage, and its calculation formula is: Where Q, K, and V represent query, key, and value matrices respectively, ⊙ represents the Hadamard product, and d k Indicates the dimension of the key matrix K, used for QK T The results are normalized; The improved HRNet neural network architecture introduces a breed feature constraint loss function in the network training stage. Assume that the key bone length ratio of a certain breed of beef cattle in the breed feature database is The corresponding bone length ratio predicted by the network is Then the variety characteristic constraint loss function formula is expressed as: Among them, j represents different bone ratios, m is the number of bone ratio relationships, and the network parameters are adjusted by the back propagation algorithm. At the same time, combined with the bone length constraint loss function, its formula is expressed as: in, represents the i-th predicted bone length value, Represents the pre-established beef cattle breed anatomical database, represents the prior bone length value of the i-th corresponding breed, and n represents the number of bone length data involved in the calculation.

4. The beef cattle body size measurement system based on image recognition according to claim 3, characterized in that: The dynamic posture estimation and compensation module also includes a posture compensation algorithm submodule, which includes constructing a kinematic model of beef cattle containing 16-18 joint points, wherein the joint points include the key parts of the head, neck, shoulder, elbow, wrist, chest, waist, buttocks, hip, knee, and ankle, and using the inverse Jacobian matrix method to solve the joint angle compensation amount. Assume that the operating space position vector is x=[x, y, z, θ x ,θ y ,θ z ] T , the joint space angle vector is q=[q1,q2,…,q n ] T , the Jacobian matrix J satisfies the formula When the posture needs to be compensated, the desired change in the operating space position Δx is known, and the inverse of the Jacobian matrix J is solved. -1 Calculate the joint angle compensation Δq, the formula is expressed as: Δq=J -1 (q)Δx When the determinant in the Jacobian matrix is 0, the pseudo-inverse matrix The approximate solution is as follows: Among them, the pseudo-inverse matrix Calculation is performed using the singular value decomposition method; The kinematic model of beef cattle is constructed using the DH parameter method. The rotation constraints of each joint are defined, and a coordinate system is established for each joint. The relative position and posture relationship between adjacent joint coordinate systems is described by four DH parameters. The four DH parameters include joint offset d, joint rotation angle θ, connecting rod length a, and connecting rod torsion angle α. For adjacent joints i and i+1, the homogeneous transformation matrix from the joint i coordinate system to the joint i+1 coordinate system is Expressed as: The kinematic model of the beef cattle is obtained by sequentially connecting the homogeneous transformation matrices of each joint. The total transformation matrix from the base coordinate system to the end effector is for: Where n is the total number of joints.

5. The beef cattle body size measurement system based on image recognition according to claim 1, characterized in that: The model building and measurement module uses a multi-view point cloud registration submodule based on the improved ICP algorithm to perform point cloud stitching, and its objective function is: Among them, R is the rotation matrix used to describe the rotation transformation of the point cloud in space, p i represents the i-th point in the source point cloud, t is the translation vector, q i Indicates the target point cloud with p i The corresponding i-th point, N represents the number of point pairs involved in the registration, λ is the weight coefficient, θ(R) represents the rotation smoothness constraint term, and the beef cattle 3D point cloud model is constructed through multiple iterations.

6. The beef cattle body size measurement system based on image recognition according to claim 5, characterized in that: The model building and measurement module also includes a feature extraction submodule, which includes a key anatomical point positioning algorithm based on curvature analysis and thermal imaging feature fusion, fits the surface of the beef cattle through the moving least squares method, calculates the Gaussian curvature to identify the highest point of the buttocks, and the calculation formula is: Among them, L and N represent the coefficients of the second basic form, M is used to represent the degree of distortion of the surface, E and G represent the coefficients of the first basic form, F represents the angle between the two main directions on the tangent plane, and K is the Gaussian curvature; The deep point of the beef brisket is located by the temperature gradient change of the thermal imaging, and the temperature gradient is calculated after the thermal imaging data collected by the thermal imaging sensor is denoised. At each pixel point (x, y) of the image, the gradient operator is used to calculate the horizontal direction G x and vertical direction G y The gradient component of the horizontal gradient G x The calculation formula is: The vertical gradient G y The calculation formula is: Among them, * represents convolution operation, I filtered (x,y) represents the pixel value of the denoised thermal image at coordinate (x,y); Through the horizontal direction G x and vertical direction G y The gradient component of the pixel point is calculated using the following formula: Based on the thermal imaging image of beef cattle, a rectangular area is delineated as the region of interest (ROI) with the center of the beef cattle chest as the reference. Within the region of interest, the local maximum point of the gradient amplitude is searched for the pixels that meet the requirements, and the point is used as the candidate chest depth point. The candidate chest depth point with the maximum gradient amplitude close to the symmetry axis of the beef cattle chest is selected as the chest depth point.

7. The beef cattle body size measurement system based on image recognition according to claim 5, characterized in that: The model building and measurement module also includes an intelligent measurement engine submodule, which includes body size calculation rules. The body size calculation rules include body height calculation, body length calculation, chest circumference calculation, and tube circumference calculation. The body height calculation includes plane fitting of ground point cloud data using a random sampling consistency algorithm. Assuming that the fitted ground equation is ax+by+cz+d=0, the calculation formula of body height H is: in, The highest point of the shoulder shoulder-max The z coordinate of The calculation formula of body length L in the body length calculation is: in, and The nose tip point p of beef cattle nose and the posterior end point of the buttocks hip-end The three-dimensional coordinates of The chest circumference calculation is performed by using a convex hull algorithm to calculate the convex hull of the beef cattle chest point cloud. The point set on the convex hull is C, and the convex hull perimeter is calculated using a discretization method. Suppose there are n points on the chest convex hull, which are p1, p2, ..., p n , beef cattle breast circumference C chest The calculation formula is: in, represents the three-dimensional coordinates of the i-th point on the chest convex hull; The tube circumference calculation includes identifying the tube diameter point set T of the beef cattle tube bone by a local feature extraction algorithm, and calculating the beef cattle tube circumference by the convex hull algorithm. Assume that there are m points on the convex hull of the tube bone, which are q1, q2, ..., q n , beef tubular bone C pipe The calculation formula is: in, Represents the three-dimensional coordinates of the j-th point on the convex hull of the tubular bone.

8. The beef cattle body size measurement system based on image recognition according to claim 1, characterized in that: The multi-source data fusion analysis module includes a data processing framework submodule and a fusion algorithm submodule. The data processing framework submodule is used to extract features from image data and thermal imaging data. The fusion algorithm submodule performs data fusion on image data features, thermal imaging data features and historical growth data features in different time and space dimensions based on a spatiotemporal attention fusion network. Assume that the input data is image data X1, thermal imaging data X2, and historical growth data X3. The formula of the fused data X is: X=w1X1+w2X2+w3X3, where w1, w2, and w3 are weight coefficients. The weight coefficients are calculated by the spatiotemporal attention mechanism. The spatiotemporal attention weight w t,s The calculation formula is as follows: w ts =σ(W t ·h t +W s ·h s +b) Among them, h t is the time characteristic, h s is the spatial feature, W t and W s are the weight matrices corresponding to temporal features and spatial features respectively, b is the bias term, σ is the activation function, and the activation function is the Sigmoid function; The multi-source data fusion analysis module also includes a variety characteristic knowledge graph, which is constructed based on the Neo4j graph database and includes a "has_standard" relationship between variety nodes and standard body size parameters, an "affects" relationship between environmental temperature and humidity nodes and growth rate, and an "indicates" relationship between abnormal body size measurements and disease types.

9. The beef cattle body size measurement system based on image recognition according to claim 1, characterized in that: The visualization interaction module includes an AR enhanced display submodule based on Hololens2, an abnormal posture warning submodule, and a growth trend prediction submodule. The AR enhanced display submodule is used to fuse the three-dimensional point cloud model with the real-time collected beef cattle image and display the beef cattle body size annotation information, including body height, body length, and chest circumference, on a visualization interface. The abnormal posture warning submodule calculates the degree of deviation between the current posture of the beef cattle and the standard posture based on the dynamic posture estimation submodule and the compensation module, and displays it in the form of a heat map on the visualization interface, marking it with different colors according to the degree of deviation; The growth trend prediction submodule learns the historical body size of beef cattle through the long short-term memory network and outputs the prediction result of the body size change trend of beef cattle. Assume that the input sequence is X=[x1,x2,…,x T ], where x t =[H t ,L t ,C t ] T The data of height, length and chest depth at the time t are calculated by the long short-term memory network, and the output sequence is Y = [y1, y2, ..., y T ],in, Represents the predicted body height, body length and chest depth data at the tth moment.

10. The beef cattle body size measurement system based on image recognition according to claim 9, characterized in that: The AR enhanced display submodule includes performing coordinate transformation on the three-dimensional point cloud model. Assume that the point coordinates of the three-dimensional point cloud model in the original coordinate system are P t =[x cloud ,y cloud ,z cloud ] T , the coordinates of the point transferred to the Hololens2 coordinate system through the coordinate transformation matrix T are P aligned =TPcloud, where T is a 4×4 homogeneous transformation matrix containing rotation and translation information, and features are extracted from the real-time collected beef cattle images and matched with corresponding feature points in the three-dimensional point cloud model.

Images