Cow identity recognition method based on datum point detection

Abstract

The invention belongs to the technical field of intelligent dairy cow monitoring, and particularly relates to a dairy cow identity recognition method based on datum point detection. The method comprises the following steps: firstly, acquiring a back three-dimensional point cloud of a dairy cow to be detected, and positioning two waist angle bone areas and two hip angle bone areas in the back three-dimensional point cloud; respectively selecting a point with the maximum curvature in each waist angle bone area as a waist angle bone point of the corresponding area, and respectively selecting a point with the maximum curvature in each hip angle bone area as a hip angle bone point of the corresponding area; carrying out horizontal plane projection on the back three-dimensional point cloud, and determining a key area which contains waist angle bone points and hip angle bone points and can extract enough body spot features of the dairy cow in projected data; and processing the point cloud projection of the key area to obtain a two-dimensional body spot image, and performing individual identity recognition of the to-be-detected dairy cow by using the two-dimensional body spot image. According to the invention, the three-dimensional structure characteristics and the body spot characteristics of the back of the dairy cow are combined to identify the individual identity of the dairy cow, so that the individual identity identification precision of the dairy cow is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of intelligent monitoring of dairy cows, and in particular relates to a dairy cow identity recognition method based on reference point detection. Background Art

[0002] Dairy farming is an important part of my country's animal husbandry. Due to the high labor intensity, time-consuming and labor-intensiveness of artificial farming, and strong subjectivity, my country's dairy farming is developing in the direction of informatization, intelligence and precision under the support of my country's increasingly developed information technology and intelligent technology. Precision intelligent farming can monitor the behavior information of individual dairy cows in real time and make timely management decisions. Individual dairy cow identity recognition is the premise and basis for farmers to monitor the health status of dairy cows, understand the growth of dairy cows, and analyze the milk production of dairy cows. In the precision farming of dairy cows, it is particularly important to accurately and quickly identify the identity of dairy cows.

[0003] At present, radio frequency identification technology (RFID) is a common individual identification method used in large dairy farms. It uses tags attached to the cows' bodies (usually ear tags) and wireless transmission technology to record the cows' individual information. Compared with traditional manual methods, RFID technology has indeed improved identification accuracy and work efficiency, but wearing ear tags is labor-intensive and may cause animal stress, affect animal welfare, and ear tags may be lost or damaged.

[0004] With the development of computer technology, non-contact cow identification methods based on computer vision have gradually emerged. This method can collect cow image information through a camera to extract biometric features, including the nose and mouth patterns, iris and facial contour textures, and body spot patterns. It can achieve automatic non-contact and accurate image recognition without a lot of manual operation. However, as physiological features of the head, the extraction of nose and mouth patterns, iris and facial contour texture features requires a very high degree of cooperation from the cow, and is easily affected by the shooting angle and position when collecting images.

[0005] Body spots refer to the regular distribution of black and white hair on the trunk of Holstein cows. Body spots are distributed over a wide area, and their pattern distribution characteristics are distinguishable and time-invariant. Body spot images can be obtained by collecting side and top view images or videos of cows walking. When taking top view images, the cow's gait will affect the angle of the cow captured by the camera, resulting in inconsistent direction and position of the color image. The Chinese invention patent with application publication number CN116012889A and application publication date 2023.04.25 discloses a method for identifying individual cows, which is based on cow side-view videos for cow identification. The overall idea of ​​the method is to first obtain the cow side-view image and input it into the torso area positioning model of the YOLOX series to obtain the position information of the cow torso area, and then crop the cow side-view image according to the position information of the cow torso area to obtain the cow body spot image; secondly, the cow body spot image is binarized and segmented to obtain a two-dimensional body spot image; then the category of the cow is determined based on the black / white pixels in the two-dimensional body spot image; finally, the two-dimensional body spot image is input into the EfficientNet-B0 individual classification model corresponding to the category of the cow to obtain the cow individual identification result. However, when shooting from the side, it is easy to have overlapping occlusion of multiple cows, which leads to a decrease in the accuracy of cow torso positioning, thereby making the cow identity recognition accuracy low. Summary of the invention

[0006] The object of the present invention is to provide a method for identifying a dairy cow based on reference point detection, so as to solve the problem of low accuracy of dairy cow identification in the prior art.

[0007] In order to solve the above technical problems, the present invention provides a method for identifying a dairy cow based on reference point detection, comprising the following steps:

[0008] 1) Obtain a three-dimensional point cloud of the back of the cow to be tested, and locate two lumbar bone regions and two hip angle bone regions; select the point with the maximum curvature in each lumbar bone region as the lumbar bone point of the corresponding region, and select the point with the maximum curvature in each hip angle bone region as the hip angle bone point of the corresponding region;

[0009] 2) Projecting the back three-dimensional point cloud on a horizontal plane, determining a key area in the projected data that includes the waist angle bone point and the hip angle bone point and can extract sufficient body spot features of the cow;

[0010] 3) The point cloud projection of the key area is processed to obtain a two-dimensional body spot image, and the two-dimensional body spot image is used to identify the individual identity of the cow to be detected.

[0011] Furthermore, the method for selecting the point with the maximum curvature in the lumbar angle bone area / hip angle bone area is: for a given point, determine its nearest neighbor point set, and use the nearest neighbor point set to fit the local surface around the point; determine the principal curvature based on the fitted local surface, and then determine the Gaussian curvature based on the principal curvature, and the Gaussian curvature is the curvature of the given point.

[0012] Furthermore, the key area is determined by: determining the waist width, which is the length of the line connecting the two waist angle bone points; extending the waist width from the two waist angle bone points in the direction opposite to the hip angle bone, thereby obtaining a square area with a side length equal to the waist width; merging the area enclosed by the two waist angle bone points and the two hip angle bone points and the square area as the key area.

[0013] Furthermore, the method of locating the lumbar bone region and the hip bone region is: inputting the three-dimensional point cloud of the back of the cow to be detected into the trained segmentation model to segment two lumbar bone regions and two hip bone regions.

[0014] Furthermore, the method of using the two-dimensional body spot image to perform individual identity recognition of the cow to be detected is: binarizing the two-dimensional body spot image, inputting the binarized two-dimensional body spot image into the trained classification model, and obtaining the individual identity recognition result of the cow to be detected.

[0015] Furthermore, the two-dimensional volume spot image that is binarized is a two-dimensional volume spot image that is size-normalized using an interpolation method.

[0016] Furthermore, the obtained three-dimensional point cloud of the cow's back is preprocessed data, and the preprocessing includes posture normalization processing; the process of the posture normalization processing is: extracting the ground point cloud in the three-dimensional point cloud data of the cow's back before preprocessing, and performing plane fitting to obtain a fitting plane, and extracting the normal vector of the fitting plane; determining the normal vector of the set XOY plane; using the Rodrigues formula to calculate the rotation matrix for the normal vector of the fitting plane to the normal vector of the XOY plane, and using the rotation matrix to rotate the three-dimensional point cloud data of the cow's back before preprocessing until the ground point cloud coincides with the XOY plane.

[0017] Furthermore, the obtained three-dimensional point cloud of the cow's back is pre-processed data, and the pre-processing includes noise background removal processing; the process of the noise background removal processing is: removing the ground point cloud in the three-dimensional point cloud data of the cow's back before pre-processing, and further processing the data after removing the ground point cloud using a clustering method.

[0018] Furthermore, the obtained three-dimensional point cloud of the cow's back is pre-processed data, and the pre-processing includes unified direction processing; the process of the unified direction processing is: extracting the spine line in the three-dimensional point cloud data of the cow's back before pre-processing, and rotating the three-dimensional point cloud data of the cow's back before pre-processing so that the spine line is parallel to the set X-axis, and the X-axis is a direction extending along the horizontal plane.

[0019] Furthermore, the method of obtaining the three-dimensional point cloud of the back of the cow to be detected is: obtaining a top-view back depth image and a color image of the cow to be detected, and generating a three-dimensional point cloud of the back of the cow to be detected after registration.

[0020] The beneficial effects are as follows: the present invention is an improved invention, and the cow identification method of the present invention is a method for identifying cows based on the three-dimensional structural features and body spot signs of the cow's back. First, based on the characteristics of the three-dimensional skeletal structure of the cow, the three-dimensional structural features of the cow are used to perform rough and then fine positioning of the reference points including the horn bone and the hip bone. Specifically, rough positioning is performed first to find the hip bone area and the horn bone area, and then the curvature characteristics of the hip bone area of ​​the horn bone area are analyzed to accurately locate the horn bone point and the hip bone point; then, based on the precise positioning results of the horn bone point and the hip bone point, a key area is determined that can extract sufficient body spot features of the cow. Compared with the method of locating the cow's trunk based only on two-dimensional images in the prior art, this method can obtain a more accurate trunk. The point cloud projection of the key area is then processed to obtain a two-dimensional body spot image, which is used to identify the individual identity of the cow to be detected. The three-dimensional structural features of the cow's back are added on the basis of the body spot features, and the three-dimensional structural features of the cow's back and the body spot features are combined to realize the individual identity identification of the cow, which effectively reduces the influence of the surrounding environment on identity identification and improves the accuracy of individual identity identification of the cow. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is the overall technical roadmap of the cow identification method based on reference point detection of the present invention;

[0022] Figure 2 is a schematic diagram of a data acquisition device of the present invention;

[0023] Figure 3 is a flow chart of point cloud data preprocessing of the present invention;

[0024] Figure 4 It is a flow chart of determining key areas of the present invention. DETAILED DESCRIPTION

[0025] The main idea of ​​the present invention is to first accurately locate the reference points including the lumbar angle bone point and the hip angle bone point based on the three-dimensional structural characteristics of the cow by using a rough-first-then-fine method, and then determine a key area that can extract sufficient body spot features of the cow based on the accurate positioning results of the lumbar angle bone point and the hip angle bone point, and then process the point cloud projection of the key area to obtain a two-dimensional body spot image, and use the two-dimensional body spot image to perform individual identification of the cow to be detected. Thereby, the three-dimensional structural features and body spot features of the cow's back are combined to perform individual identification of the cow, effectively reducing the impact of the surrounding environment on identification and improving the accuracy of individual identification of the cow. In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0026] An embodiment of a method for identifying a dairy cow based on reference point detection according to the present invention has an overall technical route as follows: Figure 1 As shown, the details are as follows:

[0027] Step 1: obtain the top view back depth image and color image of the detected cow, and register them to generate a three-dimensional point cloud of the cow's back.

[0028] In this embodiment, the data collected are from lactating Holstein cows in the feeding process. The Intel Real Sense D455 depth camera is mounted on the cow's back with a camera stand, 2.9 meters above the ground. Figure 2 The depth camera simultaneously obtains the color image and depth image of the cow's top view back. The resolution of the cow's top view image is 1280 pixels × 720 pixels. In order to analyze the three-dimensional features of the cow's back, the depth image and color image collected by the Intel RealSense D455 depth camera are registered to generate a three-dimensional point cloud image of the cow's top view back.

[0029] Step 2: preprocess the obtained three-dimensional point cloud of the cow's back.

[0030] The specific preprocessing can be selected according to the actual situation, such as posture normalization, noise background removal, and unified direction processing. In this embodiment, the preprocessing includes posture normalization, noise background removal, and unified direction processing. The overall process is as follows: Figure 3 shown.

[0031] 1) In order to facilitate the subsequent unified processing of the cow point cloud, the cow point cloud should be subjected to posture normalization. Posture normalization refers to using the ground in the background point cloud as a reference, translating and rotating the cow point cloud until the ground point cloud coincides with the set XOY plane. The specific process is: ① First, perform ground fitting using methods such as the least squares method. Specifically, substitute the x, y, and z coordinates of all points in the point cloud into the plane equation, use the least squares method to find the optimal plane parameters A, B, C, and D, and calculate the plane equation of the fitting plane to be Ax+By+Cz+D=0. ② Rotate the fitting plane to coincide with the XOY plane. Specifically, extract the normal vector in the fitting plane. and the normal vector of the XOY plane The Rodrigues formula is used to calculate a rotation matrix, which is used to align the normal vector of the plane to the positive direction of the Z axis (that is, the normal vector of the XOY plane). The core of the Rodrigues formula is to rotate the rotation axis And the rotation angle θ is converted into a rotation matrix, for the vector and the target vector For rotations between , use the cross product to calculate the rotation axis and the dot product to calculate the cosine of the rotation angle.

[0032]

[0033] Where R is the rotation matrix, I is the 3×3 identity matrix, and θ is the angle of rotation, which is calculated from the angle between two vectors (calculated using the dot product and the vector modulus). is the rotation axis, which is the normalized cross product of the two vectors. yes The antisymmetric matrix of (also called the cross product matrix or antisymmetric matrix) is defined as:

[0034]

[0035] where v x , v y , v z is a vector The various quantities.

[0036] 2) Since the images of cows eating were taken in a dairy farming scene, the environment is complex. In order to eliminate the impact of the environment, the background and noise of the collected data need to be removed. In addition, the cows next to it will have an impact, so the cow trunk needs to be extracted. The ground point cloud can be deleted by setting the z value (this processing takes into account the small z value of the ground point cloud). The remaining part after deletion uses a clustering method to perform cluster analysis on the cow point cloud. The clustering method here can specifically select the DBSCAN (Density-Based Spatial Clustering Of Applications with Noise) clustering method.

[0037] 3) In order to facilitate subsequent processing, the cow's point cloud needs to be rotated at a certain angle so that the spine line is parallel to the X-axis. According to the three-dimensional structural characteristics of the cow, for a column, find the maximum z value (height value, depth value in this embodiment) of all points in the column; then perform linear fitting on the maximum z values ​​of all columns to obtain the spine fitting line. Among them, the direction of the column refers to the column in the point cloud image, in which the cow's head is roughly facing the positive direction of the X-axis, and the cow's tail is roughly facing the negative direction of the X-axis. In order to easily distinguish the left and right waist angle bone points and hip angle bone points later, the cow point cloud is rotated with this spine line as the reference until the spine line is parallel to the X-axis, and translated to coincide with the X-axis.

[0038] Step three: Use the three-dimensional segmentation model to segment the lumbar angle bone area and the hip angle bone area.

[0039] The three-dimensional segmentation model used in this step is a trained PointNet++ segmentation model to achieve the rough positioning of the lumbar angle bone and hip angle bone points. Among them, a large number of data sets are obtained in the same way as steps 1 and 2. In this embodiment, 7,600 color images and corresponding 7,600 depth images of 40 cows are collected to obtain 7,600 cow point cloud data. The training set, validation set and test set are constructed in a ratio of 7:2:1, and the data set is annotated, and then the annotated data set is used to train, verify and test the PointNet++ segmentation model.

[0040] Of course, other 3D segmentation models in the prior art, such as PointNet, MVSNet, etc., can also be used.

[0041] Step 4: Accurately locate the waist angle bone point and hip angle bone point (i.e., the reference point) based on their three-dimensional characteristics.

[0042] Because the lumbar angle bone point and the hip angle bone point are located in the convex part of the hip angle bone and the lumbar angle bone area, some parts of the convex surface have a large principal curvature, and the edge of the convexity may have a smaller average curvature. Therefore, the curvature features of the three-dimensional area of ​​the extracted hip angle bone and the lumbar angle bone area are used to accurately locate the lumbar angle bone point and the hip angle bone point, that is, the points with the largest curvature in the hip angle bone and the lumbar angle bone area correspond to the hip angle bone point and the lumbar angle bone point.

[0043] In this embodiment, the curvature is Gaussian curvature. The specific process for calculating Gaussian curvature is as follows: ① First determine the local neighborhood of the point. For a given point p i , we need to find the point set N(p i ), these neighborhood points are used to approximate the local surface around the point. ② Fitting the local surface, specifically, principal component analysis or other methods can be used to fit a local surface (such as a quadratic surface), and the fitted surface can be written as:

[0044] z=ax 2 +by 2 +cxy+dx+ey+f (3)

[0045] Where z is the value in the normal direction, x and y are the coordinates in the local tangent plane, and a, b, c, d, and e represent plane parameters.

[0046] ③ Calculate the principal curvature. Based on the fitted quadratic surface equation, the principal curvature can be obtained from the second-order derivative through the differential geometry formula. The formula for the principal curvature is:

[0047]

[0048] Where λ1, λ2, λ3 are the second-order derivatives of the local surface in different directions.

[0049] ④ Calculate Gaussian curvature. Once the two principal curvatures k1 and k2 are obtained, the Gaussian curvature K can be obtained by the following formula:

[0050] K=k1·k2 (5)

[0051] Step 5: Under the precise positioning of the lumbar angle bone point and the hip angle bone point, determine the key area, which contains the lumbar angle bone point and the hip angle bone point and can extract enough body spot features of the cow. The specific process is as follows Figure 4 As shown, the details are as follows:

[0052] After accurately locating the lumbar angle bone points and hip angle bone points, the three-dimensional point cloud is first projected on the XOY plane. Then, because the edge of the point cloud is irregular, in order to facilitate feature extraction and recognition, but to ensure that the body spot features of the extracted area are sufficient, the key area is divided according to the hip angle bone points and the lumbar angle bone points. The key area is divided as follows: connect the lumbar angle bone points and the hip angle bone points in sequence, where the length of the line connecting the lumbar angle bone points is the waist width, and on the basis of the area enclosed by the hip angle bone points and the lumbar angle bone points, extend the square area of ​​the length of the waist width from the two lumbar angle bone points along the positive direction of the X-axis. The sum of the extended area and the area enclosed by the original hip angle bone points and the lumbar angle bone points is the determined key area. Among them, in order to facilitate the subsequent body spot processing, the adjacent point interpolation filling is used to generate new points to fill the blank area, and the key area becomes a rectangular area.

[0053] Step 6: Convert the key area from a 3D point cloud to a 2D volume spot image (RGB image), and perform binarization on the 2D volume spot image.

[0054] Specifically, in this step, the bicubic interpolation method is first used to perform size normalization processing on the two-dimensional body spot image of the cow, and then the Otsu method is used to perform binarization processing.

[0055] Step seven, input the binarized two-dimensional body spot image of the cow to be detected into the trained classification model to obtain the individual identity recognition result of the cow to be detected.

[0056] The classification model in this embodiment chooses to use the ConvNeXt model as the main convolutional neural network architecture. While retaining the structural advantages of the classic convolutional neural network (CNN), ConvNeXt integrates the design ideas of the modern Transformer model. ConvNeXt significantly improves the representation ability and performance of the model by introducing technologies such as larger receptive field, deeper network structure and LayerNormalization on the basis of CNN. ConvNeXt uses a multi-level ConvNeXt Block, and each stage reduces the spatial dimension through Down sample (down sampling) and increases the number of channels of the feature map.

[0057] In summary, the present invention has the following characteristics: the present invention can realize accurate recognition of the identity of a cow under top-view conditions, and proposes a method for individual cow identity recognition based on PointNet++ and ConvNeXt networks, which has high robustness to the position and angle of the cow in the top-view field of view; in view of the complex cow detection environment, a method for individual cow identity recognition based on the three-dimensional structural features and body spot features of the cow's back is proposed, which increases the identity recognition features and improves the accuracy of cow identity recognition; a method for locating a cow's trunk based on a three-dimensional structure is proposed, specifically, based on the PointNet++ model to segment the lumbar angle bone and hip angle bone area, analyze the curvature characteristics, accurately locate the lumbar angle bone points and hip angle bone points, and then accurately determine the key areas, which is more accurate than the method of locating the cow's trunk based only on two-dimensional images in the prior art.

[0058] Specific implementation methods are given above, but the present invention is not limited to the described implementation methods. The basic idea of ​​the present invention lies in the above basic scheme. For ordinary technicians in this field, it does not take creative work to design various deformed models, formulas, and parameters according to the teachings of the present invention. Changes, modifications, substitutions, and variations of the implementation methods without departing from the principles and spirit of the present invention still fall within the scope of protection of the present invention.

Claims

1. A method for identifying cows based on reference point detection, characterized in that: The steps include: 1) Obtain a three-dimensional point cloud of the back of the cow to be tested, and locate two lumbar bone regions and two hip angle bone regions; select the point with the maximum curvature in each lumbar bone region as the lumbar bone point of the corresponding region, and select the point with the maximum curvature in each hip angle bone region as the hip angle bone point of the corresponding region; 2) Projecting the back three-dimensional point cloud on a horizontal plane, determining a key area in the projected data that includes the waist angle bone point and the hip angle bone point and can extract sufficient body spot features of the cow; 3) The point cloud projection of the key area is processed to obtain a two-dimensional body spot image, and the two-dimensional body spot image is used to identify the individual identity of the cow to be detected.

2. The method for identifying a dairy cow based on reference point detection according to claim 1, characterized in that: The method for selecting the point with the maximum curvature in the lumbar angle bone area / hip angle bone area is: for a given point, determine its nearest neighbor point set, and use the nearest neighbor point set to fit the local surface around the point; determine the principal curvature based on the fitted local surface, and then determine the Gaussian curvature based on the principal curvature, and the Gaussian curvature is the curvature of the given point.

3. The method for identifying a dairy cow based on reference point detection according to claim 1, characterized in that: The key areas are determined by: Determine the waist width, which is the length of the line connecting the two hip angle bone points; extend the waist width from the two hip angle bone points in the direction opposite to the hip angle bone, thereby obtaining a square area with a side length equal to the waist width; The area enclosed by the two waist angle bone points and the two hip angle bone points and the square area are combined as the key area.

4. The method for identifying a dairy cow based on reference point detection according to claim 1, characterized in that: The method of locating the lumbar bone region and the hip bone region is: inputting the three-dimensional point cloud of the back of the cow to be detected into the trained segmentation model, and segmenting two lumbar bone regions and two hip bone regions.

5. The method for identifying a dairy cow based on reference point detection according to claim 1, characterized in that: The method of using the two-dimensional body spot image to identify the individual identity of the cow to be detected is: binarizing the two-dimensional body spot image, inputting the binarized two-dimensional body spot image into the trained classification model, and obtaining the individual identity recognition result of the cow to be detected.

6. The method for identifying a dairy cow based on reference point detection according to claim 5, characterized in that: The two-dimensional volume spot image subjected to binarization processing is a two-dimensional volume spot image subjected to size normalization processing by using an interpolation method.

7. The method for identifying a dairy cow based on reference point detection according to any one of claims 1 to 6, characterized in that: The obtained three-dimensional point cloud of the cow's back is pre-processed data, and the pre-processing includes posture normalization processing; The process of posture normalization processing is: extracting the ground point cloud in the three-dimensional point cloud data of the cow's back before preprocessing, performing plane fitting to obtain a fitting plane, and extracting the normal vector of the fitting plane; determining the normal vector of the set XOY plane; The Rodrigues formula is used to calculate a rotation matrix for transforming the normal vector of the fitting plane to the normal vector of the XOY plane, and the rotation matrix is ​​used to rotate the three-dimensional point cloud data of the cow's back before preprocessing until the ground point cloud coincides with the XOY plane.

8. The method for identifying a dairy cow based on reference point detection according to any one of claims 1 to 6, characterized in that: The obtained three-dimensional point cloud of the cow's back is pre-processed data, and the pre-processing includes noise background removal processing; The process of the noise background removal processing is: removing the ground point cloud in the three-dimensional point cloud data of the cow's back before preprocessing, and further processing the data after removing the ground point cloud by using a clustering method.

9. The method for identifying a dairy cow based on reference point detection according to any one of claims 1 to 6, characterized in that: The obtained three-dimensional point cloud of the cow's back is pre-processed data, and the pre-processing includes unified direction processing; The unified direction processing process is: extracting the spine line in the three-dimensional point cloud data of the cow's back before preprocessing, rotating the three-dimensional point cloud data of the cow's back before preprocessing so that the spine line is parallel to the set X-axis, and the X-axis is the direction extending along the horizontal plane.

10. The method for identifying a dairy cow based on reference point detection according to any one of claims 1 to 6, characterized in that: The method of obtaining the three-dimensional point cloud of the back of the cow to be detected is: obtaining the top-view back depth image and the color image of the cow to be detected, and generating the three-dimensional point cloud of the back of the cow to be detected after registration.

Images