Pig physical appearance scoring method and device

Abstract

The invention provides a pig physical appearance scoring method and device, and the method comprises the steps: carrying out the filtering, registration and segmentation of the point cloud data of a to-be-detected pig, and obtaining the boundary segmentation data and body size data of a first part of the to-be-detected pig; processing the image data based on a target detection model to obtain an identification result of the second part of the to-be-detected pig; scoring the first part according to the boundary segmentation data, and scoring the body length and the body height of the to-be-measured pig according to the body size data to obtain a first scoring result; scoring the second part according to the identification result to obtain a second scoring result; and obtaining a target scoring result according to the first scoring result and the second scoring result. According to the method, the cloud processing technology is combined, multi-dimensional physical features with higher characterization capability are extracted based on a multi-head self-attention and deformable network target detection model, and the accuracy of pig physical feature evaluation is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for scoring pig physical appearance. Background Art

[0002] At present, the evaluation of pigs' body shape and appearance plays an important role in improving the appearance structure, body shape characteristics and prolonging the service life of breeding pigs, especially in improving the structure and quality of breeding pigs' limbs and hooves.

[0003] In the related technologies, traditional physical appearance scoring methods usually rely on the subjective judgment and experience of experts. Scoring a large number of pigs and irregular scoring is not only time-consuming and labor-intensive, but may also lack consistency due to differences between evaluators, making it difficult to adapt to the needs of large-scale farming. In response to this situation, computer vision-based pig physical appearance scoring technology can obtain richer pig information, but the existing technology only uses the pig's body size characteristics (body length and height) and other angles to describe the appearance of the pig, and is unable to mine more effective physical appearance information, resulting in limited feature expression capabilities, which in turn affects the physical appearance evaluation results. Summary of the invention

[0004] The present invention provides a pig appearance scoring method and device, which are used to solve the defect that the existing technology relies on manual experience or the limited representation ability of selected body size characteristics when evaluating the pig appearance, resulting in inaccurate pig appearance scoring results, and improves the accuracy of pig appearance evaluation.

[0005] The present invention provides a method for scoring pig physical appearance, comprising: Obtaining point cloud data and image data of the pig to be tested; The point cloud data is filtered, registered and segmented to obtain boundary segmentation data and body size data of a first part of the pig to be tested; the image data is processed based on a target detection model to obtain a recognition result of a second part of the pig to be tested; wherein the target detection model is obtained by training a deformable convolutional neural network based on sample images in combination with a multi-head self-attention mechanism; the first part includes at least one of the back, abdomen and limbs of the pig to be tested, the body size data includes at least one of the body length and height of the pig to be tested, and the second part includes at least one of the leg buttocks, coat color, hoof and nipple of the pig to be tested; The first part is scored according to the boundary segmentation data, and the body length and height of the pig to be tested are scored according to the body size data to obtain a first scoring result; the second part is scored according to the recognition result to obtain a second scoring result; and a target scoring result is obtained according to the first scoring result and the second scoring result.

[0006] According to a pig appearance scoring method provided by the present invention, the deformable convolutional neural network includes a deformable convolutional layer and a backbone network based on a multi-head self-attention mechanism; The target detection model is obtained through the following steps: Extracting multiple part features and feature offsets from the sample image based on a deformable convolution layer, and adjusting the multiple part features according to the feature offsets to obtain target features; the sample images include coat color sample images, hoof sample images, nipple sample images, and leg and hip sample images; The backbone network based on the multi-head self-attention mechanism encodes the target features corresponding to each sample image to obtain multiple encoded features; The multiple encoded features are fused to obtain fused features, and the deformable convolutional neural network is iteratively trained based on the fused features and using a bounding box regression loss and a non-maximum suppression algorithm to obtain the target detection model.

[0007] According to a pig appearance scoring method provided by the present invention, the filtering, registration and segmentation of the point cloud data to obtain the boundary segmentation data and body size data of the first part of the pig to be tested includes: Using a domain curvature change method to filter out outliers from the point cloud data to obtain filtered data; Based on the affine invariance of the point pairs, the filtered data is aligned with the point cloud in the parallel direction, and based on the Super-4PCS algorithm, the filtered data is aligned with the point cloud in the vertical direction to obtain the point cloud of the target pig; The target pig point cloud is segmented using the PointNet++ model to obtain the boundary segmentation data and the body size data; wherein the PointNet++ model is based on back-propagation training of the neural network using sample pig point cloud images as training samples, encoding features as training features, and cross entropy loss as training loss; the encoding features are sampled on the sample pig point cloud data based on a multi-layer octree architecture, and the sampled features are returned to the feature point set for determination through an inverse distance weighted IDW interpolation algorithm.

[0008] According to a pig body appearance scoring method provided by the present invention, the body size data includes the ear base point coordinates, tail base point coordinates and forebody point cloud coordinates of the pig to be tested; The first part is scored according to the boundary segmentation data, and the body length and height of the pig to be tested are scored according to the body size data, and the first scoring result is obtained, including: Perform curve fitting on the boundary segmentation data to obtain positive and negative curvature change data, and determine the scoring result of the first part according to the positive and negative curvature change data; calculate the body length of the pig to be tested according to the ear base point coordinates and the tail base point coordinates, score the body length according to the body length threshold, and obtain the body length scoring result; determine the body height of the pig to be tested according to the ordinate values ​​of all point clouds in the precursor point cloud coordinates, score the body height according to the body height threshold, and obtain the body height scoring result.

[0009] According to a pig appearance scoring method provided by the present invention, the step of obtaining point cloud data and image data of the pig to be tested comprises: The point cloud data is collected by a depth camera, and the image data is collected by a color camera. The depth camera and the color camera both adjust the camera posture through a three-way relative position adjustment structure; the point cloud data and the image data are associated with IDs corresponding to different pigs.

[0010] According to a pig physical appearance scoring method provided by the present invention, obtaining a target scoring result according to the first scoring result and the second scoring result comprises: The scores corresponding to the various indicators in the first scoring result and the second scoring result are weighted and summed to obtain the target scoring result.

[0011] The present invention also provides a pig body appearance scoring device, comprising: A data acquisition module, used to acquire point cloud data and image data of the pig to be tested; A segmentation and detection module, used for filtering, registering and segmenting the point cloud data to obtain boundary segmentation data and body size data of a first part of the pig to be tested; processing the image data based on a target detection model to obtain a recognition result of a second part of the pig to be tested; wherein the target detection model is obtained by training a deformable convolutional neural network based on sample images in combination with a multi-head self-attention mechanism; the first part includes at least one of the back, abdomen and limbs of the pig to be tested, the body size data includes at least one of the body length and height of the pig to be tested, and the second part includes at least one of the leg buttocks, coat color, hoof and nipple of the pig to be tested; A scoring module is used to score the first part according to the boundary segmentation data, score the body length and height of the pig to be tested according to the body size data, and obtain a first scoring result; score the second part according to the recognition result to obtain a second scoring result; and obtain a target scoring result according to the first scoring result and the second scoring result.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned pig appearance scoring methods is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for scoring pig physical appearance as described above is implemented.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned pig physical appearance scoring methods.

[0015] The pig appearance scoring method and device provided by the present invention obtain boundary segmentation data and body size data of a first part by filtering, registering and segmenting point cloud data of the pig to be tested; process the image data based on a target detection model to obtain a recognition result of a second part; obtain a first scoring result based on the boundary segmentation data and the body size data; obtain a second scoring result based on the recognition result, and finally obtain a target scoring result based on the first scoring result and the second scoring result. The method of the present invention combines cloud processing technology and a target detection model based on multi-head self-attention and a deformable network to propose a physical appearance feature with stronger representation ability, thereby improving the accuracy of pig appearance evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is one of the flow charts of the pig physical appearance scoring method provided by the present invention.

[0018] Figure 2 It is a schematic diagram of the principle of the multi-head attention mechanism provided by the present invention.

[0019] Figure 3 It is a structural diagram of the improved PointNet++ network structure provided by the present invention.

[0020] Figure 4 It is a schematic diagram of the structure of the high-throughput stress-free collection device for pig body appearance provided by the present invention.

[0021] Figure 5 This is the second flow chart of the pig body appearance scoring method provided by the present invention.

[0022] Figure 6 It is a structural schematic diagram of the pig body appearance scoring device provided by the present invention.

[0023] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] Combine the following Figure 1-Figure 6 The invention describes a method and device for scoring pig physical appearance.

[0026] Figure 1 FIG. 1 is one of the flow charts of the pig physical appearance scoring method provided by the present invention, such as Figure 1 As shown, the pig appearance scoring method comprises the following steps: Step 110: Obtain point cloud data and image data of the pig to be tested.

[0027] In this step, the pigs can be either female or male.

[0028] In this embodiment, the point cloud data and image data of the above-mentioned pigs to be tested can be collected by a high-throughput non-stress collection device for pig body appearance, and can also be obtained from a pig sample image library.

[0029] In this embodiment, the point cloud data may be collected by a depth camera, and the image data may be obtained by photographing a color camera.

[0030] Step 120, filtering, registering and segmenting the point cloud data to obtain boundary segmentation data and body size data of a first part of the pig to be tested; processing the image data based on the target detection model to obtain a recognition result of a second part of the pig to be tested; wherein the target detection model is obtained by training a deformable convolutional neural network based on sample images in combination with a multi-head self-attention mechanism; the first part includes at least one of the back, abdomen and limbs of the pig to be tested, the body size data includes at least one of the body length and height of the pig to be tested, and the second part includes at least one of the leg buttocks, coat color, hoof and nipple of the pig to be tested.

[0031] In this step, the physical scoring parameters of live pigs can be divided into two categories: physical characteristics of breeding pigs and physical characteristics of sows; these two types of physical characteristics are further subdivided into 9 types of quantitative physical scoring parameters, namely: body length (BL), body height (BH), back waist (BA), leg and hip (LH), abdomen (AB), hoof (SW), number of nipples (NN), limbs (LI), and coat color (CL).

[0032] In this step, the algorithms for filtering the point cloud data include but are not limited to the field curvature change method, etc.; the algorithms for point cloud registration include but are not limited to the step-by-step pig point cloud registration method, etc.; the segmentation algorithms include but are not limited to the segmentation algorithms based on neural networks and the segmentation algorithms based on machine learning.

[0033] In this step, an improved deep learning model (Yolov8) can be used to achieve physical scoring of the second part of the pig (including coat color CL, leg and hip LH, hoof SW, and number of nipples NN).

[0034] Specifically, the deformable convolutional neural network includes a deformable convolutional layer and a backbone network based on a multi-head self-attention mechanism; the target detection model is obtained through the following steps: extracting multiple part features and feature offsets from a sample image based on the deformable convolutional layer, and adjusting the multiple part features according to the feature offsets to obtain target features; the sample images include coat color sample images, hoof sample images, nipple sample images, and leg and hip sample images; the backbone network based on the multi-head self-attention mechanism encodes the target features corresponding to each sample image to obtain multiple encoded features; the multiple encoded features are fused to obtain fused features, and the deformable convolutional neural network is iteratively trained according to the fused features and using a bounding box regression loss and a non-maximum suppression algorithm to obtain a target detection model.

[0035] In this embodiment, the deformable convolutional neural network is obtained specifically through the following steps: (1) Divide the training set and the test set; obtain from the sample images four side view angles, front top view angle, and oblique rear top view angle images of multiple (e.g., 20) pigs (e.g., 1820 images for each view angle); combine the physical appearance scoring standard, use LabelImg software to label the tail, top, and hoof areas of the pigs, classify the coat color (1-5 points), leg and hip (1-5 points), and hoof (1-5 points), and label the nipples. The five categories of the sample category data set are randomly divided into the training set and the test set at a ratio of 8:2, wherein the training set samples are 1456 and the test set samples are 364.

[0036] (2) Model training: An improved deep learning model (Yolov8) was used to realize the physical appearance scoring of pig coat color (CL), leg and hip (LH), hoof (SW), and number of teats (NN).

[0037] In this embodiment, the improved Yolov8 model includes the following four network structures: 1) Input layer: The labeled images of coat color, hoof, nipple, leg and hip are input into the model for the next step of training.

[0038] 2) Backbone network layer: It uses a variety of feature extraction modules to extract features of pig hair color, leg buttocks, hooves, and teats and generate semantic information of the target.

[0039] Specifically, in order to further improve the model's feature extraction of live pigs, this embodiment adopts a more adaptable and generalizable DCv2 structure to improve the traditional convolution kernel of YOLOv8; at the same time, the multi-head self-attention (MHSA) module is integrated into the backbone network (Backbone), which can improve the expressive ability of the model and expand the receptive field of the backbone network, thereby enhancing the model's ability to obtain information on live pigs' hair color, hooves, teats, and legs and buttocks.

[0040] In this embodiment, the working principle of the DCv2 structure is as follows: first, a feature map is extracted from the input image using a standard convolution kernel; then, the obtained feature map is used as the input of another convolution layer, which generates an offset field responsible for DC deformation. The dimension of the offset field is 2N, and the translation on the plane is achieved by modifying the values ​​in the X and Y directions; in the training stage, the convolution kernel that generates the output feature and the convolution kernel that generates the offset are learned, and the offset output by the convolution layer is learned through back propagation and interpolation algorithms. The corresponding calculation formula is as follows: ; in, is the learned offset; is the learned weight, is the original position, is the reference position, Used to define the base and target of the transformation, for The corresponding weight.

[0041] exist Figure 2 In the embodiment shown, the working principle of the multi-head self-attention module is as follows: 1. The input data sequence X is encoded into an input matrix [X1, X2, ..., X n ], and then use three trainable parameter matrices W Q ,WK and W V Perform linear transformation to obtain three new matrices, namely Query (Q), Key (K) and Key Value (V); 2. Calculate the dot product between the query and the corresponding key value, normalize it, and multiply it with the matrix V to obtain the weighted sum; the calculation formula for this process is: ; in, Used to prevent the gradient of the result from disappearing, Represents the size associated with the dimension of the matrix K.

[0042] 3) Neck feature fusion layer: reduces the network depth while enhancing the fusion capability of the pig’s tail and top fur color feature information.

[0043] 4) Precision output layer: uses bounding box regression loss and non-maximum suppression algorithm to improve the model's recognition accuracy of coat color, pig tail and hoof; Figure 3 In the figure, h1, h2 and h3 are concatenated to achieve feature fusion and output through the connection layer.

[0044] Step 130, score the first part according to the boundary segmentation data, score the body length and height of the pig to be tested according to the body size data, and obtain a first scoring result; score the second part according to the recognition result to obtain a second scoring result; obtain a target scoring result according to the first scoring result and the second scoring result.

[0045] In this step, the AC point cloud boundary segmentation algorithm is used to retain the boundary points of the middle body point cloud, front leg point cloud and hind leg point cloud respectively, and then the slopes of the upper body point cloud curve, the lower body point cloud curve and the front and rear leg point cloud curve are calculated, and then combined with the pig physical appearance scoring standard, the back waist, abdomen and limbs are scored respectively; in addition, according to the ear base points and tail base points of the two ears of the pig to be tested in the body size data, the length of the pig's spine is calculated with these as endpoints to obtain the body length, and the body height of the pig to be tested is determined according to the ordinate values ​​of all point clouds in the forebody point cloud coordinates, and then the corresponding first scoring result is obtained.

[0046] In this step, the second part is identified by the deformable convolutional neural network to obtain the scores of coat color, leg and hoof, i.e., the second scoring result; wherein, the number of nipples detected in the nipple area is counted by the convolutional neural network, and finally scored according to the physical appearance scoring standard.

[0047] Furthermore, obtaining a target scoring result according to the first scoring result and the second scoring result includes: performing a weighted sum calculation on the scores corresponding to each indicator in the first scoring result and the second scoring result and the indicator weight to obtain the target scoring result.

[0048] In this embodiment, the above-mentioned scoring results are summarized, and with reference to the pig physical appearance scoring standard, the final pig physical appearance score, i.e., the target scoring result, is obtained according to the following formula: Total score = ∑ (score of each trait * indicator weight); Among them, the scores of each trait include the body length (BL), body height (BH), back waist (BA), abdomen (AB), limbs (LI), hoof (SW), number of nipples (NN), coat color (CL) and leg and hip (LH) of the tested pigs; the indicator weights can be set according to user needs, and different trait scores can correspond to the same or different indicator weights; the sum of the indicator weights is 1.

[0049] The pig appearance scoring method provided by the embodiment of the present invention obtains boundary segmentation data and body size data of a first part by filtering, aligning and segmenting point cloud data of the pig to be tested; processes the image data based on a target detection model to obtain a recognition result of a second part; obtains a first scoring result based on the boundary segmentation data and the body size data; obtains a second scoring result based on the recognition result, and finally obtains a target scoring result based on the first scoring result and the second scoring result. The method of the present invention combines cloud processing technology and a target detection model based on multi-head self-attention and a deformable network to extract multidimensional appearance features with stronger representation capabilities, thereby improving the accuracy of pig appearance evaluation.

[0050] Furthermore, the point cloud data is filtered, registered and segmented to obtain the boundary segmentation data and body size data of the first part of the pig to be tested, including: filtering out outliers from the point cloud data using the domain curvature change method to obtain filtered data; performing point cloud registration in parallel directions on the filtered data based on the affine invariance of point pairs, and performing point cloud registration in vertical directions on the filtered data based on the Super-4PCS algorithm to obtain the point cloud of the target pig; using the PointNet++ model to segment the point cloud of the target pig to obtain boundary segmentation data and body size data; wherein the PointNet++ model is based on back propagation training of the neural network using sample pig point cloud images as training samples, encoding features as training features, and cross entropy loss as training loss; the encoding features are sampled on the sample pig point cloud data based on a multi-layer octree architecture, and the sampled features are returned to the feature point set for determination through the inverse distance weighted IDW interpolation algorithm.

[0051] It should be noted that during the data collection process, dust, hair and external environmental factors will cause interference. These interferences can be effectively removed by applying filtering algorithms, thereby reducing the impact of noise points on point cloud processing.

[0052] In this embodiment, point cloud filtering is specifically implemented through the following steps: (1) Use the domain curvature change method to filter out outliers; for example, use the distance query region growth method to filter out distant outliers, and use the SVLOF algorithm to eliminate close outliers; specifically, let the surface change degree of the metric index in the pig point cloud be θ K , suppose that for any point p on the three-dimensional point cloud, it does not contain the k-neighborhood NB of point p k The collection is defined as: ; Including p point k area Defined as: ; Use the outlier point curvature method to calculate the average curvature of the point ; The specific calculation formula is: ; in, , , is the curvature of the three k-neighborhood points with the largest changes, That is, the curvature change of the k-domain points; the judgment value of the near outlier point is calculated as the ratio of the curvature change of the k-domain points containing the point to the curvature change of the k-domain points excluding the point The specific calculation formula is:

[0053] ; Among them, calculate each point in the potential outlier value, and Points with values ​​greater than the set tolerance are considered outliers and filtered out from the point cloud; then the random machine sampling consensus algorithm (RANSAC) is used to identify the ground plane point cloud and check the geometric relationship between all points to obtain all points belonging to the ground plane.

[0054] (2) Randomly select N points in the point cloud as tuple G: ; A plane P is determined based on the selected sample image. All points in the point cloud whose distance to plane P is less than 20 mm are added to plane P. If the vertical coordinate span of the plane point cloud exceeds 60 mm, go back to the first step and re-execute. If the number of iterations is greater than 2000, the extraction ends.

[0055] In this embodiment, point cloud registration is specifically implemented through the following steps: This embodiment adopts a step-by-step live pig point cloud registration method (for example, based on the Super-4PCS algorithm) to gradually register the point clouds of 7 perspectives to obtain a complete live pig point cloud, that is, the target pig point cloud; specifically, first perform same-side point cloud registration, and use the affine invariance of point pairs to register the point clouds taken by the top three cameras. At the same time, the point clouds taken by the four cameras on both sides are also registered; then use the super-4PCS algorithm to complete the point cloud registration in the vertical direction for the remaining points.

[0056] In this embodiment, after obtaining the complete point cloud of the live pig, in order to obtain the data of each body length (BL), body height (BH), back waist (BA), abdomen (AB), and limb (LI) parameters, this patent uses an improved PointNet++ method to divide the complete point cloud of the live pig into 6 different regions: head, front legs, hind legs, forebody, midbody, and hind body.

[0057] Figure 3 is a schematic diagram of the structure of the improved PointNet++ network structure provided by the present invention. Figure 3 In the embodiment shown, point cloud segmentation is specifically implemented through the following steps: (1) Input the preprocessed single complete pig point cloud data with a dimension of [N×npoint×3], where N represents the number of pig point clouds input in batches. Each point cloud contains more than 90,000 points (npoint), and the coordinates of each point are (X, Y, Z).

[0058] (2) The octree is used to extract feature points, sample and group the pig body point cloud. The octree has five layers. Let N1 be the number of sampling center points, and the number of neighborhood points of each sampling center point (nsample) is 64.

[0059] (3) Encode the local point cloud through PointNet++ and convert the sampled grouped point cloud data into a feature vector with a dimension of [N×1024×256].

[0060] (4) Repeat steps 2 and 3, using a four-layer octree architecture, setting n2 as the number of points near the center point, and the dimension of the encoded feature vector is [N×256×512].

[0061] (5) Use PointNet++ to further encode these feature vectors; since the input feature points are sub-sampled after feature learning of the hierarchical point set, it is necessary to use the inverse distance weighted (IDW) interpolation algorithm to transfer the features of the sampled points back to the feature point set. The final data dimension is [N×4096×256].

[0062] Among them, the IDW interpolation formula is as follows: ; ; f is the interpolated eigenvalue; D (x, xi) is the distance between x and xi; P is the index of d(x, xi); k is the number of adjacent points (default p = 2, k = 3).

[0063] (6) After convolution, batch normalization, activation function, and back propagation, 6 types of feature points (head, front legs, hind legs, forequarters, mid-quarters, and hindquarters) are obtained; the maximum value is taken as the output result to complete the segmentation of the target pig body point cloud. Point cloud segmentation is a multi-classification problem for each point, so the loss function we use to train the deep network model is the cross entropy loss function. The formula is as follows: ; Among them, y i,k Indicates that the true label of sample i is k, there are k label values ​​and N samples, p i,k It represents the probability that sample i is predicted to be k label values.

[0064] The pig appearance scoring method provided by the embodiment of the present invention filters out outliers from point cloud data by filtering using the domain curvature change method to obtain filtered data; performs point cloud alignment in parallel directions on the filtered data based on the affine invariance of point pairs, and performs point cloud alignment in vertical directions on the filtered data based on the Super-4PCS algorithm to obtain a target pig point cloud; uses the PointNet++ model to segment the target pig point cloud to obtain boundary segmentation data and body size data. The present invention uses point cloud processing technology to perform in-depth analysis on the data, and can obtain pig appearance characteristics with more accurate characterization capabilities.

[0065] Furthermore, the body size data includes the coordinates of the ear base point, the tail base point and the precursor point cloud coordinates of the pig to be tested; the first part is scored according to the boundary segmentation data, and the body length and body height of the pig to be tested are scored according to the body size data, and the first scoring result includes: curve fitting the boundary segmentation data to obtain positive and negative curvature change data, and determining the scoring result of the first part according to the positive and negative curvature change data; the body length of the pig to be tested is calculated according to the ear base point coordinates and the tail base point coordinates, and the body length is scored according to the body length threshold to obtain the body length scoring result; the body height of the pig to be tested is determined according to the ordinate values ​​of all point clouds in the precursor point cloud coordinates, and the body height is scored according to the body height threshold to obtain the body height scoring result.

[0066] In this embodiment, each parameter after point cloud segmentation is scored through the following steps: 1. Score the body length and height of the pigs to be tested; Specifically, the ear roots and tail roots of the two ears in the body size data are measured, and the length of the pig's spine is calculated using these as endpoints to obtain the body length; the ear root point is the midpoint of the line connecting the center points of the local head point cloud, and the tail root point is the center point of the tail point cloud; point p1 is obtained according to the center point of the overlapping area between the slice plane formed by the center points of the front left and right legs and the trunk and the upper point cloud, point p3 is obtained according to the center point of the overlapping area between the abdominal circumference slice plane and the upper point cloud, point p5 is obtained according to the center point of the overlapping area between the slice plane formed by the center points of the back two legs and the trunk and the upper point cloud, and p2 and p4 are the center points of p1p3 (line segment) and p3p5 (line segment) respectively; the ear root point, tail root point, p1, p2, p3, p4 and p5 are fitted as the spine curve, and the body length is calculated for the spine point set to obtain the body length.

[0067] The calculation formula of body height BH can be expressed as: ; in are all z coordinates in the precursor point cloud, and Mz are the z coordinates of all point clouds; the body height is scored according to the pig appearance scoring standard.

[0068] 2. Score the first part of the pig to be tested; Specifically, the AC point cloud boundary segmentation algorithm is used to retain the boundary points of the middle body point cloud, the front leg point cloud and the hind leg point cloud respectively; wherein the method steps of the AC point cloud boundary segmentation algorithm are: (1) Let the current point be p, project its neighborhood Np onto its tangent plane, and then connect the projected neighboring points with p as the corner point in a counterclockwise order to form a series of angles , where n=||N p ||-1.

[0069] (2) Traversal The largest angle , and finally set the angle threshold k. >k, p is the boundary point, and all p boundary points that meet the conditions are retained; the point cloud of the middle body boundary points is divided into the upper body and the lower body according to the z coordinate; the point cloud of the leg boundary points is divided into the front half point cloud and the back half point cloud according to the x coordinate.

[0070] (3) Calculate the slopes of the upper drive body point cloud curve, the lower drive body point cloud curve, and the front and rear legs point cloud curve; the specific process is as follows: Any point in the point cloud has a certain surface , the neighborhood point cloud approximating the point, the curvature at a point can be characterized by the local surface curvature fitted by the point and its neighborhood points. Through least squares fitting, the local area can be characterized by a quadratic surface, and the principal curvature at each point can be calculated. , mean curvature H and Gaussian curvature K: ; ; ; Where: ; ; ; ; ; , , is the partial differential of the surface. E, F, and G are called the first basic invariants of the surface. L, M, and N are the second basic invariants of the surface. Two different basic invariants constitute a curve. Take a point in the point cloud. , and Take n points uniformly in the point cloud as the center, and use the least squares method to fit the quadratic surface through these n points ,After solving the coefficients, the normal curvature of the data points is calculated according to the properties of the space surface curve.

[0071] Next, for the m-neighborhood of any data point in the measured point cloud, according to the least squares principle (see the following formula), the following formula needs to be minimized: ; Solve for the quadratic surface coefficients and write the curve equation in the form of a surface parametric equation: ; From the first basic formula of the surface we can get: ; If P is a point on the curve r, and t and n represent the unit tangent vector and unit normal vector of point P respectively, the curvature vector can be decomposed into: ; The normal vector n can be expressed as: ; The second basic formula of the surface is: ; The normal curvature can be expressed as: ; ; (4) After the curvature is obtained, the back, waist, abdomen and limbs are scored separately according to the pig body appearance scoring standards.

[0072] Specifically, for the back and waist: after obtaining the curvature of the back and waist, determine the positive and negative changes of the curvature and score them in turn; if the curvature of the back and waist curve has one positive and negative change and changes within one unit, it is 1-2 points, otherwise it is 3 points. If the curvature changes positive and negative three times, it is 4-5 points; for the abdomen: after obtaining the curvature of the abdomen, determine whether the curvature is negative or positive, and score them in turn; if the curvature of the abdomen curve changes within 3%, it is 5 points, 3-10%, it is 3 points, and more than 10% is 1 point; for the limbs: after obtaining the curvature of the limbs, determine the positive and negative changes of the curvature and score them in turn; if the curvature of the limb curve changes within 10%, it is 5 points, 10-15%, it is 3 points, and more than 15% is 1 point.

[0073] The pig appearance scoring method provided in an embodiment of the present invention performs curve fitting on the boundary segmentation data to obtain the positive and negative curvature change data, and determines the scoring result of the first part according to the positive and negative curvature change data; calculates the body length of the pig to be tested according to the coordinates of the ear root point and the tail root point, scores the body length according to the body length threshold, and obtains the body length scoring result; determines the body height of the pig to be tested according to the ordinate values ​​of all point clouds in the precursor point cloud coordinates, scores the body height according to the body height threshold, and obtains the body height scoring result, thereby realizing automatic scoring of the pig's appearance and being able to monitor and analyze the key body parameters of the pig in real time.

[0074] Furthermore, obtaining point cloud data and image data of the pigs to be tested includes: collecting point cloud data through a depth camera, collecting image data through a color camera, and both the depth camera and the color camera adjust the camera posture through a three-way relative position adjustment structure; both the point cloud data and the image data are associated with the IDs corresponding to different pigs.

[0075] In this embodiment, a high-throughput, stress-free pig appearance acquisition device is designed to quantify appearance scoring parameters and acquisition angles, two-dimensional and three-dimensional data acquisition nodes, data integration processing nodes, data processing nodes, appearance scoring nodes and other processes.

[0076] Figure 4 The schematic diagram of the structure of the high-throughput non-stress collection device for pig body appearance provided by the present invention is shown in FIG. Figure 4 In the embodiment shown, the high-throughput stress-free collection device for pig body appearance mainly includes the following parts, and each functional module is introduced as follows:

[0077] (1) Gantry structure: The main frame of the image data acquisition device, mainly used to install various functional modules. The structure is overall stable and supports 12mm height adjustment and movement functions to ensure the stability and flexibility of the equipment to meet the needs and applications of different scenarios.

[0078] (2) Livestock guiding channel: used to guide livestock to move from one end of the collection device to the other end, preventing livestock from running around in the collection device and damaging the equipment. The channel design is conducive to the smooth passage of livestock and improves the efficiency and safety of data collection.

[0079] (3) Controller: Installed in the gantry base, used to connect all depth and color cameras to collect and process visual and related data.

[0080] (4) Touch display: used to display various data and operation information, supporting multi-touch. Users can view and operate data through the touch operation interface, achieving intuitive and convenient interaction.

[0081] (5) Depth and color cameras: used to collect depth and color image data. These cameras can provide high-precision image information, providing a basis for subsequent data processing and analysis.

[0082] (6) Universal adjustment module: It includes the camera's X, Y, and Z relative position adjustment mechanism and the camera's angle universal adjustment mechanism, which enables the camera to have omnidirectional adjustment function. This allows the camera to be better adjusted to the desired angle and more comprehensive data to be collected.

[0083] (7) Infrared trigger grating: used to detect the presence of livestock and trigger the system to collect image data. By automatically starting the image acquisition process, the acquisition system is highly automated, improving the efficiency and accuracy of data acquisition.

[0084] (8) RFID reader: used to read the RFID ear tag of livestock and obtain ID information. This enables individual identification and data association of livestock. It should be noted that due to the diversity of data sources, standard time and individual information may be out of sync, so data integration is required. By unifying the pig ear tag number as the only identifier, the individual information collected by all cameras is ensured to be consistent. All time data uses the UTC / GMT+8 standard to ensure consistency in the time dimension.

[0085] In this embodiment, a pig physical appearance scoring channel containing 7 Kinect point cloud cameras is built, and point cloud data and image data from 7 angles are collected by using the physical appearance scoring acquisition device; specific data types include: instantaneous three-dimensional point cloud data (pcd) files and RGB image (png) files from 7 viewing angles.

[0086] The pig appearance scoring method provided by the embodiment of the present invention collects point cloud data through a depth camera and collects image data through a color camera. The depth camera and the color camera both adjust the camera posture through a three-way relative position adjustment structure. The point cloud data and the image data are associated with the IDs corresponding to different pigs, thereby improving the accuracy, consistency and rapid high-throughput automatic acquisition of pig appearance scoring, realizing real-time scoring and analysis of pig appearance, and improving the efficiency and scientificity of pig appearance scoring.

[0087] Figure 5 This is the second flow chart of the pig body appearance scoring method provided by the present invention. Figure 5 In the embodiment shown, a pig appearance scoring model uses the above-mentioned pig appearance scoring method to achieve pig appearance scoring; collects two-dimensional and three-dimensional data of the pig to be tested; in the point cloud processing stage, the two-dimensional and three-dimensional data are successively subjected to SVLOF point cloud filtering, distributed Super-4PCS registration and pointnet++ point cloud part segmentation to obtain AC boundary segmentation data and body size data, and the curve curvature is obtained by fitting the curve according to the AC boundary segmentation data, and the back waist score, abdomen score and limb score are determined according to the curvature change; the body size is determined according to the body size data, and then the improvement score and body length score are obtained; in the image processing stage, a pig appearance scoring data set is constructed through sample images, and then the improved Yolov8 appearance scoring model is trained to obtain a target detection model, and the two-dimensional and three-dimensional data are input into the target detection model to output weights, and the leg and hip scores, coat color scores and question scores are determined according to the output weights; the nipple number score is determined according to the number of nipples output by the target detection model; finally, the weighted sum calculation is performed on the scores obtained in the above steps to determine the comprehensive scoring result.

[0088] The pig physical appearance scoring device provided by the present invention is described below. The pig physical appearance scoring device described below and the pig physical appearance scoring method described above can be referenced to each other.

[0089] Figure 6 Schematic diagram of the structure of the pig body appearance scoring device provided by the present invention. Figure 6 As shown, the pig physical appearance scoring device includes: a data acquisition module 610, a segmentation and detection module 620 and a scoring module 630.

[0090] The data acquisition module 610 is used to acquire the point cloud data and image data of the pig to be tested; The segmentation and detection module 620 is used to filter, register and segment the point cloud data to obtain boundary segmentation data and body size data of the first part of the pig to be tested; the image data is processed based on the target detection model to obtain the recognition result of the second part of the pig to be tested; wherein the target detection model is obtained by training a deformable convolutional neural network based on sample images combined with a multi-head self-attention mechanism; the first part includes at least one of the back, abdomen and limbs of the pig to be tested, the body size data includes at least one of the body length and height of the pig to be tested, and the second part includes at least one of the leg buttocks, coat color, hoof and nipple of the pig to be tested; The scoring module 630 is used to score the first part according to the boundary segmentation data, score the body length and height of the pig to be tested according to the body size data, and obtain a first scoring result; score the second part according to the recognition result to obtain a second scoring result; and obtain a target scoring result based on the first scoring result and the second scoring result.

[0091] In some embodiments, the pig physical appearance scoring device can be integrated into an intelligent breeding management system and combined with other identification and health monitoring technologies to provide farmers with comprehensive data support and decision-making basis; by comprehensively scoring data collected from multiple angles and referring to the scoring experience of the pig physical appearance scoring standards, this embodiment can accurately evaluate the physical appearance of pigs, optimize breeding decisions, and improve the economic benefits and competitiveness of the breeding industry.

[0092] The pig appearance scoring device provided by the embodiment of the present invention obtains boundary segmentation data and body size data of a first part by filtering, aligning and segmenting point cloud data of the pig to be tested; processes the image data based on a target detection model to obtain a recognition result of a second part; obtains a first scoring result based on the boundary segmentation data and the body size data; obtains a second scoring result based on the recognition result, and finally obtains a target scoring result based on the first scoring result and the second scoring result. The method of the present invention combines cloud processing technology and a target detection model based on multi-head self-attention and a deformable network to extract multi-dimensional appearance features with stronger representation capabilities, thereby improving the accuracy of pig appearance evaluation.

[0093] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor (processor) 710 , a communication interface (Communications Interface) 720 , a memory (memory) 730 and a communication bus 740 , wherein the processor 710 , the communication interface 720 , and the memory 730 communicate with each other through the communication bus 740 . The processor 710 can call the logic instructions in the memory 730 to execute the pig appearance scoring method, which includes: obtaining point cloud data and image data of the pig to be tested; filtering, aligning and segmenting the point cloud data to obtain boundary segmentation data and body size data of a first part of the pig to be tested; processing the image data based on the target detection model to obtain a recognition result of a second part of the pig to be tested; wherein the target detection model is obtained by training a deformable convolutional neural network based on sample images combined with a multi-head self-attention mechanism; the first part includes at least one of the back, abdomen and limbs of the pig to be tested, the body size data includes at least one of the body length and height of the pig to be tested, and the second part includes at least one of the leg buttocks, coat color, hoof and nipple of the pig to be tested; scoring the first part according to the boundary segmentation data, and scoring the body length and height of the pig to be tested according to the body size data to obtain a first scoring result; scoring the second part according to the recognition result to obtain a second scoring result; and obtaining a target scoring result according to the first scoring result and the second scoring result.

[0094] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0095] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the pig appearance scoring method provided by the above methods, which includes: obtaining point cloud data and image data of the pig to be tested; filtering, registering and segmenting the point cloud data to obtain boundary segmentation data and body size data of a first part of the pig to be tested; processing the image data based on a target detection model to obtain an identification result of a second part of the pig to be tested; wherein the target detection model The deformable convolutional neural network is trained based on sample images in combination with a multi-head self-attention mechanism; the first part includes at least one of the back, abdomen and limbs of the pig to be tested, the body size data includes at least one of the body length and height of the pig to be tested, and the second part includes at least one of the leg buttocks, coat color, hoof and nipple of the pig to be tested; the first part is scored according to the boundary segmentation data, and the body length and height of the pig to be tested are scored according to the body size data to obtain a first scoring result; the second part is scored according to the recognition result to obtain a second scoring result; and a target scoring result is obtained according to the first scoring result and the second scoring result.

[0096] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by the processor to execute the pig body appearance scoring method provided by the above methods, the method comprising: obtaining point cloud data and image data of the pig to be tested; filtering, registering and segmenting the point cloud data to obtain boundary segmentation data and body size data of the first part of the pig to be tested; processing the image data based on the target detection model to obtain the recognition result of the second part of the pig to be tested; wherein the target detection model is based on the sample image combined with the multi-head self-attention The deformable convolutional neural network is trained by the mechanism; the first part includes at least one of the back, abdomen and limbs of the pig to be tested, the body size data includes at least one of the body length and height of the pig to be tested, and the second part includes at least one of the leg buttocks, coat color, hoof and nipple of the pig to be tested; the first part is scored according to the boundary segmentation data, and the body length and height of the pig to be tested are scored according to the body size data to obtain a first scoring result; the second part is scored according to the recognition result to obtain a second scoring result; and a target scoring result is obtained according to the first scoring result and the second scoring result.

[0097] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0098] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for scoring pig physical appearance, characterized in that: include: Obtaining point cloud data and image data of the pigs to be tested; Filtering, registering and segmenting the point cloud data to obtain boundary segmentation data and body size data of the first part of the pig to be tested; The image data is processed based on a target detection model to obtain a recognition result of the second part of the pig to be tested; wherein the target detection model is obtained by training a deformable convolutional neural network based on sample images in combination with a multi-head self-attention mechanism; the first part includes at least one of the back, abdomen and limbs of the pig to be tested, the body size data includes at least one of the body length and height of the pig to be tested, and the second part includes at least one of the leg buttocks, coat color, hoof and nipple of the pig to be tested; The first part is scored according to the boundary segmentation data, and the body length and height of the pig to be tested are scored according to the body size data to obtain a first scoring result; the second part is scored according to the recognition result to obtain a second scoring result; and a target scoring result is obtained according to the first scoring result and the second scoring result.

2. The pig body appearance scoring method according to claim 1, characterized in that: The deformable convolutional neural network includes a deformable convolutional layer and a backbone network based on a multi-head self-attention mechanism; The target detection model is obtained through the following steps: Extracting multiple part features and feature offsets from the sample image based on a deformable convolution layer, and adjusting the multiple part features according to the feature offsets to obtain target features; the sample images include coat color sample images, hoof sample images, nipple sample images, and leg and hip sample images; The backbone network based on the multi-head self-attention mechanism encodes the target features corresponding to each sample image to obtain multiple encoded features; The multiple encoded features are fused to obtain fused features, and the deformable convolutional neural network is iteratively trained based on the fused features and using a bounding box regression loss and a non-maximum suppression algorithm to obtain the target detection model.

3. The pig physical appearance scoring method according to claim 1, characterized in that: The filtering, registration and segmentation processing of the point cloud data to obtain the boundary segmentation data and body size data of the first part of the pig to be tested includes: Using a field curvature change method to filter out outliers from the point cloud data to obtain filtered data; Based on the affine invariance of the point pairs, the filtered data is aligned with the point cloud in the parallel direction, and based on the Super-4PCS algorithm, the filtered data is aligned with the point cloud in the vertical direction to obtain the point cloud of the target pig; The target pig point cloud is segmented using the PointNet++ model to obtain the boundary segmentation data and the body size data; wherein the PointNet++ model is based on back-propagation training of the neural network using sample pig point cloud images as training samples, encoding features as training features, and cross entropy loss as training loss; the encoding features are sampled on the sample pig point cloud data based on a multi-layer octree architecture, and the sampled features are returned to the feature point set for determination through an inverse distance weighted IDW interpolation algorithm.

4. The pig physical appearance scoring method according to claim 1, characterized in that: The body size data includes the ear base point coordinates, tail base point coordinates and forebody point cloud coordinates of the pig to be tested; The first part is scored according to the boundary segmentation data, and the body length and height of the pig to be tested are scored according to the body size data, and the first scoring result is obtained, including: Perform curve fitting on the boundary segmentation data to obtain positive and negative curvature change data, and determine the scoring result of the first part according to the positive and negative curvature change data; calculate the body length of the pig to be tested according to the ear base point coordinates and the tail base point coordinates, score the body length according to the body length threshold, and obtain the body length scoring result; determine the body height of the pig to be tested according to the ordinate values ​​of all point clouds in the precursor point cloud coordinates, score the body height according to the body height threshold, and obtain the body height scoring result.

5. The pig physical appearance scoring method according to claim 1, characterized in that: The step of obtaining the point cloud data and image data of the pig to be tested includes: The point cloud data is collected by a depth camera, and the image data is collected by a color camera. The depth camera and the color camera both adjust the camera posture through a three-way relative position adjustment structure; the point cloud data and the image data are associated with IDs corresponding to different pigs.

6. The pig body appearance scoring method according to claim 1, characterized in that: Obtaining a target scoring result according to the first scoring result and the second scoring result includes: The scores corresponding to the various indicators in the first scoring result and the second scoring result are weighted and summed to obtain the target scoring result.

7. A pig body appearance scoring device, characterized in that: include: A data acquisition module, used to acquire point cloud data and image data of the pig to be tested; A segmentation and detection module, used for filtering, registering and segmenting the point cloud data to obtain boundary segmentation data and body size data of the first part of the pig to be tested; The image data is processed based on a target detection model to obtain a recognition result of the second part of the pig to be tested; wherein the target detection model is obtained by training a deformable convolutional neural network based on sample images in combination with a multi-head self-attention mechanism; the first part includes at least one of the back, abdomen and limbs of the pig to be tested, the body size data includes at least one of the body length and height of the pig to be tested, and the second part includes at least one of the leg buttocks, coat color, hoof and nipple of the pig to be tested; A scoring module is used to score the first part according to the boundary segmentation data, score the body length and height of the pig to be tested according to the body size data, and obtain a first scoring result; score the second part according to the recognition result to obtain a second scoring result; and obtain a target scoring result according to the first scoring result and the second scoring result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the pig physical appearance scoring method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for pig physical appearance scoring according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for pig physical appearance scoring according to any one of claims 1 to 6 is implemented.

Images