Living pig rating method and system based on three-dimensional point cloud and storage medium

Abstract

The invention provides a live pig rating method and system based on a three-dimensional point cloud and a storage medium. The method comprises the following steps: collecting an initial point cloud of a live pig and preprocessing the initial point cloud; utilizing a DynEdgeNet point cloud segmentation model to segment the preprocessed initial point cloud into five parts, namely a head cheek part, a trunk part, four limbs parts, a hip part and an abdomen part; using a TransFusionNet deep learning network to extract features of the point clouds of each part obtained by segmentation, and respectively calculating comprehensive scores of the head, cheek, muscle feeling, abdomen in and posture uniformity of the pig according to the features of the point clouds of each part; all the comprehensive scores of the pigs are jointly input into a GradingInteractionNet classification network, and the rating of the live pigs is obtained; according to the method, pig rating is carried out based on the three-dimensional point cloud and deep learning, rapid, comprehensive and stress-free rating of live pigs can be realized, and the objectivity, precision and working efficiency of live pig rating are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional point cloud processing and pig rating, and more specifically, to a live pig rating method, system and storage medium based on three-dimensional point cloud. Background Art

[0002] With the growth of the global population and the increasing demand for high-quality meat products, the livestock industry is facing the dual challenges of improving production efficiency and product quality. Especially in the field of pork production, how to scientifically, quickly and stress-free evaluate the meat quality and growth status of live pigs has become the key to improving breeding efficiency. Traditional pig grading methods mainly rely on manual observation and simple body measurement. These methods are not only time-consuming and labor-intensive, but also highly subjective and inaccurate, and it is difficult to meet the needs of modern precision breeding.

[0003] For the rating of live pigs, traditional schemes mostly rely on manual measurement and subjective judgment by experienced buyers. Buyers use measuring tools such as semicircular instruments, calipers, and tape measures to roughly measure the body size of pigs, and then give pig grades based on their subjective experience. Each buyer has different judging criteria, and the ratings for the same pig are also different. This rating method is labor-intensive and has strong subjective factors. The rating of pigs is neither objective nor accurate. The rating process is likely to cause stress reactions in pigs, affecting the quality of the pigs' meat, and may also cause injuries to staff.

[0004] In recent years, the rapid development of deep learning technology and 3D imaging technology has provided new solutions for livestock quality assessment. By acquiring point cloud data of live pigs through high-precision 3D scanning technology and combining it with deep learning algorithms for intelligent analysis, it is possible to accurately assess multi-dimensional parameters such as pig shape, weight, and meat quality. However, existing technologies still have certain limitations in the processing and analysis of point cloud data:

[0005] 1) Acquisition and processing of point cloud data: Although point cloud data of live pigs can be obtained relatively conveniently using a single depth camera, it is often difficult to fully capture the three-dimensional information of the pigs due to the limitation of viewing angle; in addition, the accuracy and completeness of single-view data are also difficult to guarantee.

[0006] 2) Synchronization and fusion of point cloud data: Although the comprehensiveness of the data can be improved by obtaining point cloud data of pigs from different angles through multiple depth cameras, how to achieve effective synchronization and efficient fusion of multiple camera data becomes a technical difficulty; the time difference and data inconsistency between different cameras will lead to deviations in the final generated 3D model.

[0007] 3) Application of deep learning models: Although deep learning technology has made significant progress in image recognition and processing, how to design a deep learning model suitable for point cloud data and how to improve the learning efficiency and prediction accuracy of the model are still the difficulties of current research. Summary of the invention

[0008] In order to overcome the defects of traditional pig rating in the above-mentioned prior art, such as low rating efficiency, strong subjectivity, non-objective and non-uniform standards, asynchronous and difficult registration of point cloud acquisition, and poor prediction accuracy of point cloud deep learning network, the present invention provides a live pig rating method, system and storage medium based on three-dimensional point cloud, which can realize fast, comprehensive and stress-free rating of live pigs, and through the application of deep learning technology, the deep information contained in the pig point cloud data can be further mined, providing strong technical support for precision breeding and intelligent management.

[0009] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0010] A live pig rating method based on three-dimensional point cloud comprises the following steps:

[0011] S1: Collect the initial point cloud of live pigs and perform preprocessing;

[0012] S2: Use the pre-trained DynEdgeNet point cloud segmentation model to segment the pre-processed initial point cloud into five parts: head and cheeks, trunk, limbs, buttocks and abdomen;

[0013] S3: Use the pre-trained TransFusionNet deep learning network to extract the features of each part of the point cloud obtained by segmentation, and calculate the comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry based on the features of each part of the point cloud;

[0014] S4: The comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry are input into the pre-trained GradingInteractionNet classification network to obtain the rating of the living pig.

[0015] Preferably, in step S1, collecting the initial point cloud of the living pig includes:

[0016] S1.1: Calibrate several depth cameras separately to obtain the transformation matrix R corresponding to each depth camera i and the translation vector T i ;

[0017] S1.2: Use several calibrated depth cameras to simultaneously capture depth images of live pigs from different perspectives, and record the shooting timestamp of each depth camera;

[0018] S1.3: Calculate the shooting timestamp difference of all depth cameras in pairs, and determine whether all timestamp differences are less than a preset threshold. If so, execute step S1.4; otherwise, among the two depth cameras whose timestamp difference is greater than or equal to the preset threshold, select the depth camera with the earlier shooting timestamp to reshoot, and re-execute step S1.3;

[0019] S1.4: Generate a point cloud of a corresponding viewing angle using each of the depth images, and register the point clouds of each viewing angle according to the following formula to obtain the registered point clouds of each viewing angle:

[0020]

[0021] Among them, (x2, y2, z2) are the X-axis, Y-axis and Z-axis coordinates of the midpoint of the registered point cloud; (x1, y1, z1) are the X-axis, Y-axis and Z-axis coordinates of the midpoint of the unregistered point cloud; the X-axis is along the body length of the pig, the Y-axis is perpendicular to the ground, and the Z-axis is along the body width of the pig;

[0022] S1.5: Stitch the point clouds after registration from each perspective to obtain a complete initial point cloud of the living pig.

[0023] Preferably, in step S1, the preprocessing includes:

[0024] The initial point cloud of the living pig is subjected to background removal, denoising and downsampling operations in sequence to obtain a clean point cloud of the living pig, thereby completing the preprocessing.

[0025] Preferably, in step S2, the data processing steps of the pre-trained DynEdgeNet point cloud segmentation model include:

[0026] S21: Data input and feature extraction: The pre-trained DynEdgeNet point cloud segmentation model uses its multi-scale feature extraction module to perform multi-scale feature extraction on the pre-processed initial point cloud to obtain multi-scale point cloud features; the multi-scale feature extraction module has several U-Net networks arranged in parallel, and the pre-processed initial point cloud is dynamically downsampled at different scales and then input into different U-Net networks respectively, and each of the U-Net networks uses a dynamic convolution kernel to perform feature extraction;

[0027] S22: Feature fusion: Fuse multi-scale point cloud features to obtain point cloud fusion features;

[0028] S23: Semantic prediction and bounding box generation: The pre-trained DynEdgeNet point cloud segmentation model uses its semantic and bounding box prediction modules to perform semantic label prediction and bounding box prediction on the point cloud fusion features;

[0029] S24: Data output: Output the semantic label of each point and the bounding box of each part of the point cloud to complete the segmentation, and segment the preprocessed initial point cloud into five parts: head and cheeks, trunk, limbs, buttocks and abdomen.

[0030] Preferably, step S3 comprises:

[0031] S31: performing sampling and regularization operations on each part of the point cloud obtained by segmentation;

[0032] S32: Input the regularized point clouds of each part into the pre-trained TransFusionNet deep learning network, use its P-Transformer feature extraction module to perform feature extraction, and extract the local features F1, intermediate features F2 and global features F3 of each part of the point cloud in turn;

[0033] The P-Transformer feature extraction module performs feature extraction based on FlatFormer;

[0034] S33: Feature fusion: The local features F1, intermediate features F2 and global features F3 of each part of the point cloud are concatenated and fused to obtain the fused feature vector F concat ;

[0035] S34: Feature processing: The fusion feature vector F of each part of the point cloud concat Input the multi-layer perceptron for feature processing to obtain the final feature vector of each part of the point cloud;

[0036] S35: Classification output: The final feature vector of each part of the point cloud is input into the fully connected layer for classification, and the category scores of each part of the point cloud for the pig's head and cheeks, muscularity, abdomen and body symmetry are output;

[0037] S36: The category scores of the pig's head and cheeks, muscularity, abdomen and body symmetry of each part of the point cloud are weighted and summed up respectively to obtain a comprehensive score of the pig's head and cheeks, muscularity, abdomen and body symmetry.

[0038] Preferably, the step S31 includes:

[0039] S311: For the point cloud of the pig's limbs, a curvature-based sampling method is used for sampling; for the point cloud of the pig's trunk, abdomen and buttocks, random sampling is used; for the point cloud of the pig's head and cheeks, geometric sampling is used; and the sampling operation is completed;

[0040] S312: For the sampling results of each part of the point cloud, the size of each part of the point cloud is scaled to a uniform scale by using a center normalization method to complete the regularization operation.

[0041] Preferably, step S4 comprises:

[0042] S41: The comprehensive scores of the pig's head and cheeks, muscle feeling, abdomen and body symmetry are formed into a set of four-dimensional vectors and input into the pre-trained GradingInteractionNet classification network;

[0043] S42: Feature extraction: using the first multi-layer perceptron in the pre-trained GradingInteractionNet classification network to perform preliminary feature extraction on the four-dimensional vector to obtain original score features of four comprehensive scores;

[0044] S43: Feature interaction: Calculate the product interaction between the original score features of the four comprehensive scores in pairs to obtain six interaction features;

[0045] S44: merging the six interactive features with the four original score features of the comprehensive score, and inputting them into the second multi-layer perceptron to obtain a comprehensive feature representation;

[0046] S46: Map the comprehensive feature representation output by the second multi-layer perceptron into a final rating classification to obtain the rating of the live pigs.

[0047] Preferably, the comprehensive score of the pig's head and gills is used to evaluate the pig's body fat level. The lower the comprehensive score of the head and gills, the higher the pig's body fat rate.

[0048] The comprehensive score of the muscle feel of the pigs is used to evaluate the strength of the pigs' limbs and the firmness of the muscles in the buttocks; the higher the comprehensive score of the muscle feel, the better the muscle feel;

[0049] The comprehensive score of the pig's abdomen is used to evaluate the pig's body compactness, the straightness of the abdominal line, the smoothness of the skin and the thickness of the abdominal fat; the higher the comprehensive score of the pig's abdomen, the more compact the pig's abdomen is, and the higher the lean meat rate of its offspring;

[0050] The comprehensive score of the pig's body symmetry is used to evaluate the overall appearance proportions and coordination of various parts of the pig; the higher the comprehensive score of the pig's body symmetry, the healthier the pig grows.

[0051] The present invention also provides a live pig rating system based on three-dimensional point cloud, which uses the above-mentioned live pig rating method based on three-dimensional point cloud, including:

[0052] Preprocessing unit: used to collect the initial point cloud of live pigs and perform preprocessing;

[0053] Point cloud segmentation unit: used to segment the pre-processed initial point cloud into five parts: head and cheeks, trunk, limbs, buttocks and abdomen using the pre-trained DynEdgeNet point cloud segmentation model;

[0054] Comprehensive scoring unit: used to extract the features of each part of the point cloud obtained by segmentation using the pre-trained TransFusionNet deep learning network, and calculate the comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry according to the features of each part of the point cloud;

[0055] Pig rating unit: It is used to input the comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry into the pre-trained GradingInteractionNet classification network to obtain the rating of the live pigs.

[0056] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method are implemented.

[0057] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0058] The present invention provides a method, system and storage medium for rating live pigs based on three-dimensional point clouds. First, the initial point cloud of the live pig is collected and preprocessed; then, the preprocessed initial point cloud is segmented into five parts, namely, head and cheeks, trunk, limbs, buttocks and abdomen, using a pre-trained DynEdgeNet point cloud segmentation model; then, the features of each part of the point cloud obtained by segmentation are extracted using a pre-trained TransFusionNet deep learning network, and the comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry are calculated according to the features of each part of the point cloud; finally, the comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry are input into a pre-trained GradingInteractionNet classification network to obtain the rating of the live pig;

[0059] The present invention solves the problems of low efficiency, strong subjectivity, inconsistent standards, and easy induction of stress reactions in pigs in traditional rating methods; the present invention rates pigs based on three-dimensional point clouds, which can achieve fast, comprehensive, and stress-free rating of live pigs, and through the application of deep learning technology, it can further mine the deep-level information contained in the pig point cloud data, providing strong technical support for precision breeding and intelligent management; the present invention effectively improves the objectivity, accuracy and work efficiency of live pig rating, provides a scientific basis for modern breeding, and at the same time helps to improve the overall production efficiency of pig farms, and provides a scientific pricing basis for the market, thereby improving transaction transparency and credibility, and thus improving overall economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a flow chart of a live pig rating method based on three-dimensional point cloud provided in Example 1.

[0061] Figure 2This is the original three-dimensional initial point cloud image of the living pig provided in Example 2.

[0062] Figure 3 This is the original three-dimensional point cloud image of the living pig after pretreatment provided in Example 2.

[0063] Figure 4 This is a structural diagram of the DynEdgeNet point cloud segmentation model provided in Example 2.

[0064] Figure 5 This is a segmentation result diagram obtained by segmenting the three-dimensional point cloud data of the living pig provided in Example 2 according to the target part.

[0065] Figure 6 This is the TransFusionNet deep learning network structure diagram provided in Example 2.

[0066] Figure 7 This is the GradingInteractionNet classification network structure diagram provided in Example 2.

[0067] Figure 8 This is a structural diagram of a live pig rating system based on three-dimensional point cloud provided in Example 3. DETAILED DESCRIPTION

[0068] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0069] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0070] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0071] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0072] Example 1

[0073] like Figure 1 As shown, this embodiment provides a live pig rating method based on three-dimensional point cloud, comprising the following steps:

[0074] S1: Collect the initial point cloud of live pigs and perform preprocessing;

[0075] S2: Use the pre-trained DynEdgeNet point cloud segmentation model to segment the pre-processed initial point cloud into five parts: head and cheeks, trunk, limbs, buttocks and abdomen;

[0076] S3: Use the pre-trained TransFusionNet deep learning network to extract the features of each part of the point cloud obtained by segmentation, and calculate the comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry based on the features of each part of the point cloud;

[0077] S4: The comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry are input into the pre-trained GradingInteractionNet classification network to obtain the rating of the living pig.

[0078] In the specific implementation process, the initial point cloud of the live pigs is first collected and preprocessed;

[0079] Then, the pre-trained DynEdgeNet point cloud segmentation model is used to segment the pre-processed initial point cloud into five parts: head and cheeks, trunk, limbs, buttocks and abdomen;

[0080] Then, the TransFusionNet deep network was used to learn the features of each segmented part of the pig and give a score. The TransFusionNet point cloud deep regression network model was used to learn the features of each segmented part of the pig and calculate the contribution of each part to the pig's features of head and cheeks, muscularity, abdomen, and body symmetry. The score of each feature was counted to obtain the four-element group of the total score of the pig's features [head and cheeks, muscularity, abdomen, and body symmetry].

[0081] Finally, the four-element group of the total feature score of the pig is input into the GradingInteractionNet classification network to obtain the pig's rating. The score of each part is used as input, and the multi-layer perceptron is used to gradually extract nonlinear features. The feature interaction layer explicitly models the interaction relationship between the scoring features. Finally, the rating result is output through Softmax to obtain the final rating of the pig.

[0082] This method is based on three-dimensional point clouds for pig rating, which can achieve fast, comprehensive and stress-free rating of live pigs. Moreover, through the application of deep learning technology, it can further explore the deep information contained in the pig point cloud data, providing strong technical support for precision breeding and intelligent management.

[0083] Example 2

[0084] This embodiment provides a live pig rating method based on three-dimensional point cloud, comprising the following steps:

[0085] S1: Collect the initial point cloud of live pigs and perform preprocessing;

[0086] S2: Use the pre-trained DynEdgeNet point cloud segmentation model to segment the pre-processed initial point cloud into five parts: head and cheeks, trunk, limbs, buttocks and abdomen;

[0087] S3: Use the pre-trained TransFusionNet deep learning network to extract the features of each part of the point cloud obtained by segmentation, and calculate the comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry based on the features of each part of the point cloud;

[0088] S4: The comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry are input into the pre-trained GradingInteractionNet classification network to obtain the rating of the live pig;

[0089] In step S1, collecting the initial point cloud of the live pig includes:

[0090] S1.1: Calibrate several depth cameras separately to obtain the transformation matrix R corresponding to each depth camera i and the translation vector T i ;

[0091] S1.2: Use several calibrated depth cameras to simultaneously capture depth images of live pigs from different perspectives, and record the shooting timestamp of each depth camera;

[0092] S1.3: Calculate the shooting timestamp difference of all depth cameras in pairs, and determine whether all timestamp differences are less than a preset threshold. If so, execute step S1.4; otherwise, among the two depth cameras whose timestamp difference is greater than or equal to the preset threshold, select the depth camera with the earlier shooting timestamp to reshoot, and re-execute step S1.3;

[0093] S1.4: Generate a point cloud of a corresponding viewing angle using each of the depth images, and register the point clouds of each viewing angle according to the following formula to obtain the registered point clouds of each viewing angle:

[0094]

[0095] Among them, (x2, y2, z2) are the X-axis, Y-axis and Z-axis coordinates of the midpoint of the registered point cloud; (x1, y1, z1) are the X-axis, Y-axis and Z-axis coordinates of the midpoint of the unregistered point cloud; the X-axis is along the body length of the pig, the Y-axis is perpendicular to the ground, and the Z-axis is along the body width of the pig;

[0096] S1.5: stitching the point clouds after registration from each perspective to obtain a complete initial point cloud of the live pig;

[0097] In step S1, the preprocessing includes:

[0098] The initial point cloud of the living pig is subjected to background removal, denoising and downsampling operations in sequence to obtain a clean point cloud of the living pig, thereby completing the preprocessing;

[0099] In step S2, the data processing steps of the pre-trained DynEdgeNet point cloud segmentation model include:

[0100] S21: Data input and feature extraction: The pre-trained DynEdgeNet point cloud segmentation model uses its multi-scale feature extraction module to perform multi-scale feature extraction on the pre-processed initial point cloud to obtain multi-scale point cloud features; the multi-scale feature extraction module has several U-Net networks arranged in parallel, and the pre-processed initial point cloud is dynamically downsampled at different scales and then input into different U-Net networks respectively, and each of the U-Net networks uses a dynamic convolution kernel to perform feature extraction;

[0101] S22: Feature fusion: Fuse multi-scale point cloud features to obtain point cloud fusion features;

[0102] S23: Semantic prediction and bounding box generation: The pre-trained DynEdgeNet point cloud segmentation model uses its semantic and bounding box prediction modules to perform semantic label prediction and bounding box prediction on the point cloud fusion features;

[0103] S24: Data output: Output the semantic label of each point and the bounding box of each part of the point cloud to complete the segmentation, and segment the preprocessed initial point cloud into five parts: head and cheeks, trunk, limbs, buttocks and abdomen;

[0104] The step S3 comprises:

[0105] S31: Sampling and regularization operations are performed on each part of the point cloud obtained by segmentation, including:

[0106] S311: For the point cloud of the pig's limbs, a curvature-based sampling method is used for sampling; for the point cloud of the pig's trunk, abdomen and buttocks, random sampling is used; for the point cloud of the pig's head and cheeks, geometric sampling is used; and the sampling operation is completed;

[0107] S312: for the sampling results of each part of the point cloud, the size of each part of the point cloud is scaled to a uniform scale by using a center normalization method to complete the regularization operation;

[0108] S32: Input the regularized point clouds of each part into the pre-trained TransFusionNet deep learning network, use its P-Transformer feature extraction module to perform feature extraction, and extract the local features F1, intermediate features F2 and global features F3 of each part of the point cloud in turn;

[0109] The P-Transformer feature extraction module performs feature extraction based on FlatFormer;

[0110] S33: Feature fusion: The local features F1, intermediate features F2 and global features F3 of each part of the point cloud are concatenated and fused to obtain the fused feature vector F concat ;

[0111] S34: Feature processing: The fusion feature vector F of each part of the point cloud concat Input the multi-layer perceptron for feature processing to obtain the final feature vector of each part of the point cloud;

[0112] S35: Classification output: The final feature vector of each part of the point cloud is input into the fully connected layer for classification, and the category scores of each part of the point cloud for the pig's head and cheeks, muscularity, abdomen and body symmetry are output;

[0113] S36: weighted sum of the category scores of the pig's head and cheeks, muscularity, abdomen and body symmetry for each part of the point cloud, and obtain a comprehensive score of the pig's head and cheeks, muscularity, abdomen and body symmetry;

[0114] The step S4 comprises:

[0115] S41: The comprehensive scores of the pig's head and cheeks, muscle feeling, abdomen and body symmetry are formed into a set of four-dimensional vectors and input into the pre-trained GradingInteractionNet classification network;

[0116] S42: Feature extraction: using the first multi-layer perceptron in the pre-trained GradingInteractionNet classification network to perform preliminary feature extraction on the four-dimensional vector to obtain original score features of four comprehensive scores;

[0117] S43: Feature interaction: Calculate the product interaction between the original score features of the four comprehensive scores in pairs to obtain six interaction features;

[0118] S44: merging the six interactive features with the four original score features of the comprehensive score, and inputting them into the second multi-layer perceptron to obtain a comprehensive feature representation;

[0119] S46: Mapping the comprehensive feature representation output by the second multi-layer perceptron into a final rating classification to obtain the rating of the live pig;

[0120] In this embodiment, the comprehensive score of the pig's head and gills is used to evaluate the pig's body fat level. The lower the comprehensive score of the head and gills, the higher the body fat rate of the pig;

[0121] The comprehensive score of the muscle feel of the pigs is used to evaluate the strength of the pigs' limbs and the firmness of the muscles in the buttocks; the higher the comprehensive score of the muscle feel, the better the muscle feel;

[0122] The comprehensive score of the pig's abdomen is used to evaluate the pig's body compactness, the straightness of the abdominal line, the smoothness of the skin and the thickness of the abdominal fat; the higher the comprehensive score of the pig's abdomen, the more compact the pig's abdomen is, and the higher the lean meat rate of its offspring;

[0123] The comprehensive score of the pig's body symmetry is used to evaluate the overall appearance proportions and coordination of various parts of the pig; the higher the comprehensive score of the pig's body symmetry, the healthier the pig grows.

[0124] In the specific implementation process, the initial point cloud of the live pigs is first collected and preprocessed;

[0125] Specifically, this embodiment uses a depth camera to capture pig point clouds from four different angles, and obtains the original point clouds of the pigs through preprocessing steps such as registration and denoising. This embodiment sets the depth camera directly above as the main camera and obtains the internal parameters of the main camera.

[0126] Add two groups of parallel detection lines to the image of the main camera, which are parallel within the groups and perpendicular between the groups. The detection lines form a rectangular detection frame. At the same time, the depth of the detection frame is set to be valid from the ground to the camera, which constitutes the detection space of this method. The camera counts the point cloud in the detection space in real time; when the number of point clouds in the detection space is greater than 60,000, it means that the pigs have entered the best acquisition area. The host sends an acquisition signal to the camera and immediately starts the depth image acquisition; after receiving the depth image acquisition signal, all cameras immediately acquire the depth image of the current frame and record the precise acquisition time; then cyclically check the shooting time of all cameras. If the timestamp difference is less than 3 milliseconds, proceed to the next step; otherwise, the camera with an earlier timestamp will reshoot the current frame and repeat the cyclic check, which ensures the synchronization of the depth information collected by all cameras;

[0127] After verification, the depth image is obtained and the point cloud P corresponding to the depth image is generated. n}, p i ={X,Y,Z}, where X, Y, and Z are the three-dimensional coordinates of the point respectively; obtain the translation vector and transformation matrix, obtain the point cloud of the calibration object from the four cameras respectively, and obtain the transformation matrix R corresponding to each camera through the calibration object registration method i and the translation vector T i ;

[0128]

[0129] The point clouds captured by each camera can be aligned in the same coordinate system through the above formula, where the x-axis is along the body length of the pig, the y-axis is perpendicular to the ground, and the z-axis is along the body width of the pig;

[0130] The point clouds of each view are aligned to obtain the complete initial point cloud. Figure 2 As shown, it contains invalid information such as background and noise;

[0131] The original point cloud is preprocessed by removing background, denoising, downsampling and other operations to obtain a clean original point cloud of pigs, such as Figure 3 As shown;

[0132] Then, the pre-trained DynEdgeNet point cloud segmentation model is used to segment the pre-processed initial point cloud into five parts: head and cheeks, trunk, limbs, buttocks and abdomen; Figure 4 Shown is the structural diagram of the DynEdgeNet point cloud segmentation model;

[0133] Specifically, manually segment part of the original point cloud, select part of the initial point cloud obtained in the above step for manual segmentation, and segment the pig point cloud into five parts: head and cheeks, trunk, limbs, buttocks, and abdomen;

[0134] 1) Through the input layer and feature extraction layer: In the feature extraction module, DynEdgeNet uses lightweight convolution combined with sparse convolutional networks to perform multi-scale feature extraction on point cloud data; through convolution operations, the model can capture the global geometric structure and local details in the point cloud; the U-Net structure of the model enables it to have powerful downsampling and upsampling capabilities: the downsampling stage extracts global features, and the upsampling stage restores fine-grained local details;

[0135] 2) Semantic prediction and bounding box generation: After extracting multi-scale point cloud features, DynEdgeNet performs semantic prediction and bounding box prediction for each point; the point-level predictor assigns a semantic label to each point (such as head, cheek, torso, etc.); at the same time, the model also generates an axis-aligned bounding box for each instance to ensure that each instance is accurately represented in terms of geometric range and to provide strong support for subsequent convolution operations;

[0136] 3) Dynamic sampling and feature aggregation: In order to perform instance segmentation more efficiently, DynEdgeNet introduces a dynamic sampling mechanism; this mechanism ensures that foreground objects are given priority by dynamically adjusting the sampling ratio of foreground and background points, especially for small objects and complex scenes; then, the multi-scale attention mechanism. By aggregating features of different scales, it enhances the detection ability of small objects and the understanding of the global scene;

[0137] 4) Dynamic convolution kernel generation: In the process of generating convolution kernels, DynEdgeNet combines the bounding box and point features of the instance to generate instance-specific convolution kernels; the convolution kernel not only combines appearance information, but also refers to geometric shape cues; this combination improves the segmentation accuracy of convolution operations in complex scenes, especially for irregular objects and complex boundaries, the effect is more significant;

[0138] 5) Output layer: The output layer of DynEdgeNet generates semantic labels and instance masks for each point. By combining the calculation results of all modules, the model performs accurate semantic and instance segmentation on the input point cloud data, and outputs the results as the category label and segmentation mask of each point. Finally, the original point cloud of the pig body is segmented into five parts: head, gills, trunk, limbs, buttocks, and abdomen. The final effect is as follows: Figure 5 As shown;

[0139] Then, the TransFusionNet deep network was used to learn the features of each segmented part of the pig and give a score. The TransFusionNet point cloud deep regression network model was used to learn the features of each segmented part of the pig and calculate the contribution of each part to the pig's features of head and cheeks, muscularity, abdomen, and body symmetry. The score of each feature was counted to obtain the four-element group of the pig's total feature score [head and cheeks, muscularity, abdomen, and body symmetry]; Figure 6 The figure shows the structure of the TransFusionNet deep network. The specific process is as follows:

[0140] 1) Point cloud preprocessing: sampling and regularization operations on the segmented point cloud;

[0141] Since the point cloud of each part has different characteristics, the point cloud of the pig leg is dense and its shape and curvature contain more information; the point cloud of the pig trunk has a lower density and its volume can better reflect its characteristics; etc. Therefore, using different sampling strategies for point clouds of different parts can better highlight the characteristics of the point cloud; in detail: 1. For the point cloud of the limbs, curvature-based sampling (Curvature-based Sampling) is adopted; the point cloud of the limbs is characterized by high density and more curvature changes. Each bend may contain more useful information, so it is necessary to retain key edge and corner features; 2. For the point cloud of the trunk, abdomen, and buttocks, random sampling (Random Sampling) is adopted; the point cloud of the trunk is characterized by uniform distribution and no complex shapes. The features contained in it are volume and area, etc., and sampling is unlikely to affect the collection of these features, so the most efficient random sampling is adopted; 3. For the point cloud of the head and cheeks, geometric sampling is adopted; the point cloud of the head and cheeks has obvious geometric characteristics, and the use of geometric sampling can retain the geometric features to the greatest extent;

[0142] Center normalization is used to ensure that the geometric center of the point cloud is located at the origin, and the point cloud size is scaled to a uniform scale, that is, the maximum distance from the point cloud to the origin is 1;

[0143] 2) Input the point cloud deep learning network P-Transformer feature extraction module, which is based on FlatFormer (Transformer suitable for point cloud). It is disassembled and multi-feature fusion is added to build a more efficient P-Transformer feature extraction module;

[0144] Extracting local features F1: The early layer Transformer module captures the local information of each point and its neighboring points, capturing lower-level features such as local geometric structure;

[0145] Extracting intermediate features F2: The mid-layer Transformer begins to focus on the relationship between wider points and combines local and global information to form intermediate features F2;

[0146] Extracting global features F3: The later layer Transformer captures the global features of the entire point cloud and outputs highly abstract global semantic information;

[0147] 3) Feature fusion: The features F1, F2 and F3 extracted from each layer are concatenated into a long feature vector F concat ; By fusing features of different scales, this model can capture the interactive relationships and complex patterns between different features, thereby improving the model's expressiveness; in addition, feature fusion can integrate multi-scale information, effectively reduce information loss, and enhance the model's ability to recognize and classify point clouds; when a single feature performs poorly due to noise or incomplete information, fusing other features can effectively alleviate this effect, making the model more stable and robust;

[0148] 4) Feature processing layer: The fused features are input into a multi-layer perceptron for further feature processing; the feature processing layer consists of multiple fully connected layers and activation functions; the multi-layer perceptron processes features layer by layer and enhances feature expression capabilities through nonlinear transformation and regularization operations;

[0149] 5) Classification output layer: After the feature processing layer, the final features of the model are input into the classification layer; this layer usually includes a fully connected layer to map the high-dimensional features to score categories 0 to 10;

[0150] 6) Each segmented part will be trained with a separate score, and the weight of the part for the head and cheeks, muscularity, abdomen, and body symmetry will be given to this part. The contribution of all parts to each feature will be calculated to finally calculate the score of the pig; the weight will be continuously optimized as the model learns until it tends to be balanced;

[0151] Finally, the four-element group of the total score of the pig's features is input into the GradingInteractionNet classification network to obtain the pig's rating. The scores of each part are used as input, and the multi-layer perceptron is used to gradually extract nonlinear features. The feature interaction layer explicitly models the interaction relationship between the scoring features. Finally, the rating result is output through Softmax to obtain the final rating of the pig; Figure 7 The following is the structure diagram of the GradingInteractionNet classification network. The specific process is as follows:

[0152] 1) Input feature scores. The input data mainly includes four scores, including head and cheek score, muscle score, abdomen score and body symmetry score. Each score ranges from 0 to 10. These scores together form a four-dimensional vector to describe the various physical characteristics of the pig.

[0153] Explanation of the head and cheek feature score: The head and cheek refer to the head and neck area of ​​the pig, especially the muscles and fat tissue near the cheeks; the hypertrophy or relaxation of the head and cheeks is often related to the body fat level of the pig; the score is based on the following: the thickness of the head and cheek fat; the firmness of the head and cheeks; the proportion of the head and cheeks; the thicker the fat, the lower the score; the firmer the muscle, the higher the score; the larger the proportion of the head and cheeks to the whole pig, the higher the score; the head and cheeks are an important indicator to measure the quality of the pig;

[0154] Explanation of the muscle feeling characteristic score; Muscle feeling is mainly affected by the strength of the limbs and the firmness of the buttocks muscles; Pigs with strong muscle feeling have full and thick muscles in their limbs, shoulders and buttocks; the muscles are evenly distributed on both sides with good symmetry, showing good bone and muscle structure; the stronger the muscle feeling, the higher the score;

[0155] Explanation of the abdominal characteristic score; the compactness of the pig's body, the straightness of the abdominal line, the smoothness of the skin and the thickness of the abdominal fat; a tight abdomen and a straight waist and abdomen line usually indicate that the pig is healthy and has good muscle development, while a moderate fat thickness can ensure the taste and market value of the meat; too much abdominal fat or loose abdomen may affect economic benefits, which is an indicator that needs to be paid attention to during the breeding process; at the same time, this is also an important indicator for breeding. The more compact the abdomen of the pig, the higher the lean meat rate of its offspring will be;

[0156] Explanation of the scoring of the body symmetry feature; The body symmetry of pigs is an important criterion in pig scoring, which mainly evaluates the overall appearance proportion and coordination of various parts of the pig; pigs with symmetrical bodies usually have straight backs, well-developed and evenly distributed muscles on the shoulders, waist and buttocks, and strong limbs that are in proportion to the body; A symmetrical body reflects that the pigs are healthy and nutritionally balanced during their growth, and are generally considered to have higher production and economic value; Pigs with asymmetrical bodies may be partially overweight or too thin, or have uneven distribution of muscles and fat, which affects market demand;

[0157] 2) Input layer: The input layer receives four scoring features [x1, x2, x3, x4], which represent the scores of head and cheek, muscularity, abdomen, and body symmetry, respectively. Each score is a floating point number between 0 and 10.

[0158] 3) The first multi-layer perceptron: The first multi-layer perceptron layer receives the four rating features transmitted by the input layer; this layer maps the four ratings to a 16-dimensional feature space through a fully connected layer, expanding the representation capability of the features; the activation function ReLU is used to introduce nonlinearity to help the model capture the complex relationships in the rating features; after this layer of processing, the output result is a 16-dimensional feature vector, which contains the preliminary feature extraction of the original ratings;

[0159] 4) Feature Interaction Layer: The feature interaction layer captures the mutual influence between features by calculating the pairwise interaction terms between the scoring features. For the four scoring features, the pairwise product interactions between them are calculated to generate six interaction features. These interaction features are obtained by calculating x1×x2, x1×x3, x1×x4, x2×x3, x2×x4, and x3×x4. Through the interaction layer, the model can capture the dependencies and interactions between the scoring features.

[0160] 5) Second multi-layer perceptron: The second multi-layer perceptron layer receives the output of the feature interaction layer and merges it with the original score features. This layer maps these inputs to a 32-dimensional feature space through a fully connected layer and uses the ReLU activation function to further enhance the expressiveness of the features. After this layer of processing, the output result is a 32-dimensional feature vector that contains a comprehensive representation of the score features and their interaction features.

[0161] 6) Output layer: The output layer receives the output features of the multi-layer perceptron layer 2 and maps them to the final rating category; through a fully connected layer, the model maps the 32-dimensional feature vector to 4 output values, corresponding to 4 rating levels ("first class", "second class", "third class", "fourth class"); the output layer uses the Softmax function to convert the 4 output values ​​into probability distributions, indicating the probability of the pig's rating belonging to each level; finally, the model selects the category with the largest probability as the pig's rating result;

[0162] Table 1 shows an example of the results of using this method to grade 6 live pigs:

[0163] Table 1 Rating results example table

[0164]

[0165]

[0166] As can be seen from Table 1, this method can achieve fast, comprehensive and stress-free rating of live pigs based on three-dimensional point clouds for pig rating. Moreover, through the application of deep learning technology, the deep information contained in the pig point cloud data can be further mined, providing strong technical support for precision breeding and intelligent management.

[0167] Example 3

[0168] like Figure 8 As shown, this embodiment provides a live pig rating system based on three-dimensional point cloud, using a live pig rating method based on three-dimensional point cloud described in embodiment 1 or 2, including:

[0169] Preprocessing unit 301: used to collect initial point clouds of live pigs and perform preprocessing;

[0170] Point cloud segmentation unit 302: used to segment the pre-processed initial point cloud into five parts: head and cheek, trunk, limbs, buttocks and abdomen using the pre-trained DynEdgeNet point cloud segmentation model;

[0171] Comprehensive scoring unit 303: used to extract the features of each part of the point cloud obtained by segmentation using the pre-trained TransFusionNet deep learning network, and calculate the comprehensive scores of the pig's head and cheeks, muscle feeling, abdomen and body symmetry according to the features of each part of the point cloud;

[0172] The pig rating unit 304 is used to input the comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry into the pre-trained GradingInteractionNet classification network to obtain the rating of the live pig.

[0173] In the specific implementation process, firstly, the preprocessing unit 301 collects the initial point cloud of the live pig and performs preprocessing;

[0174] Then the point cloud segmentation unit 302 uses the pre-trained DynEdgeNet point cloud segmentation model to segment the pre-processed initial point cloud into five parts: head and cheeks, trunk, limbs, buttocks and abdomen;

[0175] Then, the comprehensive scoring unit 303 uses the TransFusionNet deep network to learn the features of each segmented part of the pig and gives a score. The TransFusionNet point cloud deep regression network model is used to learn the features of each segmented part of the pig and calculate the contribution of each part to the pig's features of head and cheeks, muscularity, abdomen, and body symmetry. The score of each feature is counted to obtain the four-element group of the total score of the pig's features [head and cheeks, muscularity, abdomen, and body symmetry].

[0176] Finally, the pig rating unit 304 inputs the four-element group of the total feature score of the pig into the GradingInteractionNet classification network to obtain the pig rating, takes the score of each part as input, uses the multi-layer perceptron therein to gradually extract nonlinear features, and explicitly models the interaction relationship between the scoring features in the feature interaction layer, and finally outputs the rating result through Softmax to obtain the final rating of the pig;

[0177] This system grades pigs based on three-dimensional point clouds, and can achieve fast, comprehensive, and stress-free rating of live pigs. Moreover, through the application of deep learning technology, it can further explore the deep information contained in the pig point cloud data, providing strong technical support for precision breeding and intelligent management.

[0178] The same or similar reference numerals correspond to the same or similar components;

[0179] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0180] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A live pig rating method based on three-dimensional point cloud, characterized in that: The following steps are involved: S1: Collect the initial point cloud of live pigs and perform preprocessing; S2: Use the pre-trained DynEdgeNet point cloud segmentation model to segment the pre-processed initial point cloud into five parts: head and cheeks, trunk, limbs, buttocks and abdomen; S3: Use the pre-trained TransFusionNet deep learning network to extract the features of each part of the point cloud obtained by segmentation, and calculate the comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry based on the features of each part of the point cloud; S4: The comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry are input into the pre-trained GradingInteractionNet classification network to obtain the rating of the living pig.

2. A live pig rating method based on three-dimensional point cloud according to claim 1, characterized in that: In step S1, collecting the initial point cloud of the live pig includes: S1.1: Calibrate several depth cameras separately to obtain the transformation matrix R corresponding to each depth camera i and the translation vector T i ; S1.2: Use several calibrated depth cameras to simultaneously capture depth images of live pigs from different perspectives, and record the shooting timestamp of each depth camera; S1.3: Calculate the shooting timestamp difference of all depth cameras in pairs, and determine whether all timestamp differences are less than a preset threshold. If so, execute step S1.4; otherwise, among the two depth cameras whose timestamp difference is greater than or equal to the preset threshold, select the depth camera with the earlier shooting timestamp to reshoot, and re-execute step S1.3; S1.4: Generate a point cloud of a corresponding viewing angle using each of the depth images, and register the point clouds of each viewing angle according to the following formula to obtain the registered point clouds of each viewing angle: Among them, (x2, y2, z2) are the X-axis, Y-axis and Z-axis coordinates of the midpoint of the registered point cloud; (x1, y1, z1) are the X-axis, Y-axis and Z-axis coordinates of the midpoint of the unregistered point cloud; the X-axis is along the body length of the pig, the Y-axis is perpendicular to the ground, and the Z-axis is along the body width of the pig; S1.5: Stitch the point clouds after registration from each perspective to obtain a complete initial point cloud of the living pig.

3. The method for rating live pigs based on three-dimensional point cloud according to claim 1, characterized in that: In step S1, the preprocessing includes: The initial point cloud of the living pig is subjected to background removal, denoising and downsampling operations in sequence to obtain a clean point cloud of the living pig, thereby completing the preprocessing.

4. The method for rating live pigs based on three-dimensional point cloud according to claim 1, characterized in that: In step S2, the data processing steps of the pre-trained DynEdgeNet point cloud segmentation model include: S21: Data input and feature extraction: The pre-trained DynEdgeNet point cloud segmentation model uses its multi-scale feature extraction module to perform multi-scale feature extraction on the pre-processed initial point cloud to obtain multi-scale point cloud features; the multi-scale feature extraction module has several U-Net networks arranged in parallel, and the pre-processed initial point cloud is dynamically downsampled at different scales and then input into different U-Net networks respectively, and each of the U-Net networks uses a dynamic convolution kernel to perform feature extraction; S22: Feature fusion: Fuse multi-scale point cloud features to obtain point cloud fusion features; S23: Semantic prediction and bounding box generation: The pre-trained DynEdgeNet point cloud segmentation model uses its semantic and bounding box prediction modules to perform semantic label prediction and bounding box prediction on the point cloud fusion features; S24: Data output: Output the semantic label of each point and the bounding box of each part of the point cloud to complete the segmentation, and segment the preprocessed initial point cloud into five parts: head and cheeks, trunk, limbs, buttocks and abdomen.

5. The method for rating live pigs based on three-dimensional point cloud according to claim 1, characterized in that: The step S3 comprises: S31: performing sampling and regularization operations on each part of the point cloud obtained by segmentation; S32: Input the regularized point clouds of each part into the pre-trained TransFusionNet deep learning network, use its P-Transformer feature extraction module to perform feature extraction, and extract the local features F1, intermediate features F2 and global features F3 of each part of the point cloud in turn; The P-Transformer feature extraction module performs feature extraction based on FlatFormer; S33: Feature fusion: The local features F1, intermediate features F2 and global features F3 of each part of the point cloud are concatenated and fused to obtain the fused feature vector F concat ; S34: Feature processing: The fusion feature vector F of each part of the point cloud concat Input the multi-layer perceptron for feature processing to obtain the final feature vector of each part of the point cloud; S35: Classification output: The final feature vector of each part of the point cloud is input into the fully connected layer for classification, and the category scores of each part of the point cloud for the pig's head and cheeks, muscularity, abdomen and body symmetry are output; S36: The category scores of the pig's head and cheeks, muscularity, abdomen and body symmetry of each part of the point cloud are weighted and summed up respectively to obtain a comprehensive score of the pig's head and cheeks, muscularity, abdomen and body symmetry.

6. A live pig rating method based on three-dimensional point cloud according to claim 5, characterized in that: The step S31 comprises: S311: For the point cloud of the pig's limbs, a curvature-based sampling method is used for sampling; for the point cloud of the pig's trunk, abdomen and buttocks, random sampling is used; for the point cloud of the pig's head and cheeks, geometric sampling is used; and the sampling operation is completed; S312: For the sampling results of each part of the point cloud, the size of each part of the point cloud is scaled to a uniform scale by using a center normalization method to complete the regularization operation.

7. The method for rating live pigs based on three-dimensional point cloud according to claim 5, characterized in that: The step S4 comprises: S41: The comprehensive scores of the pig's head and cheeks, muscle feeling, abdomen and body symmetry are formed into a set of four-dimensional vectors and input into the pre-trained GradingInteractionNet classification network; S42: Feature extraction: using the first multi-layer perceptron in the pre-trained GradingInteractionNet classification network to perform preliminary feature extraction on the four-dimensional vector to obtain original score features of four comprehensive scores; S43: Feature interaction: Calculate the product interaction between the original score features of the four comprehensive scores in pairs to obtain six interaction features; S44: merging the six interactive features with the four original score features of the comprehensive score, and inputting them into the second multi-layer perceptron to obtain a comprehensive feature representation; S46: Map the comprehensive feature representation output by the second multi-layer perceptron into a final rating classification to obtain the rating of the live pigs.

8. A live pig rating method based on three-dimensional point cloud according to any one of claims 1 to 7, characterized in that: The comprehensive score of the pig's head and gills is used to evaluate the pig's body fat level. The lower the comprehensive score of the head and gills, the higher the pig's body fat rate. The comprehensive score of the muscle feel of the pigs is used to evaluate the strength of the pigs' limbs and the firmness of the muscles in the buttocks; the higher the comprehensive score of the muscle feel, the better the muscle feel; The comprehensive score of the pig's abdomen is used to evaluate the pig's body compactness, the straightness of the abdominal line, the smoothness of the skin and the thickness of the abdominal fat; the higher the comprehensive score of the pig's abdomen, the more compact the pig's abdomen is, and the higher the lean meat rate of its offspring; The comprehensive score of the pig's body symmetry is used to evaluate the overall appearance proportions and coordination of various parts of the pig; the higher the comprehensive score of the pig's body symmetry, the healthier the pig grows.

9. A live pig rating system based on three-dimensional point cloud, using a live pig rating method based on three-dimensional point cloud as claimed in any one of claims 1 to 8, characterized in that: include: Preprocessing unit: used to collect the initial point cloud of live pigs and perform preprocessing; Point cloud segmentation unit: used to segment the pre-processed initial point cloud into five parts: head and cheeks, trunk, limbs, buttocks and abdomen using the pre-trained DynEdgeNet point cloud segmentation model; Comprehensive scoring unit: used to extract the features of each part of the point cloud obtained by segmentation using the pre-trained TransFusionNet deep learning network, and calculate the comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry according to the features of each part of the point cloud; Pig rating unit: It is used to input the comprehensive scores of the pig's head and cheeks, muscularity, abdomen and body symmetry into the pre-trained GradingInteractionNet classification network to obtain the rating of the live pigs.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 8 are implemented.

Images