A method for automatically measuring the body dimensions of livestock from a farming scene

Abstract

The application provides a method for automatically measuring livestock body size in a breeding scene, comprising the steps of: S1: collecting scene point cloud data of the breeding scene, wherein the breeding scene contains at least a target livestock; S2: inputting the scene point cloud data into a pre-trained livestock segmentation model to obtain complete point cloud data of the target livestock, wherein the livestock segmentation model is trained by using labeled target livestock point cloud data and other scene point cloud data; S3: inputting the complete point cloud data of the target livestock into a pre-trained part segmentation model to obtain part point cloud data of each anatomical part of the target livestock, wherein the part segmentation model is trained by using livestock point cloud data labeled with known part point cloud data; and S4: performing posture normalization on the part point cloud data of each anatomical part to obtain standard point cloud data, and calculating each body size parameter of the target livestock based on the standard point cloud data, so as to obtain livestock body size data that can adapt to complex environments, realize full-process automation, and achieve high-precision measurement.

Description

Technical Field

[0001] This invention relates to the field of agricultural informatization, and in particular to a method for automatically measuring the body size of livestock in a breeding setting. Background Technology

[0002] In the field of precision livestock farming, the efficient and accurate acquisition of livestock body size parameters is the core foundation for achieving growth monitoring, health management, and breeding optimization. Among them, key body size indicators such as body length, body height, and chest circumference are directly related to core farming processes such as weight estimation and feed conversion ratio assessment. The automation upgrade of their measurement technology is of great significance for improving farming efficiency and promoting the intelligent transformation of the industry.

[0003] Currently, body size measurement in the industry mainly relies on two traditional methods, both of which have significant limitations: First, manual contact measurement, which involves manual operation using tools such as measuring tapes and rods. This method is not only inefficient and difficult to adapt to the batch testing needs of large-scale farming scenarios, but also highly susceptible to the influence of the operator's experience and technique, making it difficult to guarantee consistency and more likely to cause stress reactions in livestock, posing risks to animal welfare and measurement safety. Second, point cloud segmentation measurement based on traditional algorithms. This method is only suitable for controlled scenarios with a simple environment and no interference. In real farming environments, due to complex factors such as obstruction by farming facilities, interference from personnel movement, and random changes in livestock posture, the robustness of the model decreases significantly, and problems such as incomplete point cloud segmentation and feature extraction deviations are very likely to occur, ultimately leading to body size measurement errors that exceed the practical range.

[0004] In recent years, breakthroughs in point cloud processing using deep learning technology have provided new directions for non-contact measurement. However, existing deep learning-based livestock body size measurement solutions still face key technical bottlenecks:

[0005] Firstly, pre-segmentation relies on traditional algorithms. Most model training and validation are based on pre-processed "clean" single livestock point cloud data. In actual applications, traditional algorithms are needed to initially extract targets from complex scenes. This pre-processing step is prone to problems such as incomplete segmentation and blurred boundaries in the complex environment of the breeding scene. The error is directly transmitted to the subsequent part segmentation process, resulting in a significant decrease in the accuracy of part recognition.

[0006] Secondly, the posture adaptability is poor. Existing solutions mostly rely on single principal component analysis or simple geometric constraints to establish a measurement coordinate system, which is not adaptable to the random orientation and standing posture of livestock in the scene. It is easy to make errors in the coordinate system axis definition, which in turn leads to deviations in the calculation of body size parameters.

[0007] Third, feature fusion is insufficient. Traditional networks struggle to effectively balance the fusion efficiency of local geometric details and global semantic information in point clouds. Under complex background interference, the accuracy of boundary recognition between livestock and the environment, as well as between different parts of livestock, is insufficient, further exacerbating measurement errors.

[0008] The aforementioned technical deficiencies make it difficult for existing measurement schemes to meet the practical needs of real-world farming scenarios, thus hindering the large-scale implementation of precision livestock farming technology. Summary of the Invention

[0009] The purpose of this invention is to provide a method for automatically measuring the body size of livestock in a breeding setting, which can adapt to complex environments, achieve full-process automation, and obtain high-precision measurements of livestock body size.

[0010] To achieve the above objectives, this technical solution provides a method for automatically measuring the body size of livestock in a farming setting, comprising the following steps:

[0011] S1: Collect point cloud data of the breeding scene, which must contain at least the target livestock;

[0012] S2: Input the scene point cloud data into the pre-trained livestock segmentation model to obtain the complete point cloud data of the target livestock. The livestock segmentation model is trained using labeled target livestock point cloud data and other scene point cloud data.

[0013] S3: Input the complete point cloud data of the target livestock into the pre-trained part segmentation model to obtain the part point cloud data of each anatomical part of the target livestock. The part segmentation model is trained using livestock point cloud data labeled with known part point cloud data.

[0014] S4: Normalize the pose of the point cloud data of each anatomical part to obtain standard point cloud data, and calculate various body size parameters of the target livestock based on the standard point cloud data.

[0015] The livestock segmentation model and the part segmentation model share the same basic architecture, both including an encoder and a decoder connected in sequence. The encoder contains multiple levels of encoding modules, and the decoder contains multiple levels of decoding modules. Each level of encoding module and the corresponding level of decoding module are connected in a skip connection. Each encoding module contains a feature learning layer and a geometric perception global-local interaction layer, and each decoding module contains a feature propagation layer and a boundary perception channel attention layer.

[0016] Compared with existing technologies, this technical solution has the following characteristics and beneficial effects:

[0017] In terms of adaptability to complex environments, a two-level segmentation strategy of "scene point cloud → livestock point cloud → body part point cloud" is adopted. First, the pre-trained livestock segmentation model extracts the target livestock point cloud from the scene point cloud containing breeding facilities and personnel. Then, the body part segmentation model accurately segments the point clouds of 8 anatomical parts such as head, neck and chest, and limbs. Both models adopt an "encoder + decoder" architecture. The encoder strengthens feature extraction through the feature learning layer and the geometric perception global-local interaction layer, while the decoder optimizes the segmentation accuracy with the help of the feature propagation layer and the boundary perception channel attention layer, effectively resisting interference from occlusion, noise and changes in livestock posture.

[0018] In terms of automation efficiency, from collecting scene point clouds from multiple devices and angles, to processing part point clouds through posture normalization, and then to calculating indicators such as body length and height based on standard point clouds, the entire process requires no manual intervention, completely replacing inefficient manual contact measurement and solutions that rely on traditional pre-segmentation.

[0019] In terms of measurement accuracy, not only is error propagation avoided through layered segmentation, but also anatomically adapted calculation logic is designed for each body size index. Combined with targeted data enhancement during training, the model's generalization ability is improved, ensuring the consistency and high accuracy of measurement data in different scenarios, and providing reliable body size data support for precision animal husbandry. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a method for automatically measuring the body size of livestock in a farming setting according to an embodiment of the present invention.

[0021] Figure 2 This is a network diagram illustrating livestock component segmentation according to an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of complete livestock point cloud segmentation according to an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram illustrating the segmentation of eight key parts of a complete livestock point cloud according to an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram illustrating the location and measurement of key points for the body length of livestock according to an embodiment of the present invention.

[0025] Figure 6 This is a schematic diagram illustrating the location and measurement of key points on the body height of livestock according to an embodiment of the present invention;

[0026] Figure 7 This is a schematic diagram illustrating the location and measurement of key points for the body width and hip width of livestock according to an embodiment of the present invention.

[0027] Figure 8 This is a schematic diagram illustrating the location and measurement of key points for the hip height of livestock according to an embodiment of the present invention.

[0028] Figure 9 This is a schematic diagram illustrating the location and measurement of key points for the chest circumference of livestock according to an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0030] Those skilled in the art should understand that, in the disclosure of this invention, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limiting this invention.

[0031] Example 1

[0032] like Figure 1 As shown, this solution provides a method for automatically measuring the body size of livestock in a farming setting, including the following steps:

[0033] S1: Collect point cloud data of the breeding scene, which must contain at least the target livestock;

[0034] S2: Input the scene point cloud data into the pre-trained livestock segmentation model to obtain the complete point cloud data of the target livestock. The livestock segmentation model is trained using labeled target livestock point cloud data and other scene point cloud data.

[0035] S3: Input the complete point cloud data of the target livestock into the pre-trained part segmentation model to obtain the part point cloud data of each anatomical part of the target livestock. The part segmentation model is trained using livestock point cloud data labeled with known part point cloud data.

[0036] S4: Normalize the pose of the point cloud data of each anatomical part to obtain standard point cloud data, and calculate various body size parameters of the target livestock based on the standard point cloud data.

[0037] The livestock segmentation model and the part segmentation model share the same basic architecture, both including an encoder and a decoder connected in sequence. The encoder contains multiple levels of encoding modules, and the decoder contains multiple levels of decoding modules. Each level of encoding module and the corresponding level of decoding module are connected in a skip connection. Each encoding module contains a feature learning layer and a geometric perception global-local interaction layer, and each decoding module contains a feature propagation layer and a boundary perception channel attention layer.

[0038] Regarding step S1 of this plan:

[0039] This solution collects scene point cloud data of the livestock farming scene by deploying a 3D point cloud acquisition device in the scene. Specifically, the 3D point cloud acquisition device scans the farming scene containing the target livestock to obtain the corresponding scene point cloud data. Since the farming scene contains at least the target livestock, the scene point cloud data includes the point cloud data of the target livestock.

[0040] Furthermore, this solution is applicable to the measurement of livestock body size in complex breeding scenarios. Therefore, the breeding scenario will also contain breeding equipment and breeding personnel. Correspondingly, the point cloud data collected from the breeding scenario will contain point cloud data of breeding equipment and breeding personnel.

[0041] In some embodiments, when training the livestock segmentation model and the part segmentation model, the collected scene point cloud data can be enhanced by applying data augmentation techniques such as rotation, translation and introducing noise to the original dataset, so as to effectively improve the generalization performance of the subsequent livestock point cloud segmentation and recognition model against environmental interference and posture changes.

[0042] Furthermore, for common livestock postures (standing, head down, sideways), random rotations are performed within a 3D space at ranges of ±15° on the X-axis (body length direction), ±10° on the Y-axis (body height direction), and ±5° on the Z-axis (body width direction) to avoid excessive rotation that could lead to unrealistic posture data and ensure that the augmented data conforms to the physiological and structural characteristics of livestock. Random translations are performed in the X, Y, and Z axes within a range of ±0.5m to simulate changes in the distance between the data acquisition device and the livestock. Gaussian noise is used for noise enhancement, and noise is only added to non-critical areas to avoid feature distortion in the livestock point cloud due to noise interference.

[0043] In some embodiments, since the breeding scenario of this solution is relatively complex, a multi-device, multi-angle deployment of 3D point cloud acquisition devices can be used to obtain comprehensive scene point cloud data. For example, with the target livestock activity area as the center, 2-4 3D point cloud acquisition devices can be arranged at an angle of 30°-45° in the horizontal direction, and 3D point cloud acquisition devices can be deployed at heights of 0.5m and 1.5m above the ground in the vertical direction, respectively, to cover the top, sides, and bottom blind spots of the livestock's body, avoiding point cloud loss due to occlusion from a single viewpoint.

[0044] After acquiring scene point cloud data, this solution first inputs the scene point cloud data into a pre-trained livestock segmentation model to obtain the corresponding target livestock point cloud data. Then, it further inputs the target livestock point cloud data into a pre-trained part segmentation model to obtain the part point cloud data of each anatomical part of the target livestock. The advantage of this approach is that it can be well adapted to complex farming scenarios. First, it filters out background interference (such as farming facilities and personnel) with a high proportion through semantic segmentation, simplifying the task to binary classification to efficiently purify the livestock point cloud data. Then, it focuses on the details of the parts for the pure livestock point cloud data, greatly reducing the complexity of the single-step task. Moreover, it can avoid the accumulation of errors in the point cloud recognition process. This solution first ensures that the livestock point cloud is complete and without redundancy, and then accurately identifies the parts on this basis, preventing the error of misjudging the scene-livestock boundary in one-step segmentation from being transmitted to the part recognition stage, thus ensuring progressive segmentation accuracy.

[0045] Furthermore, this solution optimizes the basic architecture of both the livestock segmentation model and the body part segmentation model. The basic architecture of both models is identical to ensure continuous and powerful feature learning and discrimination capabilities for complex physiological structures in livestock (such as limb connections and trunk contours). The only difference is that, because the body part segmentation model has higher requirements for local details, some parameters of the network in the body part segmentation model (such as the number of feature channels and neighborhood search radius) are adaptively adjusted during the training phase, and it is trained independently on the corresponding labeled data.

[0046] The basic architecture of the livestock segmentation model and the part segmentation model will be introduced below:

[0047] Figure 2 This is a schematic diagram illustrating the segmentation of point cloud data for a specific part, such as... Figure 2 As shown, the livestock segmentation model or part segmentation model includes an encoder and a decoder connected in sequence. The encoder has multiple levels of encoding modules, and the decoder has multiple levels of decoding modules. Each level of encoding module and the corresponding level of decoding module are connected in a skip connection. Each encoding module contains a feature learning layer and a geometric perception global-local interaction layer, and each decoding module contains a feature propagation layer and a boundary perception channel attention layer.

[0048] It is worth mentioning that the encoder in this scheme constructs a geometrically perceptive global-local interaction layer to enhance the initial features, thereby improving the fusion capability between local geometric features of the point cloud and global semantic information. The decoder part of this scheme utilizes distance interpolation and hierarchical skip connections to construct a feature propagation layer. Furthermore, to enhance the accuracy of boundary recognition between the target livestock and the background environment, as well as between different parts of the livestock, the feature vectors are further input into the boundary-aware channel attention module to improve the segmentation of boundary regions.

[0049] Regarding the encoding module:

[0050] The feature learning layer in the encoding module is used to accurately capture the local geometric features and structural information of the input point cloud data. The feature learning layer sequentially performs farthest point sampling, octree partitioning, multi-layer shared MLP, and pooling on the input features to obtain the initial point cloud features for output. Specifically, the features input to the feature learning layer are sampled from the farthest point to obtain several centroids. These centroids are then partitioned into multiple groups using an octree to obtain multiple grouped neighborhoods. After learning local features through a multi-layer shared MLP, these grouped neighborhoods are pooled to obtain the initial point cloud features corresponding to the center point of each grouped neighborhood.

[0051] In some embodiments, the feature learning layer includes an average / max pooling layer for pooling processing.

[0052] Specifically, regardless of whether it's a livestock segmentation model or a part segmentation model, the features input to the feature learning layer are all point cloud data. The purpose of farthest point sampling is to select representative centroids from the input point cloud data, reducing the amount of data while retaining key geometric features, thus providing efficient input for subsequent local feature learning. The purpose of octree partitioning is to ensure that each centroid corresponds to a subset of point clouds containing local geometric information. The purpose of multi-layer shared MLP is to encode the features of the local point clouds in each group neighborhood, transforming the 3D coordinate information into a high-dimensional feature vector. The purpose of pooling is to generate the initial point cloud features corresponding to each centroid by aggregating the high-dimensional features of the group neighborhood.

[0053] Specifically, in the process of “the features input into the feature learning layer are sampled from the farthest point to obtain several centroids”, an initial centroid is randomly selected from the input features, the minimum Euclidean distance from the remaining points to the selected centroid is calculated, the point with the largest distance is selected as the next centroid, and the calculation process is repeated to obtain multiple centroids.

[0054] In the process of "grouping several centroids into multiple grouped neighborhoods by partitioning them into octrees", the extreme values ​​of the X, Y, and Z axis coordinates of the input point cloud data in 3D space are calculated to determine the overall bounding box of the point cloud. The bounding box is recursively divided into 8 sub-cubes of equal volume until the number of point clouds contained in each sub-cube does not exceed a preset threshold. For each centroid, the octree sub-cube in which it is located is located, and all point clouds in the sub-cube are divided into corresponding grouped neighborhoods. Finally, the set of grouped neighborhoods corresponding one-to-one with the centroid is output.

[0055] In the step of "learning local features from multiple grouped neighborhoods through a multi-layer shared MLP and then performing pooling to obtain the initial point cloud features of the center point of each grouped neighborhood", the point cloud coordinates in each grouped neighborhood are converted into relative coordinates with the centroid as the origin. Multi-layer convolution is used to process the relative coordinates to obtain the high-dimensional encoding of the grouped neighborhood. Then, the high-dimensional encoding output by the multi-layer shared MLP is processed by average pooling and max pooling to obtain the initial point cloud features. All grouped neighborhoods share the same set of MLP parameters.

[0056] The geometry-aware global-local interaction layer in the encoding module is used to deeply mine the geometric attributes in the initial point cloud features to enhance the recognition of local details and avoid confusion with similar structures. At the same time, it can also establish the association between local features and global semantic information (such as the overall outline of livestock and spatial relationships of parts) to ensure that local features conform to the global structural logic. Meanwhile, through geometric invariance and global correction, it improves the robustness of features against noise and pose changes, providing more reliable enhanced features for subsequent segmentation.

[0057] Specifically, the geometry-aware global-local interaction layer includes a geometry-aware feature enhancement module and a global-local interaction module connected in sequence. The geometry-aware feature enhancement module performs density adaptation, local covariance calculation, singular value decomposition calculation, HOG feature calculation, and self-attention mechanism processing on the input initial point cloud features in sequence to obtain enhanced point features. The global-local interaction module performs global feature processing and spatial-channel attention processing on the downsampled points output by the feature learning layer to obtain global features. The global features and enhanced point features are then subjected to cross-attention processing to obtain encoded features.

[0058] That is, the initial downsampled point cloud input to the geometric perception feature enhancement module dynamically determines the neighborhood of each center point through an adaptive density perception function, then constructs a local covariance matrix for the neighborhood at each scale, and performs singular value decomposition on the local covariance matrix to extract the feature vectors corresponding to the maximum and minimum eigenvalues ​​as the principal direction vector and normal vector, respectively. Based on the principal direction vector, the azimuth and elevation angles are calculated and differentiable HOG features are generated through Gaussian weighted soft binning mapping. After L1 normalization, a stable HOG descriptor is obtained. Finally, the initial point cloud features, principal direction vector, normal vector, and HOG descriptor are serialized and iteratively weighted and fused using a self-attention mechanism to form enhanced point features.

[0059] It should be noted that the enhanced point features generated by this scheme are rotation-invariant and noise-robust.

[0060] Furthermore, in the section "Dynamically determining the neighborhood of each center point using the adaptive density sensing function of the initial downsampled point cloud input to the geometric perception feature enhancement module", the neighborhood search radius of each point in the input initial downsampled point cloud is dynamically adjusted through the adaptive density sensing function, thereby calculating the neighborhood of each center point to define the precise range for subsequent local geometric feature extraction.

[0061] In the step of “constructing a local covariance matrix for each scale neighborhood”, the coordinate deviations of all points in each neighborhood relative to the center point are calculated, and a local covariance matrix is ​​constructed based on the coordinate deviations, where the matrix elements in the local covariance matrix are the covariances of the coordinate deviations on each coordinate axis.

[0062] In the step of “performing singular value decomposition on the local covariance matrix to extract eigenvectors corresponding to the largest and smallest eigenvalues ​​as principal direction vectors and normal vectors respectively”, singular value decomposition is performed on each local covariance matrix to obtain eigenvectors corresponding to the largest and smallest eigenvalues ​​as principal direction vectors and normal vectors respectively.

[0063] In the step of "calculating azimuth and elevation angles based on the principal direction vector and generating differentiable HOG features through Gaussian weighted soft bin mapping, and obtaining stable HOG descriptors after L1 normalization", the neighborhood space is divided into 8 angular intervals based on the principal direction vector. The gradient of each point relative to the center point is calculated and the gradient direction is mapped to the corresponding angular interval. The sum of the gradient magnitudes of each angular interval is counted to form HOG features. The HOG features are then L1 normalized to obtain stable HOG descriptors.

[0064] In the step of "serializing the initial point cloud features with the principal direction, normal vector and HOG descriptor and performing iterative weighted fusion using a self-attention mechanism to form enhanced point features", the initial point cloud features are serialized with the principal direction, normal vector and HOG descriptor and concatenated according to the channel dimension to obtain a multi-dimensional sequence feature vector. The multi-dimensional sequence feature vector is then input into the self-attention mechanism for iterative weighted fusion to form enhanced point features.

[0065] In addition, the downsampled points input to the global-local interaction module are processed to generate a set of global feature points. The global feature points are captured by multi-hop weighted aggregation of the graph attention network to extract the global context. The global context is modulated by spatial-channel attention to obtain global features. At the same time, the cross attention mechanism is used to enable the global features and the enhanced point features to achieve collaborative iterative interaction and fusion through learnable weights to obtain the encoded features.

[0066] Furthermore, in the step of "processing downsampling points input to the global-local interaction module to generate a set of global feature points", T representative points are selected from the downsampling points output by the feature learning layer through farthest point sampling to generate a set of global feature points.

[0067] In the section "Capturing the topological relationships between global feature points through multi-hop weighted aggregation of global feature points via graph attention network to extract global context", each global feature point is taken as a node, and the Euclidean distance between any two nodes is calculated. If the distance is less than a preset threshold, an edge is established between the two nodes. The edge weight is calculated based on the enhanced features corresponding to the node through graph neural network. Then, a multi-hop weighted strategy is used to fuse features based on the edge weight and the node to obtain the global associated features of each global feature point. All global associated features are integrated to form the global context.

[0068] It should be noted that the global feature points generated in this scheme are generated by mapping the sampling points to a feature extractor consisting of one-dimensional convolution, batch normalization and ReLU activation function, to generate high-dimensional global feature representations of global feature points. Then, the generated global feature points are captured by multi-hop weighted aggregation of graph attention network to extract the topological relationships between the global feature points and extract the global context.

[0069] In "Modulation of Global Context with Spatial-Channel Attention", channel statistics of the global context are extracted by global average pooling and global max pooling, input into MLP and activated by Sigmoid to generate channel attention weights. The channel-modulated features are then subjected to average pooling and max pooling along the channel dimension. Spatial attention weights are generated through convolutional layers, and the global context is finely modulated based on the channel attention weights and spatial attention weights.

[0070] In the section "Utilizing the cross-attention mechanism to achieve collaborative iterative interaction and fusion of global features and enhancement point features through learnable weights to obtain encoded features", the global features are projected into query vectors and value vectors, and the enhancement point features are projected into key vectors. The local-global attention weights are calculated using the scaling dot product attention formula. The local-global attention weights are used to perform a weighted summation of the value vectors to obtain the local features of global semantic enhancement. The local features of global semantic enhancement and the original enhancement point features are concatenated by channel to generate encoded features.

[0071] In some embodiments, the encoder includes multiple encoding modules. The output features of the previous encoding module are input into the next encoding module to construct a feature representation of multi-level, highly abstract features. Finally, combined with the decoder, semantic features of point cloud data that can distinguish different features can be obtained.

[0072] In some embodiments, the encoder includes three encoding modules.

[0073] About the decoding module:

[0074] The decoding module consists of a feature propagation layer and a boundary awareness channel attention layer. The feature propagation layer gradually restores the encoded features output by the encoding module into fine-grained features that match the scale of the original point cloud to achieve hierarchical feature restoration and cross-level information complementarity. The boundary awareness channel attention layer enhances the distinguishability of boundary features and the ability to resist interference.

[0075] Specifically, the feature propagation layer performs distance interpolation and splicing fusion on the input features. The boundary awareness channel attention layer processes the fused input features in parallel with boundary probability values ​​and channel attention to obtain boundary features and weights. The weighted boundary features and weights are used to obtain channel modulation weights. The input features are then weighted channel by channel based on the channel modulation weights to obtain the decoded features.

[0076] Furthermore, in the section "the feature propagation layer performs distance interpolation and concatenation on the input features to obtain fused features", for the high-level point cloud in the input feature propagation layer, a farthest point sampling strategy is used to select some points as the basis for upsampling, and the features of the high-level point cloud are interpolated according to their coordinate information to restore the feature representation at the corresponding position. The interpolated features of the high-level point cloud are then concatenated with the corresponding low-level features to fuse the features.

[0077] Furthermore, in the section "the boundary-aware channel attention layer processes the input fusion features in parallel with boundary probability values ​​and channel attention to obtain boundary features and weights, respectively; the weighted boundary features and weights are used to obtain channel modulation weights; and the input features are weighted channel-wise based on the channel modulation weights to obtain decoded features," the input fusion features are processed with channel attention to obtain weights. A parameter-independent boundary prediction subnetwork generates a boundary probability value for each point. The boundary probability value is fused with the weights to generate boundary-enhanced channel modulation weights. Finally, the decoded features are output by weighting the original features channel-wise based on the channel modulation weights.

[0078] The formula for calculating the weights is as follows:

[0079] ;

[0080] in The activation function is Sigmoid, MLP is a multilayer perceptron with shared parameters, and MaxPool and AvgPool represent global max pooling and average pooling, respectively. It is the fused feature output by the feature transfer layer.

[0081] Furthermore, the boundary probability value of each point refers to the likelihood that the point belongs to the component boundary, and is thus used to predict the implicit boundary probability.

[0082] Furthermore, in "fusing boundary probability values ​​and weights to generate boundary-enhanced channel modulation weights", the boundary probability map composed of boundary probability values ​​and the weight matrix composed of weights are multiplied element-wise to generate boundary-sensitive channel modulation weights.

[0083] Furthermore, in "channel-by-channel weighting of the original features based on channel modulation weights", the original features are reweighted channel by channel using channel modulation weights, so that the network pays more attention to the feature responses located in the prediction boundary region in the channel dimension, and finally outputs new features with enhanced boundary awareness.

[0084] In some embodiments, the decoder includes multiple decoding modules, and the output features of the previous decoding module are input into the next decoding module to construct a feature representation with multi-level abstraction capabilities, ultimately realizing the segmentation of target point cloud data from complex scenes.

[0085] In some embodiments, the decoder includes three decoding modules.

[0086] Regarding livestock segmentation models, these models are trained to extract livestock from complex farming scenarios, such as... Figure 3 As shown.

[0087] Regarding the part segmentation model, the part segmentation model is trained to output the head, neck and chest, abdomen, tail, left forelimb, right forelimb, left hindlimb, and right hindlimb of the target livestock.

[0088] In some embodiments, such as Figure 4 As shown, the part segmentation model is trained to output part labels for each 3D point, with different labels representing different parts. For example, the labels include: 0-head, 1-neck and chest, 2-abdomen, 3-tail, 4-left forelimb, 5-right forelimb, 6-left hindlimb, and 7-right hindlimb.

[0089] Furthermore, after acquiring the point cloud data of the anatomical parts, the point cloud data of each anatomical part is pose-normalized to obtain standard point cloud data. Measurement indicators for each anatomical part are then calculated based on the standard point cloud data. In other words, to achieve accurate and consistent body size measurement, the point cloud data of the segmented parts needs to be pose-normalized to place them in a unified standard measurement coordinate system to obtain standard point cloud data.

[0090] Correspondingly, step S4 further includes the following steps: using point cloud principal component analysis to analyze the point cloud data of each anatomical site to determine the initial coordinate system, and performing attitude normalization on the point cloud data of the site based on the ground normal vector to obtain standard point cloud data in the standard coordinate system, and calculating the measurement index of each anatomical site based on the standard point cloud data.

[0091] The steps for obtaining standard point cloud data in the standard measurement coordinate system are as follows:

[0092] Principal component analysis of point cloud was used to analyze the point cloud data of each anatomical part to determine the direction of the main axis of the animal's body. The centroid of the point cloud data of the part was used as the origin of the coordinate system, and the direction of the main axis of the animal's body was used to construct the initial coordinate system.

[0093] The random sampling consensus algorithm is used to estimate ground plane parameters based on the original point cloud data of different locations and to store the ground normal vector;

[0094] The correction results are obtained by standardizing and correcting the axis of the initial coordinate system and the head and tail directions of the livestock based on the ground normal vector;

[0095] The three-dimensional spatial transformation matrix is ​​updated based on the correction results, and then applied to the corresponding component point cloud data to uniformly transform the point cloud data to standard point cloud data in the standard measurement coordinate system.

[0096] In this embodiment of the scheme, the point cloud principal component analysis method is used to analyze the point cloud data of each anatomical part. The three-dimensional spatial transformation matrix of the point cloud data of the part from the initial coordinate system to the standard coordinate system is calculated by singular value decomposition. This three-dimensional spatial transformation matrix encapsulates the required rotation and translation information.

[0097] In this embodiment, point cloud principal component analysis (PCA) is used to analyze the point cloud data of each anatomical location to determine the length and height directions. The origin of the coordinate system is set at the centroid of the point cloud data, the length direction is used as the X-axis, the height direction as the Y-axis, and the width direction is determined using the right-hand rule. An initial coordinate system is then constructed with the width direction as the Z-axis. It should be noted that PCA can be used to determine the length direction as the first principal direction and the height direction as the second principal direction.

[0098] In the section "Standardizing and Correcting the Axis of the Initial Coordinate System and the Head and Tail Directions of Livestock Based on Ground Normal Vectors to Obtain Correction Results", the Y-axis of the initial coordinate system is corrected using ground normal vectors. If there is no part point cloud data on the Y-axis, the Y-axis is flipped and the 3D spatial transformation matrix is ​​updated. The 3D spatial transformation matrix is ​​applied to the part point cloud data with part labels. The distribution of the part point cloud data corresponding to the head and tail on the X-axis is detected. If the X-coordinate of the part point cloud data of the head is less than the X-coordinate of the part point cloud data of the tail, the X-axis is flipped and the 3D spatial transformation matrix is ​​updated.

[0099] In some embodiments, the ground normal vector is the maximum plane normal vector extracted from complex aquaculture scenarios using a random sample consensus algorithm.

[0100] Specifically, the body height axis of the initial coordinate system is corrected using the ground normal vector. The correctness of the axis is verified by checking whether there is livestock part point cloud data in the corrected body height direction. If no livestock part point cloud data is found in this direction, the axis containing the body height is flipped to ensure that the positive direction of the axis is perpendicular to the ground and points upwards towards the livestock, and the transformation matrix is ​​updated and saved. Similarly, to further unify the head and tail orientation, the distribution of head and tail point clouds on the body length axis needs to be checked. If the mean X-coordinate of the head point cloud is smaller than that of the tail point cloud, the body length axis is flipped to ensure that the livestock's head uniformly faces the positive X-axis direction and the tail faces the negative X-axis direction.

[0101] It should be noted that the standard point cloud data after the above standardization correction has uniformly converted the livestock point cloud into a standard coordinate system that conforms to anatomical orientation, providing a consistent geometric benchmark for subsequent body size parameter calculations.

[0102] Furthermore, after obtaining the standard point cloud data, one or more corresponding key parts of the standard point cloud data can be selected according to the measurement requirements of the target body size parameters, and key point positioning and geometric calculation can be performed to obtain the body size data of the livestock.

[0103] In some embodiments, measurement parameters for each anatomical region are calculated based on standard point cloud data, wherein the measurement parameters include any one of body length, body height, body width, hip height, hip width, and chest circumference.

[0104] The methods for obtaining each measurement indicator are as follows:

[0105] Body height measurement: The vertical distance between the highest point of the standard point cloud data of the neck and chest in the body height direction and the lowest point of the standard point cloud data of the forelimbs in the body height direction is calculated.

[0106] Specifically, body height is generally measured as the height of the apex of the scapula perpendicular to the ground. Therefore, the absolute difference between the highest point of the standard point cloud data of the neck and chest on the Y-axis and the lowest point of the standard point cloud data of the left and right forelimbs on the Y-axis is recorded as body height.

[0107] Specifically, such as Figure 6 As shown, the point with the largest Y-axis coordinate value in the standard point cloud data of the neck and chest is located as the scapula vertex. The points with the smallest Y-axis coordinate values ​​are located in the standard point cloud data of the left and right forelimbs respectively. The point with the smaller Y-value is taken as the ground reference point. The difference in Y-coordinate between the ground reference point and the scapula vertex is calculated as the body height. The calculation formula is as follows:

[0108] H1=y withers -y ground ,

[0109] Among them, y withers and y ground These are the y-coordinates of the scapula vertex and the ground reference point, respectively.

[0110] Body length measurement: The projection distance between the center points of the bilateral ear regions selected from the standard point cloud data of the head and the center point of the tail root region selected from the standard point cloud data of the tail is calculated on the plane formed by the body length and body width.

[0111] Specifically, body length is the length measured along the spine from the midpoint of the line connecting the base of the animal's two ears to the first natural ring mark at the base of the tail, using a measuring tape. Therefore, body length can be measured by determining the midpoint between the ears and the base of the tail. The midpoint between the ears is calculated by selecting the approximate positions of the ears and then determining the geometric center point. The base of the tail is calculated by selecting the approximate position of the tail. The Euclidean distance between the midpoint between the ears and the base of the tail is recorded as the body length value.

[0112] Furthermore, such as Figure 5 As shown, for the standard point cloud data of the head, the points are sorted in descending order according to the Y-axis, and the top 50% of the points are selected. Within this subset, points with X-coordinates in the 60% to 80% range are selected to form a candidate point set for the bilateral ear region. The geometric center of the candidate point set for the bilateral ear region is calculated as the midpoint of the ear root. The calculation formula is as follows:

[0113] ;

[0114] in , represents the candidate point set for the binaural region; C ear The midpoint of the ear root, (x i ,y i (i, zi) represents the coordinates of the i-th point, and N represents the total number of points.

[0115] For the standard point cloud data at the tail, sort the points in descending order according to the Y coordinate and select the first 70% of the points. From the point subset, select the 5% of points with the smallest X coordinate values ​​to form the candidate point set for the tail root region. Calculate the geometric center of the candidate point set for the tail root region as the tail root midpoint Ctail.

[0116] Project the midpoint of the ear root and the tail root onto the ZOX plane, and calculate the Euclidean distance between the two points on the two-dimensional plane as the final body length. The calculation formula is as follows: ;

[0117] Where (x) ear ,z ear ) and (x tail ,z tail ) are the projected coordinates of the midpoint of the ear root and the midpoint of the tail root on the ZOX plane, respectively.

[0118] Body width measurement: Based on the distribution of standard point cloud data of the neck and chest in the body width direction, calculate its maximum span in that direction.

[0119] Specifically, such as Figure 7 As shown, body width is defined as the maximum span of the torso at the apex of the scapula in the body width direction. Therefore, for the standard point cloud data of the neck and chest obtained after segmentation, the points are first sorted in descending order according to the Y-axis coordinate values, and the top 80% of the points are selected to eliminate interference from point cloud regions far from the scapula. Then, the maximum and minimum coordinate values ​​of the point subset in the Z-axis direction are calculated, and the absolute difference between the two is used as the body width value. The calculation formula is as follows:

[0120] W1=z max -z min Among them, z max and z min These are the maximum and minimum coordinates of the point subset along the Z-axis, respectively.

[0121] Hip height measurement: Based on the spatial relationship of standard point cloud data of hind limbs and tail, the hip reference vertex is located, and the vertical distance between the hip reference vertex and the lowest point of the hind limb is calculated.

[0122] Specifically, hip height is defined as the vertical height from the highest point of the livestock's tail (excluding the tail itself) to the ground. Therefore, based on the standard point cloud data of the left and right hind limbs obtained through segmentation, the top 10% of the points in the Y coordinates of the left and right hind limbs are selected to form a top candidate point set. The geometric center points of the top candidate point sets corresponding to the left and right hind limbs are calculated, and the midpoints of the line segments between the two geometric center points are determined.

[0123] like Figure 8 As shown, a spatial straight line is drawn along the Y-axis from the midpoint of this line segment. The intersection of this spatial straight line and the standard point cloud data of the tail is taken as the initial highest candidate point of the tail. Using the initial highest candidate point of the tail as a reference point, a higher local extremum point is searched within the cylindrical neighborhood of this reference point as the hip reference vertex. Specifically, with the initial highest candidate point of the tail M as the center, a cylindrical search region with a radius of 1 cm and extending 2 cm along the positive Y-axis is constructed. Within the cylindrical search region, it is checked whether there is a local extremum point M1 with a Y coordinate higher than M. If it exists, the new local extremum point M1 is used as the hip reference vertex; otherwise, the initial highest candidate point of the tail M is used as the hip reference vertex.

[0124] In the standard point cloud data of the left and right hind limbs, locate the point with the smallest Y-coordinate. Take the point with the smaller Y-value as the ground reference point. Calculate the absolute difference between the Y-coordinate of the hip reference vertex and the ground reference point as the hip height value. The calculation formula is as follows:

[0125] H2=y rump -y ground_hip , where =y rump With y ground_hip These are the Y-coordinates of the hip reference vertex and the ground reference point, respectively.

[0126] Hip width measurement: Based on the distribution of standard point cloud data of the tail in the body width direction, calculate its maximum span in that direction.

[0127] Specifically, hip width is calculated using a method similar to body width. The standard point cloud data of the tail is sorted in descending order along the Y-coordinate, and the first 80% of the points are selected. The maximum and minimum coordinate values ​​of the points in the Z-axis direction are taken, and the absolute difference between the two Z-coordinates is used as the hip width value. The calculation formula is as follows:

[0128] W2=z max -z min Among them, z max and z min These are the maximum and minimum coordinate values ​​along the Z-axis, respectively.

[0129] Chest circumference measurement: Based on the standard point cloud data of the forelimbs, the cross-sectional plane of the torso is determined, and the chest circumference contour point cloud slices are obtained. The chest circumference value is obtained by curve fitting and perimeter calculation.

[0130] Specifically, such as Figure 9 As shown, chest circumference is defined as the vertical circumference of the chest at the posterior edge of the scapula of a domestic animal. Correspondingly, the standard point cloud data of the left and right forelimbs are sorted in descending order of Y-coordinate. The top 10% of the points with the largest Y-coordinates in the standard point cloud data of the left and right forelimbs are selected to form a top candidate point set P. The point with the smallest X-coordinate in the top candidate point set P is denoted as Pi. left With P right ; Pass P left With P right A plane parallel to the Y-axis is used as the reference cutting plane for chest measurement. The reference cutting plane for chest measurement is extended along the X-axis, and a slice of point cloud intersecting with the torso point cloud is extracted. The contour formed by this slice of point cloud is the chest contour. The slice of point cloud is projected onto the ZOY plane, and the NURBS curve is used to fit the projected contour points to obtain a smooth and closed chest curve. The perimeter of the fitted curve is calculated as the final chest value.

[0131] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method of automatically measuring the body dimensions of livestock from a farming scene, characterized by, The method comprises the following steps: S1: collecting scene point cloud data of a breeding scene, wherein the breeding scene at least contains target livestock; S2: inputting the scene point cloud data into a pre-trained livestock segmentation model to obtain complete point cloud data of the target livestock, wherein the livestock segmentation model is trained by using labeled target livestock point cloud data and other scene point cloud data; S3: inputting the complete point cloud data of the target livestock into a pre-trained part segmentation model to obtain part point cloud data of each anatomical part of the target livestock, wherein the part segmentation model is trained by using livestock point cloud data labeled with known part point cloud data; S4: performing pose normalization on the part point cloud data of each anatomical part to obtain standard point cloud data, and calculating measurement indexes of each anatomical part of the target livestock based on the standard point cloud data; The livestock segmentation model and the part segmentation model have the same basic architecture, and each comprises an encoder and a decoder connected in sequence, wherein the encoder has a plurality of levels of encoding modules, the decoder comprises a plurality of levels of decoding modules, each level of encoding module is connected to the corresponding level of decoding module by jumping, each encoding module contains a feature learning layer and a geometry-aware global-local interaction layer, and each decoding module contains a feature propagation layer and a boundary-aware channel attention layer; The feature propagation layer performs distance interpolation and splicing fusion on the input features; the boundary-aware channel attention layer performs boundary probability value and channel attention processing on the input fused features in parallel to obtain boundary features and weights, respectively, and obtains channel modulation weights by weighting the boundary features and the weights, and obtains decoding features by weighting the input features channel by channel based on the channel modulation weights; The feature learning layer is used for accurately capturing the local geometric features and structural information of the input point cloud data, wherein the feature learning layer sequentially performs farthest point sampling, octree division, multi-layer shared MLP and pooling processing on the input point cloud data to obtain initial point cloud features and down-sampled points; The geometry-aware global-local interaction layer is used for deep mining of the geometric properties in the initial point cloud features, and the geometry-aware global-local interaction layer comprises a geometry-aware feature enhancement module and a global-local interaction module connected in sequence, wherein the geometry-aware feature enhancement module sequentially performs density self-adaptation, local covariance calculation, singular value decomposition calculation, HOG feature calculation and self-attention mechanism processing on the input initial down-sampled points to obtain enhanced point features, the global-local interaction module performs global feature processing and spatial-channel attention processing on the down-sampled points output by the feature learning layer to obtain global features, and the global features and the enhanced point features are cross-attention processed to obtain encoding features.

2. The method of automatically measuring livestock body dimensions from a farming scene according to claim 1, characterized in that, Randomly select an initial centroid point from the input point cloud data, calculate the minimum Euclidean distance from the remaining points to the selected centroid point, select the point with the maximum distance as the next centroid point and repeat the calculation process to obtain multiple centroid points, calculate the X, Y and Z axis coordinate extreme values of the input point cloud data in three-dimensional space to determine the overall bounding box of the point cloud, recursively divide the bounding box into 8 equal-volume sub-cubes until the number of point clouds contained in each sub-cube is not greater than a preset threshold, for each centroid point, locate the octree sub-cube it is in, divide all point clouds in the sub-cube into corresponding grouped neighborhoods, and finally output a set of grouped neighborhoods corresponding to the centroid points, convert the point cloud coordinates in each grouped neighborhood into relative coordinates with the centroid point as the origin, process the relative coordinates using multi-layer convolution to obtain high-dimensional encoding of the grouped neighborhood, and average pool and maximum pool the high-dimensional encoding output by the multi-layer shared MLP to obtain the initial point cloud feature, wherein all grouped neighborhoods share the same set of MLP parameters.

3. The method of automatically measuring livestock body dimensions from a farming scene according to claim 1, characterized in that, The initial down-sampled points input into the geometry-aware feature enhancement module dynamically determine the neighborhood of each center point through an adaptive density-aware function, construct a local covariance matrix for each scale of neighborhood, and perform singular value decomposition on the local covariance matrix to extract the eigenvectors corresponding to the maximum and minimum eigenvalues as the principal direction vector and normal vector, respectively, calculate the azimuth and elevation based on the principal direction vector, and generate differentiable HOG features through Gaussian weighted soft binning mapping, obtain stable HOG descriptors after L1 normalization, and finally serialize the initial point cloud feature, principal direction, normal vector and HOG descriptor sequence to iteratively and weightedly fuse enhanced point features using a self-attention mechanism, and input the enhanced point features into a global-local interaction module. The module generates a set of global feature points by performing global feature processing on the down-sampled points output by the feature learning layer, captures the topological relationship between global feature points through multi-hop weighted aggregation of the graph attention network to extract global context, and obtains global features by modulating the global context using spatial-channel attention. The global features and enhanced point features are iteratively and cooperatively fused through learnable weights to obtain encoded features. In "the global feature points capture the topological relationship between global feature points through multi-hop weighted aggregation of the graph attention network to extract global context", the channel statistical information of the global context is extracted through global average pooling and global maximum pooling, input into an MLP and activated by a Sigmoid function to generate channel attention weights, the features modulated by the channels are average-pooled and maximum-pooled along the channel dimension, spatial attention weights are generated through a convolution layer, and the global context is refined based on the channel attention weights and spatial attention weights.

4. The method of automatically measuring livestock body dimensions from a farming scene according to claim 3, characterized in that, ​ In "Global feature and enhanced point feature are cooperatively and iteratively fused through learnable weights to obtain encoded features by using cross-attention mechanism", the global feature is projected into a query vector and a value vector, and the enhanced point feature is projected into a key vector. The local-global attention weight is calculated by the scaled dot-product attention formula, and the value vector is weighted and summed by the local-global attention weight to obtain the locally enhanced global semantic feature. The locally enhanced global semantic feature and the original enhanced point feature are spliced by channel to generate the encoded feature.

5. The method of automatically measuring livestock body dimensions from a farming scene according to claim 1, wherein, For high-level point clouds in the features of the input feature propagation layer, a farthest point sampling strategy is used to select part of the points as the basis for upsampling, and the features of the high-level point clouds are interpolated according to their coordinate information to restore the feature representation at the corresponding position. The interpolated features of the high-level point clouds are spliced and fused with the features of the corresponding low-level point clouds. The boundary perception channel attention layer performs channel attention calculation on the input fused features to obtain weights. A boundary prediction subnetwork independent of parameters is used to generate a boundary probability value for each point based on the input features. The boundary probability value and the weights are fused to generate boundary-enhanced channel modulation weights. Finally, based on the channel modulation weights, the original features are weighted channel by channel to output the decoded features.

6. The method of automatically measuring livestock body dimensions from a farming scene according to claim 1, wherein, The point cloud principal component analysis method is used to analyze the part point cloud data of each anatomical part to determine the initial coordinate system, and the part point cloud data is normalized in pose based on the ground normal vector to obtain the standard point cloud data in the standard coordinate system. The measurement indicators of each anatomical part are calculated based on the standard point cloud data.

7. The method of automatically measuring livestock body dimensions from a farming scene according to claim 1, wherein, The part segmentation model is trained to output the head, neck and chest, abdomen, tail, left forelimb, right forelimb, left hind limb, and right hind limb of the target livestock; Body height measurement: based on the highest point of the standard point cloud data of the neck and chest in the body height direction, and the lowest point of the standard point cloud data of the forelimb in the body height direction, the vertical distance between the two is calculated; Body width measurement: based on the distribution of the standard point cloud data of the neck and chest in the body width direction, the maximum span in this direction is calculated; Hip height measurement: based on the spatial relationship of the standard point cloud data of the hind limbs and the tail, the hip reference vertex is located, and the vertical distance between the hip reference vertex and the lowest point of the hind limb is calculated; Hip width measurement: based on the distribution of the standard point cloud data of the tail in the body width direction, the maximum span in this direction is calculated; Body length measurement: based on the center point of the double-ear region selected from the standard point cloud data of the head, and the center point of the tail root region selected from the tail point cloud, the projection distance of the two points on the plane formed by the body length and the body width is calculated; Chest circumference measurement: based on the standard point cloud data of the forelimb, the transverse plane of the torso is determined, the chest contour point cloud slice is obtained, and the chest circumference value is obtained through curve fitting and perimeter calculation. 8.An electronic device comprising a memory and a processor, the electronic device comprising: The memory stores a computer program, and the processor is configured to run the computer program to execute the method for automatically measuring the body size of livestock in the breeding scene according to any one of claims 1 to 7.

Images