A sheep body size key point detection method and system based on weighted neighborhood aggregation and geometric constraint correspondence
By using weighted neighborhood aggregation and geometric constraints, the problem of unstable localization of key points in the 3D point cloud of sheep was solved, and the consistency and interpretability of body size measurement under natural postures were improved, especially maintaining accuracy under posture changes such as head tilting and head turning.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NORTHWEST A & F UNIV
- Filing Date
- 2026-06-02
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies are unstable in locating key points in 3D point clouds of sheep in natural postures, lack consistency in cross-individual semantic correspondence, have poor interpretability of body size measurements, and are difficult to maintain accuracy under posture changes such as head tilting and head turning.
We employ a weighted neighborhood aggregation and geometric constraint method. The neighborhood aggregation encoder generates encoded features with neighborhood contribution weights. Combined with local position offset, surface normal and curvature information, we perform geometric description and generate a sequence of corresponding output points with a fixed semantic order to extract key points of sheep body size.
It improves the consistency of key point outputs across different individuals and postures, enhances the ability to express local structures, and improves the automation and interpretability of body size measurement.
Smart Images
Figure CN122435005A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of intelligent phenotyping of livestock, three-dimensional point cloud processing, and computer vision technology. In particular, it relates to a method and system for detecting key body size points of sheep body point clouds in natural postures, which can be used for non-contact measurement of body size parameters such as body length, body height, chest width, chest circumference, cross height, and abdominal width. Background Technology
[0002] Sheep body size parameters reflect individual growth and development, body condition, and production performance, serving as crucial data for breeding evaluation, feeding management, and health monitoring in large-scale farming. Traditional body size measurements typically rely on manual contact using tools such as measuring sticks and tape measures, which suffers from drawbacks such as high labor intensity, low efficiency, significant subjective errors, and a tendency to cause stress to animals.
[0003] With the development of depth cameras, 3D reconstruction, and point cloud learning technologies, using 3D point clouds to measure animal body size has gradually become a feasible solution. Among existing methods, one type of method directly regresses body size results from images or point clouds. Although the process is shorter, it is difficult to clearly explain the source of error. Another type of method first detects anatomical key points and then calculates body size parameters based on the spatial geometric relationship between the key points, which has better interpretability and traceability.
[0004] However, in natural farming environments, sheep often exhibit behaviors such as head-down posture, head-turning, and irregular standing postures. Point clouds are also affected by factors such as acquisition perspective, occlusion, sparse boundaries, noise, and local missing data. These factors can lead to significant local morphological differences in the same anatomical keypoint across different individuals and postures, reducing the stability of keypoint localization and cross-sample semantic consistency. Existing methods that rely solely on ordinary local feature aggregation or direct coordinate regression struggle to simultaneously address both local geometric representation and cross-posture semantic consistency.
[0005] Therefore, it is necessary to propose a key point detection method that can adaptively enhance key local structures in sheep 3D point clouds and improve the stability of corresponding sequences through geometric constraints, so as to improve the automation level and reliability of sheep body size measurement under natural posture. Summary of the Invention
[0006] The purpose of this invention is to provide a method and system for detecting key points of sheep body size based on weighted neighborhood aggregation and geometric constraints, in order to solve the problems of unstable key point localization, insufficient consistency of cross-individual semantic correspondence, and poor interpretability of body size measurement in natural pose sheep point clouds.
[0007] To achieve the above objectives, the present invention adopts the following technical solution: a method for detecting key points of sheep body size, comprising the following steps: S1. Obtain the three-dimensional point cloud data of the sheep to be tested, and perform foreground extraction, noise removal, coordinate unification and normalization on the point cloud to obtain the point cloud of the sheep to be tested.
[0008] S2. Sample the point cloud of the sheep body to be detected to form an input point set with a preset number of points, and establish neighborhood relationships for the points in the input point set.
[0009] S3. Input the input point set into the weighted neighborhood aggregation encoder to jointly model the local relationship features and relative spatial positions between the center point and the neighboring points, and generate encoded features with neighborhood contribution weights.
[0010] S4. Construct a geometric constraint corresponding weight generation process based on the encoded features, wherein a geometric description containing local position offset, surface normal and curvature information is constructed for the encoded points, and the geometric description and the encoded features are used together for corresponding weight estimation.
[0011] S5. The input point coordinates are weighted and combined according to the corresponding weights to generate an output corresponding point sequence with a fixed semantic order, so that the output point sequences of different sheep samples have consistent semantic meaning in the index dimension.
[0012] S6. Based on the semantic index set predetermined in the standard template, extract 8 sheep body scale key points from the output corresponding point sequence.
[0013] S7. Output the coordinates of key points based on the extracted sheep body size key points, and calculate or assist in calculating at least one of the following body size parameters: sheep body length, body height, chest width, chest circumference, cross height, and abdominal width. Beneficial effects
[0014] (1) This invention transforms the detection of key points of sheep body size into the problem of index extraction in the corresponding point sequence with fixed semantic order, avoiding cross-pose semantic drift caused by direct regression of a small number of key points, which is conducive to improving the consistency of key point output under different individuals and different poses.
[0015] (2) The present invention introduces neighborhood contribution weight in the local feature encoding stage, which can adaptively adjust the contribution of neighborhood points according to the feature relationship and spatial relationship between the center point and the neighborhood points, thereby enhancing the local expression ability of key structural regions such as the sheep's back line, scapular region, and abdominal contour.
[0016] (3) In the corresponding point generation stage, the present invention introduces geometric information such as local position offset, surface normal and curvature, so that the corresponding weight estimation is simultaneously constrained by deep features and local geometric structure, which helps to alleviate the key point positioning offset caused by posture changes such as head down and head turn.
[0017] (4) The eight key points of body size output by the present invention correspond clearly with the artificial body size measurement logic, which can further support the calculation of parameters such as body length, body height, chest width, chest circumference, cross height and abdominal width, and improve the interpretability and traceability of non-contact body size measurement results. Attached Figure Description
[0018] Figure 1 This is a flowchart of the sheep body size key point detection method of the present invention.
[0019] Figure 2 This is a schematic diagram of the overall structure of the sheep body size key point detection system and network of the present invention.
[0020] Figure 3 This is a schematic diagram of the weighted neighborhood aggregation encoder structure of the present invention.
[0021] Figure 4 This is a schematic diagram of the geometric constraint corresponding weight generation mechanism of the present invention.
[0022] Figure 5 This is a schematic diagram illustrating the key point extraction based on a fixed semantic index according to the present invention.
[0023] Figure 6 This is a schematic diagram illustrating the relationship between key points of the sheep's body and its body size parameters in this invention. Detailed Implementation
[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments. It should be understood that the following embodiments are only used to explain the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention; any equivalent substitutions or modifications made by those skilled in the art without departing from the concept of the present invention should fall within the scope of protection of the present invention.
[0025] A depth camera, structured light camera, binocular camera, or multi-view RGB-D acquisition device is used to acquire 3D point clouds of the sheep. Preferably, the acquisition device includes a top view and left and right side views to obtain point cloud data of the sheep's back, side contours, and torso boundary regions. After acquisition, background points are removed based on image segmentation, foreground segmentation, or depth thresholding, and noise points are removed using statistical outlier filtering. Coordinate unification or registration is then performed on the multi-view point clouds. Subsequently, the point clouds are scale-normalized, and a consistent set of input points with a spatial distribution that meets detection requirements is obtained using farthest point sampling or random sampling.
[0026] For each point in the input point set, it is used as the center point, and surrounding nearest neighbors are searched to form a local neighborhood. Then, by combining the center point features, neighborhood point features, feature differences between them, and relative spatial positions, neighborhood features describing local relationships are constructed. The encoder extracts neighborhood relationship information through shared feature mapping and uses a weight prediction unit to determine the importance of each neighboring point to the center point update. After normalizing the response results within the neighborhood of the same center point, the contribution weight of each neighboring point is obtained. Then, neighborhood information is aggregated based on this contribution weight to generate the local enhanced features after the center point update. In a preferred embodiment, the weighted neighborhood aggregation encoder consists of two layers of weighted edge convolutional structures connected in series, with a sampling layer between the two layers to reduce the number of points and expand the receptive field. The output features of the second layer establish a residual connection with the downsampled shallow features and undergo normalization to obtain the encoded features. The neighborhood size can be set according to the point cloud density. In a preferred embodiment, sixteen nearest neighbors are selected for aggregation for each center point.
[0027] For each encoded point, a local nearest neighbor set is established, and the location of its neighborhood center and its local spatial distribution are statistically analyzed. The local surface orientation and curvature are estimated based on the spatial distribution of the neighborhood points. Subsequently, the point's position offset relative to the neighborhood center, local surface orientation information, and approximate curvature information are combined into a geometric description for subsequent weight estimation. The weight generation process includes a feature branch and a geometric branch. The feature branch determines the semantic matching degree between the input point and each corresponding output point based on the encoded features, while the geometric branch determines the matching degree under local morphological constraints based on the geometric description. Then, the responses from the two branches are fused according to preset weights, and the fused responses of all input points are normalized to obtain the contribution of each input point to its corresponding output point.
[0028] After obtaining the corresponding weights, the input point coordinates are weighted and combined according to the contribution of each input point to the output corresponding point, generating a set of output corresponding points with a fixed semantic order. This sequence of output corresponding points is used to establish semantic correspondences between different sheep body samples, so that the same index position maintains similar anatomical semantics in different samples as much as possible.
[0029] A sheep body sample with standardized posture, complete structure, and high point cloud quality is selected as a standard template, and an index set corresponding to eight body size key points is determined from its output point sequence. For any input sheep body sample, after the network of this invention obtains the output point sequence, the corresponding output points are directly read according to the pre-determined semantic index positions to obtain the predicted key point set. The eight body size key points include the sternal node, withers apex, left posterior scapular angle, right posterior scapular angle, widest point of the left abdomen, widest point of the right abdomen, highest point of the buttocks, and ischial tuberosity node.
[0030] Based on the predicted key points, body size parameters can be further calculated or used for auxiliary calculations. Specifically, body length can be determined based on the spatial relationship between the sternal and ischial tuberosities; body height can be determined based on the height of the withers apex relative to the ground or a reference plane; chest width can be determined based on the lateral distance between the left and right posterior angle points of the scapula or their vicinity on the chest section; abdominal width can be determined based on the lateral distance between the widest points of the left and right abdomen; cruciate height can be determined based on the height of the highest point of the hip relative to the ground or a reference plane; and chest circumference can be determined by using the left and right posterior angle points of the scapula to help determine the position of the chest section, then extracting the chest section contour points and calculating the closed contour length, or by fitting the chest section contour and calculating the closed contour length.
[0031] This invention also provides a sheep body size key point detection system, including a point cloud acquisition module, a preprocessing module, a weighted neighborhood aggregation encoding module, a geometric constraint correspondence module, a semantic index key point extraction module, and a body size calculation module. Each module sequentially completes sheep body point cloud acquisition, standardized input generation, local structure enhancement, corresponding sequence generation, key point reading, and body size parameter output. The system can be deployed on a local workstation, server, or edge computing device, or it can be connected to a farm data management platform to achieve automatic recording and management of sheep body size parameters.
Claims
1. A method for detecting key points of sheep body size based on weighted neighborhood aggregation and geometric constraints, characterized in that, Includes the following steps: S1. Obtain the three-dimensional point cloud data of the sheep to be tested, and perform foreground extraction, noise removal, coordinate unification and normalization on the three-dimensional point cloud data to obtain the point cloud of the sheep to be tested. S2. Sample the point cloud of the sheep body to be detected to form an input point set with a preset number of points, and establish neighborhood relationships for the points in the input point set; S3. Input the input point set into the weighted neighborhood aggregation encoder to jointly model the local relationship features and relative spatial positions between the center point and the neighboring points, and generate encoded features with neighborhood contribution weights. S4. Construct a geometric constraint corresponding weight generation process based on the encoded features, wherein a geometric description containing local position offset, surface normal and curvature information is constructed for the encoded points, and the geometric description and the encoded features are used together for corresponding weight estimation. S5. The input point coordinates are weighted and combined according to the corresponding weights to generate an output corresponding point sequence with a fixed semantic order; S6. Based on the semantic index set predetermined in the standard template, extract the sheep body size key points from the output corresponding point sequence; S7. Output the coordinates of key points based on the extracted sheep body size key points, and calculate or assist in calculating at least one of the following body size parameters: sheep body length, body height, chest width, chest circumference, cross height, and abdominal width.
2. The method for detecting key points of sheep body size according to claim 1, characterized in that, In step S1, the three-dimensional point cloud data is obtained by a depth camera, a structured light camera, a binocular camera, or a multi-view RGB-D acquisition device. When a multi-view RGB-D acquisition device is used, depth data from the top view and the left and right side views of the sheep are acquired respectively, and the depth data is back-projected into a three-dimensional point cloud based on the camera intrinsic parameters. The foreground extraction, noise removal, coordinate unification, and normalization processing includes removing background points based on image segmentation, foreground segmentation, or depth threshold segmentation, removing noise points using statistical outlier filtering, unifying or registering the coordinates of point clouds from different viewpoints, and normalizing the point cloud scale.
3. The method for detecting key points of sheep body size according to claim 1, characterized in that, In step S2, the farthest point sampling or random sampling is performed on the sheep body point cloud to be detected to obtain an input point set containing a preset number of points; and based on nearest neighbor search or radius neighborhood search, a neighborhood point set is determined for each center point in the input point set.
4. The method for detecting key points of sheep body size according to claim 1, characterized in that, In step S3, the weighted neighborhood aggregation encoder determines neighborhood points for any center point in the input point set, constructs local edge features for each neighborhood point consisting of center point features, differences in neighborhood point features, and relative coordinates; inputs the local edge features into a shared mapping function to obtain neighborhood relationship features, then inputs the neighborhood relationship features and relative coordinates into a weight prediction function to obtain neighborhood response scores, and normalizes the neighborhood response scores of the same center point to obtain neighborhood contribution weights; the weighted neighborhood aggregation encoder performs weighted summation on the neighborhood relationship features according to the neighborhood contribution weights to obtain center point update features, which include at least two layers of cascaded weighted edge convolutional structures, a sampling layer between adjacent weighted edge convolutional structures, and a residual connection and normalization layer between the downsampled features and the output features of the subsequent weighted edge convolution.
5. The method for detecting key points of sheep body size according to claim 1, characterized in that, In step S4, the process of constructing the geometric description includes: establishing a local nearest neighbor set for the encoded point, calculating the mean of the nearest neighbor coordinates and the neighborhood covariance matrix, performing eigenvalue decomposition on the neighborhood covariance matrix to obtain the local surface normal vector and curvature approximation, and concatenating the position offset of the point relative to the mean of the nearest neighbor coordinates, the local surface normal vector and the curvature approximation to form a local geometric description.
6. The method for detecting key points of sheep body size according to claim 1, characterized in that, The corresponding weight generation process in step S4 includes a feature branch and a geometric branch; the feature branch outputs a feature response score based on the encoded features, and the geometric branch outputs a geometric response score based on the local geometric description. The feature response score is fused with the geometric response score with weight coefficients and normalized in the input point dimension to obtain the contribution weight of each input point to each corresponding output point.
7. The method for detecting key points of sheep body size according to claim 1, characterized in that, In step S5, the output corresponding point sequence contains multiple corresponding points arranged in a fixed semantic order; any output corresponding point is obtained by weighted summation of the coordinates of all input points participating in the corresponding weight calculation according to their contribution weight to the output position.
8. The method for detecting key points of sheep body size according to claim 1, characterized in that, In step S6, the semantic index set includes 8 index positions, corresponding to the sternal node, withers vertex, left posterior angle of scapula, right posterior angle of scapula, widest point of left abdomen, widest point of right abdomen, highest point of hip, and ischial tuberosity node, respectively; in the inference stage, the points at the corresponding index positions in the output corresponding point sequence are directly read as prediction key points.
9. The method for detecting key points of sheep body size according to claim 1, characterized in that, In step S7, the body length is calculated based on the spatial relationship between the sternal node and the ischial node; the body height is calculated based on the height of the withers apex relative to the ground or reference plane; the cross height is calculated based on the height of the highest point of the buttocks relative to the ground or reference plane; the chest width is calculated based on the lateral distance between the left and right posterior angle points of the scapula or their vicinity; the abdominal width is calculated based on the lateral distance between the widest points of the left and right abdomen; and the chest circumference is calculated based on the closed contour length of the chest section contour points or by fitting the chest section contour.
10. A sheep body size key point detection system based on weighted neighborhood aggregation and geometric constraints, characterized in that, It includes a point cloud acquisition module, a preprocessing module, a weighted neighborhood aggregation encoding module, a geometric constraint correspondence module, a semantic index key point extraction module, and a body size calculation module; each module is used to collaboratively execute the sheep body size key point detection method according to any one of claims 1 to 9, and each module can be implemented by a processor executing a computer program in the memory.