Yak body size and weight measuring method and system based on DeepLabCut and depth information

Abstract

The invention provides a yak body size and weight measurement method and system based on DeepLabCut and depth information, and relates to the technical field of non-contact measurement. According to the method, a DeepLabCut frame is adopted as a basic platform for yak body size key point detection, an improved bottom-up and conditional top-down structure is combined to optimize the frame, and a key point detection model is constructed. Key point detection branches from bottom to top in the key point detection model are used for generating initial key points and an initial skeleton on the whole yak RGB image; and the top-from-bottom key point refining branch in the model is used for performing fine regression under the conditions of the initial key point, the initial skeleton and the individual image block, and outputting the pixel coordinate of each body size key point. The key point detection model supports end-to-end training, and efficient joint reasoning of feature extraction and key point positioning is realized. According to the invention, the problem that measurement precision and deployment complexity in a yak pasture scene are difficult to consider at the same time in the prior art is solved, and non-contact accurate measurement of the body size and the weight of the yak is realized.

Description

Technical Field

[0001] This invention relates to the field of non-contact measurement technology, specifically to a method and system for measuring the body size and weight of yaks based on DeepLabCut and depth information. Background Technology

[0002] In livestock farming, especially yak farming, body size and weight measurements are core technical indicators in livestock growth performance assessment systems. The data output is intuitive and accurate, dynamically reflecting the physiological development and growth rate changes of livestock at different growth stages, such as fattening and breeding. This indicator provides fundamental data support for farms to develop standardized and refined feeding management programs.

[0003] Currently, technological research and development for measuring the body size of live livestock has moved towards automation and intelligence, with the core technological path focusing on the application of image vision and multi-sensor fusion technologies. Existing research generally employs image keypoint detection algorithms combined with unilateral depth imaging, binocular vision, or LiDAR point cloud technology, or through multi-view depth information fusion, to achieve automatic extraction of livestock body size parameters. For example, in the field of bovine body size measurement, automated measurement methods based on improved keypoint detection models combined with unilateral depth data have emerged. Such technical solutions reflect the technological evolution trend from traditional monocular vision and photogrammetry techniques to multi-sensor collaborative fusion.

[0004] However, despite the advantages demonstrated by existing automated body measurement technologies in theoretical applications and some practical scenarios, significant technical adaptability issues remain in the real-world application scenarios of yak ranches. On the one hand, yak ranches are mostly located in high-altitude pastoral areas, generally facing infrastructure conditions such as limited network bandwidth and unstable power supply, posing challenges to existing technologies that rely on high computing power and high power consumption for multi-sensor fusion equipment and data transmission links. On the other hand, yaks, as animals unique to the plateau, are large in size and exhibit strong wild behavior, reacting to human intervention with stress, making it difficult for existing technologies to stably acquire high-quality measurement data in unconstrained scenarios. As described above, existing technologies struggle to balance measurement accuracy and model deployment complexity in yak ranch scenarios. Summary of the Invention

[0005] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a method and system for measuring the body size and weight of yaks based on DeepLabCut and depth information, which solves the technical problem that existing technologies struggle to balance measurement accuracy and model deployment complexity in yak pasture scenarios.

[0006] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for measuring the body size of yaks based on DeepLabCut and depth information, comprising: Acquire RGB images and pixel-level depth information of yaks captured by a depth camera; The pixel coordinates of key points of each body size were obtained by processing RGB images of yaks acquired by a depth camera using a key point detection model based on DeepLabCut. The 3D coordinates of the yak are obtained based on pixel-level depth information and pixel coordinates of key points of each body size, and the body size data of the yak is calculated based on the 3D coordinates. The keypoint detection model is an optimized model using DeepLabCut as the framework, combined with an improved bottom-up-conditional top-down structure. The keypoint detection model includes a bottom-up keypoint detection branch and a conditional top-down keypoint refinement branch. The bottom-up keypoint detection branch is used to generate initial keypoints and an initial skeleton on the entire yak RGB image. The top-down keypoint refinement branch is used to perform fine regression based on the initial keypoints, the initial skeleton, and individual yak image blocks in the entire yak RGB image, and output the pixel coordinates of each body size keypoint.

[0007] Preferably, the improved bottom-up-conditional top-down structure includes a high-resolution network with conditional attention modulation, wherein the high-resolution network with conditional attention modulation includes a high-resolution network and a conditional attention modulation network, wherein the conditional attention modulation network is embedded in the high-resolution network, and the conditional attention modulation network has several branches, each of which sequentially includes a CoAM module, an improved channel attention module, a gated sharpening attention mechanism module and a high-resolution token mixing module. The improved channel attention module is used to adaptively adjust the channel weight distribution during cross-scale information interaction. The gated sharpening attention mechanism module is used to selectively recover and enhance the high-frequency structure of the key point neighborhood while suppressing irrelevant noise introduced by hair and background texture. The high-resolution token mixing module is used to extend the effective receptive field and achieve multi-scale spatial mixing in a lightweight manner.

[0008] Preferably, the improved channel attention module includes a compression unit and an activation unit. The compression unit extracts global statistical information for each channel through global average pooling to obtain global context features. The activation unit models inter-channel dependencies through two fully connected layers and a non-linear activation function to generate a set of channel attention weights that act on the input features of the improved channel attention module.

[0009] Preferably, the gated sharpening attention mechanism module includes a learnable spatial-channel gating, which is used for position-by-position and channel-by-channel selective amplification of high-frequency residuals, and adds the enhanced information back to the input features in the form of residuals with a zero-initialized residual scaling factor.

[0010] Preferably, the high-resolution token mixing module includes parallel multi-scale depth-separable convolutional branches, which are used to acquire spatial context information under different receptive fields.

[0011] Preferably, the step of obtaining the yak's 3D coordinates based on pixel-level depth information and pixel coordinates of key points of each body size, and calculating the yak's body size data based on the 3D coordinates, includes: Based on the camera intrinsic parameter matrix, the pixel points and their corresponding depth values ​​are back-projected onto the camera coordinate system, and the following is calculated: Using the extrinsic homogeneous transformation matrix ,in and Map points from the camera coordinate system to the world coordinate system: Combining the above two steps, we obtain the complete mapping formula from pixel coordinates and depth values ​​to 3D coordinates: In the formula, This represents the horizontal pixel coordinates of the image center point; This represents the vertical pixel coordinate of the image center point; Indicates the horizontal pixel focal length; Indicates the vertical pixel focal length; Represents the pixel coordinates of key points on each scale. depth.

[0012] Secondly, the present invention provides a yak body size measurement system based on DeepLabCut and depth information, comprising: The data acquisition module is used to acquire RGB images of yaks and pixel-level depth information captured by the depth camera; The model processing module is used to process yak RGB images acquired by a depth camera using a keypoint detection model based on DeepLabCut to obtain the pixel coordinates of keypoints of each body size. The body size data calculation module is used to obtain the yak's 3D coordinates based on pixel-level depth information and pixel coordinates of key points of each body size, and to calculate the yak's body size data based on the 3D coordinates. The keypoint detection model is an optimized model using DeepLabCut as the framework, combined with an improved bottom-up-conditional top-down structure. The keypoint detection model includes a bottom-up keypoint detection branch and a conditional top-down keypoint refinement branch. The bottom-up keypoint detection branch is used to generate initial keypoints and an initial skeleton on the entire yak RGB image. The top-down keypoint refinement branch is used to perform fine regression based on the initial keypoints, the initial skeleton, and individual yak image blocks in the entire yak RGB image, and output the pixel coordinates of each body size keypoint.

[0013] Thirdly, the present invention provides a method for measuring yak weight based on DeepLabCut and depth information, including: Acquire RGB images of yaks captured by a depth camera; The pixel coordinates of key points of each body size were obtained by processing RGB images of yaks acquired by a depth camera using a key point detection model built on DeepLabCut. The 3D coordinates of the yak are obtained based on pixel-level depth information and pixel coordinates of key points of each body size, and the body size data of the yak is calculated based on the 3D coordinates. Calculate the weight of the yak based on its body measurements; The keypoint detection model is an optimized model using DeepLabCut as the framework, combined with an improved bottom-up-conditional top-down structure. The keypoint detection model includes a bottom-up keypoint detection branch and a conditional top-down keypoint refinement branch. The bottom-up keypoint detection branch is used to generate initial keypoints and an initial skeleton on the entire yak RGB image. The top-down keypoint refinement branch is used to perform fine regression based on the initial keypoints, the initial skeleton, and individual yak image blocks in the entire yak RGB image, and output the pixel coordinates of each body size keypoint.

[0014] Preferably, the step of calculating the yak weight based on the yak's body size data includes: The yak body size data and basic parameters are processed using a pre-built weight prediction model to obtain the yak weight. The construction process of the pre-built weight prediction model is as follows: The basic parameters of the yak were obtained and standardized. The raw measurements were then transformed into composite features related to weight and height, such as the square of chest circumference multiplied by body length. Then calculate the cube root form of the composite feature: And construct body proportion characteristics: , The input feature set is composed of composite features, the cube root form of composite features, body proportion features, and basic variables. A supervised learning regression model is constructed with the true body weight y as the output target; In the modeling phase, three algorithms—linear regression, ridge regression, and gradient boosting tree—were used for model training and performance comparison. - Cross-validation calculates the mean absolute error, root mean square error, and coefficient of determination, and automatically selects the optimal model as the weight prediction model based on the principle of minimizing error. Its expression is as follows: in, The regression function is trained and optimized. Indicates the age of the yak; Indicates the height of the yak; This indicates the oblique length of the yak's body; This refers to the chest circumference of a yak.

[0015] Fourthly, the present invention provides a yak weight measurement system based on DeepLabCut and depth information, comprising: The data acquisition module is used to acquire RGB images of yaks and pixel-level depth information captured by the depth camera; The model processing module is used to process yak RGB images acquired by a depth camera using a keypoint detection model based on DeepLabCut to obtain the pixel coordinates of keypoints of each body size. The body size data calculation module is used to obtain the yak's 3D coordinates based on pixel-level depth information and pixel coordinates of key points of each body size, and to calculate the yak's body size data based on the 3D coordinates. The weight data calculation module is used to calculate the weight of a yak based on its body size data. The keypoint detection model is an optimized model using DeepLabCut as the framework, combined with an improved bottom-up-conditional top-down structure. The keypoint detection model includes a bottom-up keypoint detection branch and a conditional top-down keypoint refinement branch. The bottom-up keypoint detection branch is used to generate initial keypoints and an initial skeleton on the entire yak RGB image. The top-down keypoint refinement branch is used to perform fine regression based on the initial keypoints, the initial skeleton, and individual yak image blocks in the entire yak RGB image, and output the pixel coordinates of each body size keypoint.

[0016] (III) Beneficial Effects This invention provides a method and system for measuring the body size and weight of yaks based on DeepLabCut and depth information. Compared with existing technologies, it has the following advantages: This invention employs the DeepLabCut framework as the foundational platform for yak body size keypoint detection, and optimizes the framework by incorporating an improved bottom-up conditional top-down (BUCTD) structure to construct a keypoint detection model. This keypoint detection model includes a bottom-up keypoint detection branch and a conditional top-down keypoint refinement branch. The bottom-up keypoint detection branch generates initial keypoints and an initial skeleton on the entire yak RGB image; the bottom-up keypoint refinement branch performs fine regression based on the initial keypoints, initial skeleton, and individual image patches, outputting the pixel coordinates of each body size keypoint. This keypoint detection model supports end-to-end training, enabling efficient joint inference of feature extraction and keypoint localization. This invention utilizes RGB images of yaks captured by a depth camera and pixel-level depth information. Combined with camera intrinsic parameters (focal length, baseline) and a depth mapping matrix, the pixel coordinates of key points on the yak's body are mapped to a three-dimensional physical space. The three-dimensional Euclidean distance between these key points is calculated to obtain yak body size data. A yak weight estimation system is then established using deep learning data fitting methods, and the yak's weight is calculated based on the body size data. This invention solves the problem of balancing measurement accuracy and deployment complexity in existing technologies for yak pastures, achieving non-contact and accurate measurement of yak body size and weight. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the DeepLabCut model structure; Figure 2 A schematic diagram of the structure of the improved HRNetCoAM model in the conditional top-down keypoint refinement branch; Figure 3 A schematic diagram of the improved channel attention module in the improved HRNetCoAM model; Figure 4 A schematic diagram of the gated sharpening attention mechanism module in the improved HRNetCoAM model; Figure 5A schematic diagram of the high-resolution token mixing module in the improved HRNetCoAM model; Figure 6 This is a schematic diagram of the coordinate transformation relationship; Figure 7 This is a technical roadmap for a yak weight measurement system based on DeepLabCut and depth information. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This application provides a method and system for measuring yak body size and weight based on DeepLabCut and depth information. This solves the technical problem that existing technologies struggle to balance measurement accuracy and model deployment complexity in yak pasture scenarios, enabling precise non-contact measurement of yak body size and weight.

[0021] The technical solution in this application is to solve the above-mentioned technical problems, and the general idea is as follows: This invention is based on the DeepLabCut framework and optimized with an improved BUCTD (Bottom-Up Conditional Top-Down) structure. In general, the overall model consists of two main parts: a bottom-up keypoint detection branch (Bottom-Up, BU) for generating initial keypoints and skeletons on the entire image, and a conditional top-down keypoint refinement branch (Conditional Top-Down, CTD) that further refines regression based on the bottom-up prediction results and individual image patches. Specifically, the BU branch directly predicts the initial keypoint heatmaps and skeleton structures of all yak individuals on the entire input image, providing group-level structural priors and candidate regions. The CTD module encodes the skeleton priors output by the BU branch and the corresponding local image patches into conditional information, performing layer-by-layer fusion and upsampling of features to enhance spatial details and location information. Finally, the keypoint prediction head in the CTD branch outputs pixel coordinate heatmaps and confidence scores for each body's scale keypoints. The entire model can be trained end-to-end within a unified framework, jointly optimizing bottom-to-top structure perception and top-to-bottom fine localization to achieve efficient and robust keypoint detection. In the BU (Body Buffer) part, this embodiment uses ResNet50 as the backbone network. In the CTD (Conditional Tolerance) part, this embodiment employs an improved HRNetCoAM (High-Resolution Network with Conditional Attention Modulation) structure to maintain high-fidelity information transmission across multi-resolution feature flows. Then, leveraging the multimodal data acquisition capabilities of the Intel RealSense depth camera, by simultaneously acquiring yak RGB images and pixel-level depth information, and combining the depth camera's intrinsic parameters (focal length, baseline) and depth mapping matrix, the two-dimensional pixel coordinates are mapped to three-dimensional physical space, directly calculating the three-dimensional Euclidean distance between keypoints. Furthermore, combining data fitting methods from deep learning, body size data and weight are studied and fitted to establish a yak weight estimation platform.

[0022] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0023] Example 1: This invention provides a method for measuring the body size of yaks based on DeepLabCut and depth information, including: S1. Acquire RGB images of yaks and pixel-level depth information captured by a depth camera; S2. The RGB images of yaks acquired by the depth camera are processed using the key point detection model based on DeepLabCut to obtain the pixel coordinates of key points of each body size. S3. Obtain the 3D coordinates of the yak based on pixel-level depth information and pixel coordinates of key points of each body size, and calculate the yak body size data based on the 3D coordinates. The keypoint detection model is an optimized model using DeepLabCut as the framework, combined with an improved bottom-up-conditional top-down structure. The keypoint detection model includes a bottom-up keypoint detection branch and a conditional top-down keypoint refinement branch. The bottom-up keypoint detection branch is used to generate initial keypoints and an initial skeleton on the entire yak RGB image. The top-down keypoint refinement branch is used to perform fine regression based on the initial keypoints, the initial skeleton, and individual yak image blocks in the entire yak RGB image, and output the pixel coordinates of each body size keypoint.

[0024] The following is a detailed description of each step: In step S1, the RGB image of the yak and pixel-level depth information captured by the depth camera are obtained. The specific implementation process is as follows: In this embodiment of the invention, an RGB image of a yak is extracted from a video of a yak captured by a depth camera.

[0025] In step S2, the RGB images of yaks captured by the depth camera are processed using a keypoint detection model built on DeepLabCut to obtain the pixel coordinates of keypoints for each body size. The specific implementation process is as follows: The key point detection model uses, for example, Figure 1 The DeepLabCut framework shown serves as the foundational platform for yak body size keypoint detection, and is optimized using an improved BUCTD (Bottom-Up Conditional Top-Down) structure. The keypoint detection model includes a bottom-up keypoint detection branch (Bottom-Up, BU) and a top-down keypoint refinement branch (Conditional Top-Down, CTD). The entire model supports an end-to-end training process, enabling efficient joint inference for both feature extraction and keypoint localization tasks.

[0026] The bottom-up keypoint detection branch is used to generate initial keypoints and an initial skeleton on the entire image; the top-bottom keypoint refinement branch is used to perform fine regression based on the initial keypoints, the initial skeleton, and individual image patches. The prediction head of this branch outputs the pixel coordinate heatmap and confidence score of each body-scale keypoint.

[0027] The bottom-up keypoint detection branch uses ResNet50 as the backbone network. In addition to providing estimated bounding boxes, this model also provides a pose suggestion, which is then used as a conditional input to the CTD model.

[0028] like Figure 2As shown, the top-to-bottom keypoint refinement branch employs an improved HRNetCoAM (High-Resolution Network with Conditional Attention Modulation) to maintain high-fidelity information transmission of the multi-resolution feature stream.

[0029] The improved HRNetCoAM includes an HRNet network and a CoAM network. The CoAM network includes a CoAM module, an improved channel attention module (CSE), a gated sharpening attention mechanism module (GUA), and a high-resolution token mixing module (HRTM). It should be noted that, in this embodiment of the invention, the improved HRNetCoAM structure refers to the addition of the improved channel attention module (CSE), the gated sharpening attention mechanism module (GUA), and the high-resolution token mixing module (HRTM) to the existing HRNetCoAM structure.

[0030] Improved channel attention module: The rationale for introducing this module is as follows: During on-site deployment in actual pasture environments, some pastures are open-air, and the influence of natural light can cause parts of the yak's body to be illuminated by either strong or weak light, resulting in the loss of key information about the yak's body—especially where the body outline meets the sky or other background. Furthermore, the complex background textures and non-uniform lighting in the environment further increase the difficulty of feature extraction, affecting the model's stable recognition of the yak's body structure.

[0031] To enhance the network's responsiveness to key regions and improve the robustness of feature representation, this embodiment introduces a lightweight channel attention module—Squeeze-and-Excitation (SE). Originally proposed for image classification and object recognition tasks, this module aims to explicitly model the correlations between feature channels, thereby improving the convolutional neural network's ability to represent key information. The SE module offers a good balance between performance and structural simplicity, significantly enhancing the model's representational ability in complex visual scenes through adaptive recalibration of channel features. Considering the uneven lighting, complex backgrounds, and frequent pose changes in pasture environments, the SE module provides an efficient and robust feature enhancement mechanism for keypoint detection tasks.

[0032] like Figure 3As shown, the channel attention module includes two core operations: Squeeze (compression) and Excitation (activation). The former extracts global statistical information for each channel through Global Average Pooling (GAP) to obtain global contextual features; the latter models inter-channel dependencies through two fully connected layers and non-linear activation functions (ReLU and Sigmoid), thereby generating a set of channel attention weights. These weights are used to adaptively weight the input features into this module, enabling the network to highlight key information channels relevant to the task and suppress redundant or noisy features. Through this "compression-activation" mechanism, the SE module effectively improves the network's expressive power and generalization performance without significantly increasing the number of parameters. Its lightweight structure also makes it easy to combine with various convolutional backbone networks. Considering subsequent lightweight deployment, this embodiment replaces the original two fully connected layers with a 1×1 convolution (Conv) while maintaining the overall channel recalibration mechanism of the SE module. The 1×1 convolution is computationally equivalent to a linear transformation between channels, which can preserve the spatial distribution structure of input features while reducing the number of parameters. The improved module structure is as follows. The improved calculation process is as follows: in, Indicates the input feature map, For the Sigmoid function, This indicates a channel-by-channel multiplication operation.

[0033] This improved module reduces the number of parameters and computational cost without altering the original SE attention mechanism, while maintaining its ability to model inter-channel feature dependencies. Since the flattening operation is no longer required, the improved SE module better preserves the spatial structure information of high-resolution features, thereby improving the response to fine-grained keypoint regions (such as cow legs and backs) in complex pasture scenes. This module is ultimately embedded into the multi-resolution feature fusion stage of the HRNetCoAM network to adaptively adjust the channel weight distribution during cross-scale information interaction, thereby enhancing the discriminativity and consistency of feature representations and improving the stability and robustness of keypoint detection in complex scenes.

[0034] Gated sharpening attention mechanism module: The reason for proposing this module is as follows: In the actual scenario of body size measurement, yaks usually have long and thick hair. The texture occlusion and edge blurring caused by the hair are particularly obvious in low contrast or complex backgrounds, which weakens the high-frequency response of the neighborhood of key points and increases the localization error. Some key anatomical points are often located at the edge of the hair or in the transition area of ​​local structure. Their texture features are easily weakened in complex backgrounds or low contrast conditions. This loss of local details will also lead to the accumulation of measurement errors and affect the accuracy of the final body size estimation.

[0035] To address this, this embodiment proposes and integrates a lightweight high-frequency gating module—GatedUnsharp Attention (GUA)—in HRNetCoAM to selectively recover and enhance the high-frequency structure of keypoint neighborhoods while suppressing irrelevant noise introduced by hair and background textures. Figure 4 As shown, GUA extracts high-frequency residuals based on the unsharp-mask concept. However, unlike traditional image sharpening, this module selectively amplifies the high-frequency residuals position-by-position and channel-by-channel through a learnable spatial-channel gating. It then adds the enhanced information back to the input features in residual form using a zero-initialized residual scaling factor (α), thus achieving "progressive and controllable high-frequency enhancement." Specifically, the low-frequency baseline is initialized with a depthwise average convolution as the average filter kernel, and its parameters are frozen to ensure stability. The high-frequency residuals are obtained by differencing this baseline. A lightweight projection sub-network (a lightweight quantum network within the GUA module used for projection transformation of the high-frequency residuals. This sub-network is only a local structure used to complete the "high-frequency residual → dimensionality reduction → nonlinear mapping → channel restoration" operation in the GUA module) is applied to the high-frequency residuals. This sub-network first performs bottleneck-style dimensionality reduction mapping in the channel dimension to aggregate local semantic information, and then reconstructs the original channel dimension through channel restoration mapping. During the mapping process, batch normalization and nonlinear activation are combined to enhance expressive power. The projected output is normalized by Sigmoid to form a spatially-channel variable gated tensor, which is then element-wise multiplied with the high-frequency residuals for selective enhancement. Finally, the product is scaled by a learnable scalar coefficient α (initialized to 0) and added to the input features as a residual. This mechanism can selectively amplify beneficial details while suppressing noise amplification, and the zero-initialized residual scaling ensures that the module does not cause abrupt perturbations to the pre-trained backbone during the initial fine-tuning phase.

[0036] GUA is instantiated at the branch level and inserted after the CoAM or Self-Att stage of HRNetCoAM, prioritizing high-resolution branches to directly enhance feature maps with fine-grained structures. Figure 4This is a schematic diagram of the GUA structure.

[0037] High-resolution token mixing module: The reason for proposing this module is as follows: In the task of measuring the body size of yaks, the spatial distribution of key points varies significantly. Some key points are close to each other and have weak local texture features; while other key points are more dispersed and are easily obscured by hair or complex backgrounds. This spatial heterogeneity places higher demands on the model's feature representation capabilities: the network not only needs to have the ability to accurately model local details, but also should be able to effectively integrate multi-scale contextual information in a high-resolution feature space to distinguish adjacent key points and compensate for semantic information in occluded areas.

[0038] To address the aforementioned issues, this embodiment introduces a High-Resolution Token Mixer (HRTM) module into each high-resolution branch of HRNetCoAM. This module aims to expand the effective receptive field and achieve multi-scale spatial mixing in a lightweight manner while preserving high-resolution detail, thereby enhancing the network's ability to represent complex structural regions. Specifically, HRTM captures spatial context information under different receptive fields through parallel multi-scale depthwise separable convolutional branches without changing the input spatial size. Subsequently, these multi-scale features are fused and adaptively recalibrated along the channel dimension to highlight features more important for keypoint localization and suppress redundant information. Finally, the fused features are added back to the input features input to this module in residual form, achieving efficient feature enhancement with low computational overhead. This structure utilizes depthwise separable convolution and point-by-point fusion to expand the receptive field and improve multi-scale modeling capabilities while maintaining high parameter and computational efficiency.

[0039] In the network implementation, HRTM is embedded in the high-resolution feature stream of HRNetCoAM in a branch-level manner and is invoked after the conditional attention module. Introducing HRTM into HRNetCoAM enables it to compensate for insufficient local information through parallel multi-scale filtering and adaptive channel gating when keypoints are close to each other or subject to local interference (such as hair or shadows), thus improving the ability to identify keypoint structures. When keypoints are sparsely distributed or occluded, it provides additional contextual evidence to reduce false detections or misalignments, thereby improving the accuracy and stability of keypoint detection in complex pasture environments. The structure of HRTM is as follows: Figure 5 As shown.

[0040] The enhanced HRNetCoAM network, optimized based on three key modules—SE, GUA, and HRTM—demonstrates significant advantages in feature representation and structural modeling. This network not only improves the accuracy and robustness of keypoint detection under non-uniform lighting, occlusion, and complex background conditions, but also maintains lightweight and high efficiency, making it suitable for real-time deployment on edge computing devices. Its multi-resolution feature fusion and lightweight attention mechanism design enable the model to achieve high-precision keypoint recognition and size measurement of yaks even in resource-constrained environments.

[0041] The RGB images of yaks captured by the depth camera are processed by a key point detection model to obtain the pixel coordinates of each body size key point. These pixel coordinates are the pixel coordinates (u,v) of each body size key point.

[0042] In step S3, the 3D coordinates of the yak are obtained based on pixel-level depth information and the pixel coordinates of key points on each body size, and the yak's body size data is calculated based on the 3D coordinates. The specific implementation process is as follows: To achieve pixel coordinates of key points on each body scale With actual three-dimensional coordinates To achieve precise mapping between RGB and depth images, this embodiment uses the pyrealsense2 toolkit provided by Intel RealSense to register the images. The specific conversion relationship is as follows: Figure 6 As shown. First, based on the camera intrinsic parameter matrix: pixels and their corresponding depth values Back-projected onto the camera coordinate system, the calculation is as follows Then, using the extrinsic homogeneous transformation matrix ,in and Map points from the camera coordinate system to the world coordinate system: Combining the above two steps, we can obtain the complete mapping formula from pixel coordinates and depth values ​​to 3D coordinates: In the formula, This represents the x-pixel coordinate of the image center point; This represents the vertical pixel coordinate of the image center point, in pixels. Indicates the horizontal focal length in pixels; Indicates the vertical focal length in pixels; Represents the pixel coordinates of key points on each scale. Depth, m; The depth camera used in this embodiment Pixels Pixels Pixels Pixel.

[0043] After obtaining the three-dimensional coordinates of the key points, the spatial distance between the two points can be calculated using the three-dimensional Euclidean distance formula, thus obtaining the yak body size data.

[0044] Example 2: This invention provides a yak body size measurement system based on DeepLabCut and depth information, comprising: The data acquisition module is used to acquire RGB images of yaks and pixel-level depth information captured by the depth camera; The model processing module is used to process yak RGB images acquired by a depth camera using a keypoint detection model based on DeepLabCut to obtain the pixel coordinates of keypoints of each body size. The body size data calculation module is used to obtain the yak's 3D coordinates based on pixel-level depth information and pixel coordinates of key points of each body size, and to calculate the yak's body size data based on the 3D coordinates. The keypoint detection model is an optimized model using DeepLabCut as the framework, combined with an improved bottom-up-conditional top-down structure. The keypoint detection model includes a bottom-up keypoint detection branch and a conditional top-down keypoint refinement branch. The bottom-up keypoint detection branch is used to generate initial keypoints and an initial skeleton on the entire yak RGB image. The top-down keypoint refinement branch is used to perform fine regression based on the initial keypoints, the initial skeleton, and individual yak image blocks in the entire yak RGB image, and output the pixel coordinates of each body size keypoint.

[0045] It is understood that the yak body size measurement system based on DeepLabCut and depth information provided in this embodiment of the invention corresponds to the yak body size measurement method based on DeepLabCut and depth information described above. The explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding content in the yak body size measurement method based on DeepLabCut and depth information, and will not be repeated here.

[0046] Example 3: This invention provides a method for measuring yak weight based on DeepLabCut and depth information, including: S1. Acquire RGB images of yaks captured by a depth camera; S2. The RGB images of yaks acquired by the depth camera are processed using a key point detection model based on DeepLabCut to obtain the pixel coordinates of key points of each body size. S3. Obtain the 3D coordinates of the yak based on pixel-level depth information and pixel coordinates of key points of each body size, and calculate the yak body size data based on the 3D coordinates. S4. Calculate the weight of the yak based on its body size data; The keypoint detection model is an optimized model using DeepLabCut as the framework, combined with an improved bottom-up-conditional top-down structure. The keypoint detection model includes a bottom-up keypoint detection branch and a conditional top-down keypoint refinement branch. The bottom-up keypoint detection branch is used to generate initial keypoints and an initial skeleton on the entire yak RGB image. The top-down keypoint refinement branch is used to perform fine regression based on the initial keypoints, the initial skeleton, and individual yak image blocks in the entire yak RGB image, and output the pixel coordinates of each body size keypoint.

[0047] The first three steps of this embodiment are the same as those in Embodiment 1, and will not be repeated here.

[0048] In step S4, the yak's weight is calculated based on its body measurements. This specifically includes: After obtaining body size data, this embodiment proposes an intelligent prediction method for yak weight based on multidimensional body size parameters for weight estimation.

[0049] This method processes yak body size data and basic yak parameters using a pre-built weight prediction model to obtain the yak's weight. The construction process of the pre-built weight prediction model is as follows: First, the basic parameters of the yaks, such as age, height, length, and girth, are standardized. Then, feature engineering is performed to transform the raw measurements into composite features related to weight and height, such as the square of the girth multiplied by the length. To approximate the animal's volume characteristics; then calculate its cube root form: To balance the order-of-magnitude differences between individuals of different body types; and to construct body proportion characteristics: , This provides a comprehensive description of an individual's body structure characteristics. The system combines these features with basic variables such as age, height, body length, and chest circumference to form the input feature set. A supervised learning regression model is constructed with the true body weight y as the output target.

[0050] In the modeling phase, three algorithms—linear regression, ridge regression, and gradient boosting tree (XGBoost)—were used for model training and performance comparison. - Cross-validation ( Calculate the mean absolute error (MAE), root mean square error (RMSE), and coefficient of determination ( ). The system automatically selects the optimal model based on the principle of minimizing error. After training, the resulting weight prediction model can be represented as... in, This is the regression function that has been trained and optimized.

[0051] Example 4: This invention provides a yak weight measurement system based on DeepLabCut and depth information, comprising: The data acquisition module is used to acquire RGB images of yaks and pixel-level depth information captured by the depth camera; The model processing module is used to process yak RGB images acquired by a depth camera using a keypoint detection model based on DeepLabCut to obtain the pixel coordinates of keypoints of each body size. The body size data calculation module is used to obtain the yak's 3D coordinates based on pixel-level depth information and pixel coordinates of key points of each body size, and to calculate the yak's body size data based on the 3D coordinates. The weight data calculation module is used to calculate the weight of a yak based on its body size data. The keypoint detection model is an optimized model using DeepLabCut as the framework, combined with an improved bottom-up-conditional top-down structure. The keypoint detection model includes a bottom-up keypoint detection branch and a conditional top-down keypoint refinement branch. The bottom-up keypoint detection branch is used to generate initial keypoints and an initial skeleton on the entire yak RGB image. The top-down keypoint refinement branch is used to perform fine regression based on the initial keypoints, the initial skeleton, and individual yak image blocks in the entire yak RGB image, and output the pixel coordinates of each body size keypoint.

[0052] It should be noted that, in the specific implementation process, considering the unstable network and power conditions in pastoral areas, a depth camera can be integrated into a portable device to acquire yak images and extract body size parameters. Based on this, weight prediction can be performed to establish a yak weight estimation system suitable for the wild environment. The specific route is as follows: Figure 7 As shown.

[0053] It is understood that the yak weight measurement system based on DeepLabCut and depth information provided in this embodiment of the invention corresponds to the yak weight measurement method based on DeepLabCut and depth information described above. The explanations, examples, and beneficial effects of the relevant content can be found in the corresponding content of the yak weight measurement method based on DeepLabCut and depth information, and will not be repeated here.

[0054] In summary, compared with existing technologies, it has the following beneficial effects: This invention employs the DeepLabCut framework as the foundational platform for yak body size keypoint detection, and optimizes the framework using an improved bottom-up / conditional top-down structure to construct a keypoint detection model. This keypoint detection model includes a bottom-up keypoint detection branch and a conditional top-down keypoint refinement branch. The bottom-up keypoint detection branch generates initial keypoints and an initial skeleton on the entire yak RGB image; the top-down keypoint refinement branch performs fine regression based on the initial keypoints, initial skeleton, and individual image patches, outputting the pixel coordinates of each body size keypoint. This keypoint detection model supports end-to-end training, enabling efficient joint inference of feature extraction and keypoint localization. This invention relies on RGB images of yaks and pixel-level depth information acquired by a depth camera. Combined with camera intrinsic parameters and a depth mapping matrix, the pixel coordinates of key points of each body size are mapped to three-dimensional physical space. The three-dimensional Euclidean distance between key points is calculated to obtain yak body size data. Then, a yak weight estimation system is established through deep learning data fitting methods. Based on the yak body size data, the yak weight is calculated. This solves the problem that existing technologies cannot balance measurement accuracy and deployment complexity in yak pasture scenarios, and achieves non-contact and accurate measurement of yak body size and weight.

[0055] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0056] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for measuring the body size of yaks based on DeepLabCut and depth information, characterized in that, include: Acquire RGB images and pixel-level depth information of yaks captured by a depth camera; The pixel coordinates of key points of each body size were obtained by processing RGB images of yaks acquired by a depth camera using a key point detection model based on DeepLabCut. The 3D coordinates of the yak are obtained based on pixel-level depth information and pixel coordinates of key points of each body size, and the body size data of the yak is calculated based on the 3D coordinates. The key point detection model is a model that uses DeepLabCut as a framework and optimizes the framework by combining an improved bottom-up-conditional top-down structure. The key point detection model includes a bottom-up key point detection branch and a conditional top-down key point refinement branch. The bottom-up key point detection branch is used to generate initial key points and initial skeleton on the entire yak RGB image. The top-to-bottom keypoint refinement branch is used to perform fine regression based on the initial keypoints, the initial skeleton, and individual yak image blocks in the entire yak RGB image, and outputs the pixel coordinates of each body size keypoint.

2. The yak body size measurement method based on DeepLabCut and depth information as described in claim 1, characterized in that, The improved bottom-up-conditional top-down structure includes a high-resolution network with conditional attention modulation, wherein the high-resolution network with conditional attention modulation includes a high-resolution network and a conditional attention modulation network, wherein the conditional attention modulation network is embedded in the high-resolution network, and the conditional attention modulation network has several branches, each of which sequentially includes a CoAM module, an improved channel attention module, a gated sharpening attention mechanism module and a high-resolution token mixing module. The improved channel attention module is used to adaptively adjust the channel weight distribution during cross-scale information interaction. The gated sharpening attention mechanism module is used to selectively recover and enhance the high-frequency structure of the key point neighborhood while suppressing irrelevant noise introduced by hair and background texture. The high-resolution token mixing module is used to extend the effective receptive field and achieve multi-scale spatial mixing in a lightweight manner.

3. The yak body size measurement method based on DeepLabCut and depth information as described in claim 1, characterized in that, The improved channel attention module includes a compression unit and an activation unit. The compression unit extracts global statistical information for each channel through global average pooling to obtain global context features. The activation unit models inter-channel dependencies through two fully connected layers and a non-linear activation function to generate a set of channel attention weights that act on the input features of the improved channel attention module.

4. The yak body size measurement method based on DeepLabCut and depth information as described in claim 1, characterized in that, The gated sharpening attention mechanism module includes a learnable spatial-channel gating, which is used for position-by-position and channel-by-channel selective amplification of high-frequency residuals, and adds the enhanced information back to the input features in the form of residuals with a zero-initialized residual scaling factor.

5. The yak body size measurement method based on DeepLabCut and depth information as described in claim 1, characterized in that, The high-resolution token mixing module includes parallel multi-scale depth-separable convolutional branches, which are used to obtain spatial context information under different receptive fields.

6. The yak body size measurement method based on DeepLabCut and depth information as described in any one of claims 1 to 5, characterized in that, The process of obtaining the yak's 3D coordinates based on pixel-level depth information and pixel coordinates of key points on each body size, and calculating the yak's body size data based on the 3D coordinates, includes: Based on the camera intrinsic parameter matrix, the pixel points and their corresponding depth values ​​are back-projected onto the camera coordinate system, and the following is calculated: Using the extrinsic homogeneous transformation matrix ,in and Map points from the camera coordinate system to the world coordinate system: Combining the above two steps, we obtain the complete mapping formula from pixel coordinates and depth values ​​to 3D coordinates: In the formula, This represents the horizontal pixel coordinates of the image center point; This represents the vertical pixel coordinate of the image center point; Indicates the horizontal pixel focal length; Indicates the vertical pixel focal length; Represents the pixel coordinates of key points on each scale. depth.

7. A yak body size measurement system based on DeepLabCut and depth information, characterized in that, include: The data acquisition module is used to acquire RGB images of yaks and pixel-level depth information captured by the depth camera; The model processing module is used to process yak RGB images acquired by a depth camera using a keypoint detection model based on DeepLabCut to obtain the pixel coordinates of keypoints of each body size. The body size data calculation module is used to obtain the yak's 3D coordinates based on pixel-level depth information and pixel coordinates of key points of each body size, and to calculate the yak's body size data based on the 3D coordinates. The key point detection model is a model that uses DeepLabCut as a framework and optimizes the framework by combining an improved bottom-up-conditional top-down structure. The key point detection model includes a bottom-up key point detection branch and a conditional top-down key point refinement branch. The bottom-up key point detection branch is used to generate initial key points and initial skeleton on the entire yak RGB image. The top-to-bottom keypoint refinement branch is used to perform fine regression based on the initial keypoints, the initial skeleton, and individual yak image blocks in the entire yak RGB image, and outputs the pixel coordinates of each body size keypoint.

8. A method for measuring yak weight based on DeepLabCut and depth information, characterized in that, include: Acquire RGB images of yaks captured by a depth camera; The pixel coordinates of key points of each body size were obtained by processing RGB images of yaks acquired by a depth camera using a key point detection model built on DeepLabCut. The 3D coordinates of the yak are obtained based on pixel-level depth information and pixel coordinates of key points of each body size, and the body size data of the yak is calculated based on the 3D coordinates. Calculate the weight of the yak based on its body measurements; The key point detection model is a model that uses DeepLabCut as a framework and optimizes the framework by combining an improved bottom-up-conditional top-down structure. The key point detection model includes a bottom-up key point detection branch and a conditional top-down key point refinement branch. The bottom-up key point detection branch is used to generate initial key points and initial skeleton on the entire yak RGB image. The top-to-bottom keypoint refinement branch is used to perform fine regression based on the initial keypoints, the initial skeleton, and individual yak image blocks in the entire yak RGB image, and outputs the pixel coordinates of each body size keypoint.

9. The yak weight measurement method based on DeepLabCut and depth information as described in claim 8, characterized in that, The calculation of yak weight based on yak body size data includes: The yak body size data and basic parameters are processed using a pre-built weight prediction model to obtain the yak weight. The construction process of the pre-built weight prediction model is as follows: The basic parameters of the yak were obtained and standardized. The raw measurements were then transformed into composite features related to weight and height, such as the square of chest circumference multiplied by body length. Then calculate the cube root form of the composite feature: And construct body proportion characteristics: ， The input feature set is composed of composite features, the cube root form of composite features, body proportion features, and basic variables. A supervised learning regression model is constructed with the true body weight y as the output target; In the modeling phase, three algorithms—linear regression, ridge regression, and gradient boosting tree—were used for model training and performance comparison. - Cross-validation calculates the mean absolute error, root mean square error, and coefficient of determination, and automatically selects the optimal model as the weight prediction model based on the principle of minimizing error. Its expression is as follows: in, The regression function is trained and optimized. Indicates the age of the yak; Indicates the height of the yak; This indicates the oblique length of the yak's body; This refers to the chest circumference of a yak.

10. A yak weight measurement system based on DeepLabCut and depth information, characterized in that, include: The data acquisition module is used to acquire RGB images of yaks and pixel-level depth information captured by the depth camera; The model processing module is used to process yak RGB images acquired by a depth camera using a keypoint detection model based on DeepLabCut to obtain the pixel coordinates of keypoints of each body size. The body size data calculation module is used to obtain the yak's 3D coordinates based on pixel-level depth information and pixel coordinates of key points of each body size, and to calculate the yak's body size data based on the 3D coordinates. The weight data calculation module is used to calculate the weight of a yak based on its body size data. The key point detection model is a model that uses DeepLabCut as a framework and optimizes the framework by combining an improved bottom-up-conditional top-down structure. The key point detection model includes a bottom-up key point detection branch and a conditional top-down key point refinement branch. The bottom-up key point detection branch is used to generate initial key points and initial skeleton on the entire yak RGB image. The top-to-bottom keypoint refinement branch is used to perform fine regression based on the initial keypoints, the initial skeleton, and individual yak image blocks in the entire yak RGB image, and outputs the pixel coordinates of each body size keypoint.

Images