A method, device, equipment and medium for posture recognition of a dairy cow

Abstract

This invention discloses a method, apparatus, device, and medium for cow pose recognition, comprising: acquiring cow images; inputting the cow images into a pose recognition model based on an improved YOLOv8 to obtain target detection boxes, detection confidence scores, and initial pose categories; performing segmentation call gating strategies and Top-K mechanisms to filter detection boxes based on the detection confidence scores, using the filtered detection boxes as prompts to input the segmentation model SAM2 to obtain the highest-scoring mask; converting the highest-scoring mask into a binary image and extracting contours, selecting the largest contour as the cow contour; extracting shape feature parameters based on the cow contour; performing a matching degree score on the preset pose of the cow based on the shape feature parameters to obtain a shape prior confidence score and a corresponding shape prior pose; comparing the shape prior confidence score with a set threshold, and selecting the shape prior pose or initial pose category as the final cow pose based on the comparison result and a fusion strategy.

Description

Technical Field

[0001] This invention relates to the fields of smart animal husbandry and computer vision, specifically to a method, apparatus, equipment, and medium for dairy cow posture recognition based on SAM2 image processing. Background Technology

[0002] Posture information in dairy cows is a crucial overt behavioral signal reflecting their health, comfort level, and welfare. With the development of computer vision and artificial intelligence, posture analysis has become a major topic in the field of computer vision. However, posture classification based solely on detection models suffers from insufficient boundary information and weak interpretability. To address these challenges, researchers have proposed various technical solutions, such as using improved YOLOv8 models for accurate target localization, combined with keypoint detection networks like RTMPose to extract skeletal joints. These technologies have effectively supported specific applications such as lameness detection and estrus monitoring in dairy cows, providing important tools for improving animal welfare and production efficiency.

[0003] However, background interference within the detection frame, morphological differences between individuals in the same pose, and partial occlusion can easily lead to misjudgments. Existing methods lack sufficient interpretability for pose determination, making it difficult to provide verifiable visual evidence for breeding management and animal welfare assessment.

[0004] Therefore, there is an urgent need for a method to recognize the posture of dairy cows in unstructured and complex dairy barns. Summary of the Invention

[0005] This invention aims to at least solve one of the technical problems existing in the prior art. Therefore, in response to the above-mentioned problems, the object of this invention is to provide a method, apparatus, device, and medium for cow pose recognition based on SAM2 image processing, to solve the problems of high false positive rate, occlusion influence, complex background, and insufficient interpretability in the prior art.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: In a first aspect, the present invention provides a method for recognizing the posture of dairy cows, comprising: Capture images of dairy cows; The cow images are input into a pose recognition model based on an improved YOLOv8 to obtain the target detection box, detection confidence, and initial pose category of the cows; Based on the detection confidence, a segmentation call gating strategy and a Top-K mechanism are used to filter the detection boxes. The filtered detection boxes are then used as prompts to input the segmentation model SAM2 to obtain the mask with the highest score. The mask with the highest score is converted into a binary image and its contour is extracted. The largest contour is selected as the cow contour. Shape feature parameters are extracted based on the cow silhouette; The matching degree score of the cow's preset posture is calculated based on the shape feature parameters to obtain the shape prior confidence score and the corresponding shape prior posture. The shape prior confidence score is compared with a set threshold, and the shape prior pose or initial pose category is selected as the final cow pose based on the comparison result and a fusion strategy.

[0007] Some possible implementations involve obtaining the target bounding box, detection confidence score, and initial pose category of the cow based on the improved YOLOv8, including: The images of cows are input into the improved YOLOv8 backbone network, in which the C2f modules in layers 3-5 of CSPDarknet53 are replaced with SEAM_C2f modules, and the pose features are enhanced by channel recalibration. A MultiSEAM module is added before SPPF. The image output from the backbone network is fed into the neck network. The neck network extracts spatial features through a depthwise separable convolution, then performs channel fusion through a 1×1 convolution, and finally outputs a feature map by fusing multi-scale features through learnable weights. The feature map output from the neck network is fed into the detection head, which adopts a decoupled structure including a classification head and a regression head. The classification head uses a multi-scale attention mechanism to compress the input feature map through a 1×1 convolution, then enhances key pose features through a SEAM attention mechanism, and finally outputs the classification through a 1×1 convolution. The regression head uses a coordinate attention mechanism to compress the input feature map through a 1×1 convolution, then focuses on key positions through a coordinate attention mechanism, and finally outputs the bounding box regression through a 1×1 convolution. The final output image includes the target detection box, detection confidence, and initial pose category.

[0008] Some possible implementations include using a segmentation call gating strategy and a Top-K mechanism to filter detection boxes based on detection confidence, including: When the detection confidence is lower than the set segmentation gate threshold, the segmentation model SAM2 is not invoked; When the detection confidence is higher than the set segmentation gate threshold, the segmentation model SAM2 is only called for the Top-K high-confidence boxes. The Top-K high-confidence boxes refer to the top K detection boxes selected from all detection boxes that pass the confidence gate, sorted from high to low confidence.

[0009] Some possible implementations involve using the selected detection boxes as prompts to input the segmentation model SAM2 to obtain the highest-scoring mask, including: SAM2 point hints are generated based on the center points of the selected detection boxes, and the SAM2 segmentation model is called. After receiving the image and point cues, the segmentation model SAM2 uses the Hiera image encoder to extract multi-scale features from the input image, generating cue features containing visual information at different levels. The mask decoder combines image features and cue features, and generates multiple candidate masks through multi-scale feature fusion and high-resolution skip connection mechanism. Each mask is accompanied by a segmentation score, and the mask with the highest score is selected as the best segmentation result. The mask with the highest score is converted into a binary image and the image contour is extracted. The largest contour is selected as the cow contour.

[0010] In some possible implementations, shape feature parameters are extracted based on the cow's silhouette, wherein the shape feature parameters include: Minimum aspect ratio of the bounding rectangle: ; in, AR The aspect ratio of the minimum bounding rectangle. w , h These are the length and width of the smallest bounding rectangle, respectively; Compactness: ; in, Comp i For the first i The compactness of the shape, i For index identifier, M i Indicates the first i A mask or region of a certain shape, A( M i ) is the first i The area of ​​a shape, P( M i ) is the first i The perimeter of the shape, It is a very small positive number.

[0011] In some possible implementations, a matching degree score is performed on the preset posture of the cow based on shape feature parameters to obtain a shape prior confidence score and a corresponding shape prior posture, specifically: Normalized matching degree is calculated for the minimum bounding rectangle aspect ratio threshold range of each preset posture. The degree of matching between the minimum bounding rectangle aspect ratio of the current target and the preset ideal feature range of the posture is calculated, and this matching degree is mapped to a score of [0,1]. If the feature value is within the ideal range, a set high score is directly assigned. If it is not within the range, a score of 0 is obtained or a set relatively low base score is assigned. Different postures are differentiated and scored based on compactness characteristics. Additional scores are given for the significance of compactness in resting postures, and additional negative scores are given for the significance of compactness in standing postures. The scores are weighted and summed to obtain the overall matching score for each posture.

[0012] Some possible implementations involve comparing a shape prior confidence score with a set threshold, and selecting the shape prior pose or initial pose category as the final cow pose based on the comparison result and a fusion strategy, including: Compare the highest score of the shape rating results with the preset confidence threshold; When the highest shape score exceeds the confidence threshold and the shape prior pose is inconsistent with the initial pose category, the shape prior pose is used as the final pose classification result. When the highest shape score does not exceed the confidence threshold or the shape prior pose is consistent with the initial pose category, the initial pose category result is used as the final pose classification result. The final pose classification result, shape feature parameters, and confidence information are integrated and output.

[0013] Secondly, the present invention also provides a device for recognizing the posture of dairy cows, comprising: The image acquisition unit is configured to acquire images of dairy cows; The initial pose recognition unit is configured to input cow images into a pose recognition model based on an improved YOLOv8 to obtain the target detection box, detection confidence, and initial pose category of the cow; The model segmentation unit is configured to perform segmentation call gating strategy and Top-K mechanism to filter detection boxes based on detection confidence, and use the filtered detection boxes as prompts to input the segmentation model SAM2 to obtain the mask with the highest score; The contour extraction unit is configured to convert the highest-scoring mask into a binary image and extract the contour, selecting the largest contour as the cow contour. The parameter extraction unit is configured to extract shape feature parameters based on the cow's outline; The matching scoring unit is configured to score the matching degree of the cow's preset posture based on the shape feature parameters, and obtain the shape prior confidence score and the corresponding shape prior posture. The decision fusion unit is configured to compare the shape prior confidence score with a set threshold, and select the shape prior pose or initial pose category as the final cow pose based on the comparison result and a fusion strategy.

[0014] Thirdly, the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the method described thereon.

[0015] Fourthly, the present invention also provides a computer-readable storage medium for storing one or more programs, said one or more programs including computer instructions for causing a computer to perform the method.

[0016] Because the present invention adopts the above technical solution, it has the following characteristics: 1. This invention incorporates the SEAM attention mechanism and channel-level feature recalibration, which improves the anti-occlusion ability and enhances the discrimination ability. MultiSEAM multi-scale fusion captures contextual information at different scales, and decoupling the detection head separates the classification and regression tasks, thereby improving accuracy and enhancing the robustness and practicality of dairy cow posture recognition, making it more suitable for complex environments in cowsheds.

[0017] 2. The dual-model architecture provided by this invention enables basic detection results to be provided even if one of the models fails, further improving the robustness of the system.

[0018] 3. This invention expands from relying solely on semantic discrimination within the detection box to morphological discrimination combining the true contour of the target, which can significantly suppress background interference and provide geometric features such as aspect ratio and compactness, providing an interpretable basis for recumbent posture fine classification and low-confidence error correction; at the same time, by controlling the segmentation call range through confidence gating strategy and Top-K mechanism, the system achieves an adjustable balance between accuracy and efficiency.

[0019] In summary, the method for dairy cow pose recognition provided by this invention, which combines a deep learning detection model and a segmentation model in synergy, and integrates shape prior decision-making and confidence gating strategies, has a clear structure, well-defined module boundaries, and strong scalability. It can seamlessly adapt to various backbone networks and multimodal sensing schemes, and can be extended to other livestock pose recognition tasks, demonstrating significant research and industrialization value. Compared with existing technologies, this invention improves the interpretability of dairy cow pose recognition and also solves problems related to occlusion and size scale within the cowshed, making it widely applicable in dairy cow pose recognition. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. In the drawings: Figure 1This is a flowchart of a method for recognizing cow postures according to an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the dual-model architecture of an embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of the posture classification and scoring mechanism in an embodiment of the present invention.

[0023] Figure 4 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0024] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0025] Although terms such as first, second, third, etc., may be used in this document to describe multiple elements, components, regions, layers, and / or segments, these elements, components, regions, layers, and / or segments should not be limited by these terms. These terms may be used only to distinguish one element, component, region, layer, or segment from another. Unless the context clearly indicates otherwise, terms such as "first," "second," and other numerical terms used herein do not imply order or sequence. Therefore, the first element, component, region, layer, or segment discussed below may be referred to as the second element, component, region, layer, or segment without departing from the teachings of the exemplary embodiments.

[0026] For ease of description, spatial relative terms may be used in the text to describe the relationship of one element or feature relative to another element or feature as shown in the figure. These relative terms include, for example, "inside," "outside," "middle," "outer," "below," "above," etc. Such spatial relative terms are intended to include different orientations of the device in use or operation, other than those depicted in the figure.

[0027] Existing methods for determining cow posture lack interpretability, making it difficult to provide verifiable visual evidence for livestock management and animal welfare assessment. This invention provides a method, apparatus, device, and medium for cow posture recognition, comprising: acquiring cow images; inputting the cow images into a posture recognition model based on an improved YOLOv8 to obtain target detection boxes, detection confidence scores, and initial posture categories; performing a segmentation call gating strategy and a Top-K mechanism based on the detection confidence scores to filter detection boxes, using the filtered detection boxes as prompts to input the segmentation model SAM2 to obtain the highest-scoring mask; converting the highest-scoring mask into a binary image and extracting contours, selecting the largest contour as the cow contour; extracting shape feature parameters based on the cow contour; performing a matching degree score on the preset cow posture based on the shape feature parameters to obtain a shape prior confidence score and a corresponding shape prior posture; comparing the shape prior confidence score with a set threshold, and selecting the shape prior posture or initial posture category as the final cow posture based on the comparison result and a fusion strategy. Therefore, the present invention can solve the problems of high false positive rate, occlusion effect, complex background and insufficient interpretability in the prior art of pose recognition.

[0028] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0029] Example 1: As Figure 1 As shown in this embodiment, the method for cow pose recognition based on SAM2 image processing includes: S1. Real-time acquisition of cow posture data.

[0030] In this embodiment, a standard image dataset of four cow postures (standing, lying on one's side, reclining, and sleeping) is collected. During image acquisition, a camera can capture the standard image dataset containing the four cow postures. In this embodiment, the angle between the camera and the camera connecting rod for acquiring cow images is 100°, and the angle with the ground is 45°. The camera is located at approximately one-quarter of the distance from left to right in the reclining area. This is just one example; however, the configuration can be adjusted according to actual conditions.

[0031] Furthermore, the standard image dataset collected in this embodiment contains 860 images. Since each image contains several cows in different poses, the final dataset includes 350 standing images, 200 lying images, 467 reclining images, and 275 sleeping images, covering different lighting conditions, angles, and occlusion situations. The image resolution is uniformly 1024×768 pixels, and includes cows of different sizes and coat colors. The dataset is divided into training, validation, and test sets in a 7:2:1 ratio.

[0032] S2. Cow multi-pose detection and localization based on improved YOLOv8.

[0033] In this embodiment, the improved YOLOv8 has made targeted improvements at three levels: the backbone network, the neck network, and the detection head. (1) Backbone network improvement: Replace the C2f module in layers 3-5 of CSPDarknet53 with the SEAM_C2f module to enhance pose features through channel recalibration; add the MultiSEAM module before SPPF to improve representation capabilities using multi-scale context.

[0034] (2) Neck network optimization: A weighted bidirectional feature pyramid is adopted, and multi-scale features are fused through learnable weights. Spatial attention is added to the top-down path to highlight key regions, and depthwise separable convolutions are used to balance efficiency and performance.

[0035] (3) Improved detection head: The classification and regression tasks are separated by a decoupled structure, and multi-scale attention and coordinate attention mechanisms are embedded respectively. The task balance optimization is achieved through adaptive loss weights.

[0036] Specifically, the training set is input into the improved YOLOv8 object detection model for training to obtain a pose classification and recognition model, which outputs the detection box, detection confidence score, and initial pose category for each cow target. The improved YOLOv8 object detection model employs a SEAM_C2f backbone network to enhance pose features, a neck network to optimize multi-scale fusion, and decoupling the detection head from classification and regression tasks, outputting the detection box, detection confidence score, and initial pose category. The specific training process is as follows: S21. Input a 1024×768 pixel image of a cow, normalize it, scale it to 640×640, apply MixUp data augmentation, and then input it into the backbone network of the improved YOLOv8. The specific process is as follows: First, the image is passed through an initial convolutional layer, a 3×3 convolution, the number of channels changes from 3 to 32, the stride is 2, and the output feature map size is 320×240×32.

[0037] Next, the image enters the backbone of the CSPDarknet53 backbone network. In the first two layers (layers 1-2), standard Conv modules are used for convolution operations: layer 1 is a 3×3 convolution, layer 2 is a 3×3 convolution, and the output feature map size is 80×60×128. Starting from layer 3, the SEAM_C2f module is used to perform channel reduction on the input feature map through a 1×1 convolution. Then, the SEAM attention mechanism is used to obtain channel statistics through global average pooling, generate channel weights through two fully connected layers, and apply the weights to the feature map to enhance key pose features.

[0038] Next, the feature map is processed through a 3×3 depthwise separable convolution for spatial feature extraction, followed by a 1×1 convolution to restore the number of channels, resulting in an output feature map size of 80×60×128. Layers 4 and 5 also use the SEAM_C2f module, but the number of channels is increased from 128 to 256, while the feature map size remains at 80×60×128. Before the SPPF layer, a MultiSEAM module is added. This module captures multi-scale contextual information through multiple parallel SEAM modules. For example, the first module processes a 1×1 receptive field, the second a 3×3 receptive field, the third a 5×5 receptive field, and the fourth a 7×7 receptive field. These multi-scale features are then fused to enhance the ability to capture pose features at different scales.

[0039] Finally, the image enters the SPPF (Spatial Pyramid Pooling) layer, which uses pooling kernels of 3×3, 5×5, and 7×7 to perform pooling, and the output feature map size is 80×60×256.

[0040] S22. The feature map enters the neck network. First, spatial features are extracted through a depthwise convolution, then channel fusion is performed through a 1×1 convolution, and then multi-scale features are fused through learnable weights. The output feature map size is 80×60×256.

[0041] S23. The feature map enters the detection head, which adopts a decoupled structure, consisting of a classification head and a regression head. The classification head uses a multi-scale attention mechanism to compress the input feature map through a 1×1 convolution, then enhances key pose features through a SEAM attention mechanism, and finally outputs the classification result through a 1×1 convolution. The regression head uses a coordinate attention mechanism to compress the input feature map through a 1×1 convolution, then focuses on key positions through a coordinate attention mechanism, and finally outputs the bounding box regression result through a 1×1 convolution.

[0042] In this embodiment, the confidence threshold of the improved YOLOv8 can be set to 0.1-0.5, and the IoU threshold can be set to 0.5-0.8. The purpose of setting the confidence threshold to 0.1-0.5 is to filter out the prediction boxes that the model considers unreliable during the inference stage, and only retain the detection results with high confidence. A lower threshold helps to improve the recall rate to avoid missing weak abdominal motion regions, while a higher threshold can reduce false detections. The main purpose of setting the IoU threshold to 0.5-0.8 is to judge the degree of overlap between prediction boxes during non-maximum suppression (NMS) to remove redundant and repeated detection boxes. At the same time, it is used to determine whether the prediction box is close enough to the real target during evaluation. A higher IoU threshold requires more accurate localization and is suitable for respiratory region extraction tasks with high requirements for contour accuracy. The ablation experiment results of different improved modules are shown in Table 1.

[0043] Table 1: Ablation Experiment Results of Different Improved Modules

[0044] As shown in Table 1, by adding SEAM and MultiSEAM attention mechanisms, the accuracy and recall of the bounding boxes are improved.

[0045] S3. Perform fine segmentation on the detected cow targets, generate a high-precision semantic mask, and extract the maximum contour.

[0046] In this embodiment, the mask with the highest segmentation score is selected and the maximum contour is extracted as the boundary of the cow body by means of a confidence gating strategy and a Top-K mechanism. The inference device is configured as a GPU according to the deployment device, and parameters such as detection confidence threshold, detection IOU threshold, segmentation gating threshold and the maximum number of Top-K candidates for segmentation are set.

[0047] Furthermore, based on the detection confidence score (the detection confidence score refers to the confidence score of the YOLOv8 model for the detection box; the higher the value, the more certain the model is about the detection result; a value >0.5 is generally considered a high-confidence detection, and <0.3 is generally considered a low-confidence detection), a segmentation call gating strategy is executed: (1) When the detection confidence is lower than the set segmentation gate threshold, and the confidence is lower than 0.25, the segmentation model SAM2 will not be called. These targets will not appear in the final detection_info list, nor will their segmentation mask, shape analysis lines (text starting with Shape) and shape-optimized pose be displayed in the visualization results.

[0048] (2) When the detection confidence is greater than the set segmentation gate threshold, the segmentation model SAM2 is called only for the Top-K high confidence target boxes. The Top-K high confidence target boxes refer to the top K boxes selected from all the detection boxes that have passed the confidence gate, sorted from high to low confidence. The value of K is specified by the parameter max_sam_boxes.

[0049] In this embodiment, a point cue is generated based on the center point of the detection box, and the SAM2 segmentation model is called to generate a high-precision semantic mask. After the improved YOLOv8 detects the cow target and generates a bounding box, the coordinates of the center point of the bounding box are calculated and converted into the point cue format required by the SAM2 segmentation model. Specifically, the coordinates of the top left corner (x1, y1) and bottom right corner (x2, y2) of the YOLO detection box are directly used as a box parameter to form an array [x1, y1, x2, y2] and passed to the SAM2 segmentation model. The SAM2 segmentation model normalizes this box coordinate to [0, 1] (relative to the image size) and encodes it. The SAM2 segmentation model uses the center point of the detection box as the point cue, sets the point label to the foreground point (label=1), and uses the OpenCV findContours function for contour extraction, selecting the largest contour as the cow contour. The center point coordinates are normalized to the range of 0-1 relative to the image width and height, and the point label is set to the foreground point (label=1), forming the point cue array input to the SAM2 segmentation model.

[0050] Furthermore, based on the detection confidence, a segmentation invocation gating strategy is executed to filter detection boxes. The filtered detection boxes are used as prompts to input the segmentation model SAM2 to obtain the mask with the highest score. The mask with the highest score is converted into a binary image and its contour is extracted. The largest contour is selected as the cow contour. The specific process is as follows: S31. After receiving the image and point cues, the segmentation model SAM2 uses the Hiera image encoder to extract multi-scale features from the input image, generating feature representations containing visual information at different levels. Simultaneously, the cue encoder converts the point cues into embedding vectors compatible with image features, guiding subsequent mask generation.

[0051] S32, the mask decoder combines image features and cue features, and generates multiple candidate masks through multi-scale feature fusion and high-resolution skip connection mechanism. Each mask is accompanied by a segmentation score, and the mask with the highest score is selected as the best segmentation result, avoiding background interference and occlusion effects.

[0052] S33. Convert the mask with the highest score into a binary image, use OpenCV's findContours function to extract the contour, and select the largest contour as the cow contour.

[0053] S4. Extract shape feature parameters based on the cow outline.

[0054] In this embodiment, the shape feature parameters include the aspect ratio of the minimum bounding rectangle (max(w,h) / min(w,h)) and compactness (4π×area / perimeter²), which provide a quantitative basis for attitude scoring.

[0055] Furthermore, the aspect ratio of the minimum bounding rectangle is used to calculate the length and width w and h of the minimum bounding rectangle for the mask outline, expressed as:

[0056] in, AR The aspect ratio of the minimum bounding rectangle. w , h These are the length and width of the smallest bounding rectangle, respectively; Compactness is an index describing how closely a shape resembles a circle. A value closer to 1 indicates a shape that is closer to a circle. This concept is also commonly used in particle shape analysis and general image analysis to describe properties where a value closer to a circle is closer to 1. The expression is: ; in, Comp i No. i The compactness of the shape, i As an index identifier, it is used to distinguish the first object from the second when analyzing multiple objects. i An object or shape, M i Indicates the first i A mask or region of shape, typically a connected region in a binary image, A( M i ) is the first i The area of ​​a shape, P( M i ) is the first i The perimeter of the shape, It is a very small positive number.

[0057] S5. Score the matching degree of each preset posture based on shape feature parameters.

[0058] In this embodiment, a matching score is given for each preset pose based on shape feature parameters (the aspect ratio and compactness of the minimum bounding rectangle), including: S51. For each preset posture, the minimum bounding rectangle aspect ratio threshold range is normalized and the matching degree is calculated. The degree of matching between the minimum bounding rectangle aspect ratio of the current target and the preset ideal feature range of the posture is calculated, and this matching degree is mapped to a score of [0,1]. If the feature value is within the ideal range (e.g., 0.3 <= AR <= 0.6), a fixed high score is directly assigned. If it is not within the range, a score of 0 or a very low base score is assigned.

[0059] Different postures are differentiated and scored based on compactness characteristics. Additional scores are given for the significance of compactness in resting postures, and additional negative scores are given for the significance of compactness in standing postures. The scores are weighted and summed to obtain the overall matching score for each posture.

[0060] Furthermore, the standard lying posture: when the length-to-width ratio is between 0.8 and 1.5, the score is +0.4; when the compactness is between 0.3 and 0.5, the score is +0.6. In this posture, the cow's body lies flat, with a moderate length-to-width ratio and moderate compactness, expressed as: ; Fully recumbent lateral position: A score of +0.6 is awarded when the length-to-width ratio is between 0.7 and 1.0; a score of +0.4 is awarded when the compactness is <0.4. When lying on its side, the cow's body is elongated laterally, with a length-to-width ratio close to 1, indicating low compactness. The expression for this is: ; Resting / Reclining Posture: A score of +0.4 is awarded when the length-to-width ratio is between 0.7 and 1.2; a score of +0.6 is awarded when the compactness is >0.6. When resting, the cow's body is curled up, with a moderate length-to-width ratio but high compactness, more closely resembling a circle. Its expression is: ; For standing posture, a score of +0.6 is given when the length-to-width ratio is >2.3 or <0.5; and a score of +0.4 is given when the compactness is <0.3. When standing, dairy cows exhibit a large length-to-width ratio, a distinctly elongated shape, and low compactness.

[0061] .

[0062] S52, Scoring of posture classification.

[0063] In this embodiment, the scoring process for pose classification is as follows: Standing posture judgment rules: When the aspect ratio of the smallest bounding rectangle of the mask outline is greater than 2.3 or less than 0.5, the score increases by 0.6; when the compactness is less than 0.3, the score is 0.4.

[0064] Standard reclining posture judgment rules: When the aspect ratio of the smallest bounding rectangle is between 0.8 and 1.5, the score increases by 0.4; when the compactness is between 0.3 and 0.5, the score increases by 0.6. The rules for judging the fully lying-on-side posture are as follows: when the aspect ratio of the smallest bounding rectangle is between 0.7 and 1.0, the score increases by 0.6; when the compactness is less than 0.4, the score is 0.4. Rules for judging resting and lying posture: When the minimum bounding rectangle ratio is between 0.7 and 1.2, the score increases by 0.4; when the compactness is greater than 0.5, the score increases by 0.6.

[0065] S53. Select the pose with the highest score as the shape prior cue. If the highest score is below 0.3, it is determined to be an "unknown" pose. Repeat steps S2-S5 for each frame of the video input and collect pose distribution information for all frames. Figure 3 The diagrams (a) standard lying down, (b) side lying, (c) standing, and (d) sleeping postures are used to identify postures through a scoring mechanism, showing the aspect ratio and compactness respectively.

[0066] S6. Decision-making based on the fusion of shape prior confidence score and confidence level.

[0067] In this embodiment, a set of shape thresholds for different poses is established, and the prior score of each pose is calculated based on the shape features to obtain the shape prior confidence score and the corresponding shape prior pose. The final pose is output according to the fusion strategy. The initial pose category is used by default. When the shape prior confidence score exceeds the preset threshold and conflicts with the initial pose category, the shape prior pose is allowed to cover the initial pose category, thereby realizing low confidence error correction.

[0068] Furthermore, in the decision fusion stage, the shape prior posture is compared with the initial posture category: when the highest confidence score of the shape prior exceeds a preset confidence threshold and is inconsistent with the initial posture category, the system adopts the shape prior posture to cover the initial posture; otherwise, the initial posture category is retained, and structured data containing posture type, shape feature parameters, and confidence level is output to support real-time decision-making in the aquaculture system. The specific process is as follows: Compare the highest prior confidence score for shape with the preset confidence threshold (0.5); When the highest confidence score of the shape prior exceeds the confidence threshold and is inconsistent with the initial pose category, the corresponding shape prior pose is used as the final pose classification. When the highest shape prior confidence score does not exceed the confidence threshold or is consistent with the initial pose category, the initial pose category is used as the final pose. The final pose classification result, shape feature parameters, and confidence information are integrated and output.

[0069] Example 2: Following the method for recognizing cow postures provided in Example 1, this example provides a device for recognizing cow postures. The device provided in this example can implement the cow posture recognition method of Example 1. This device can be implemented through software, hardware, or a combination of both. For ease of description, this example is described by dividing the functionality into various units. Of course, in implementation, the functions of each unit can be implemented in one or more software and / or hardware components. For example, the device may include integrated or separate functional modules or units to execute the corresponding steps in the methods of Example 1. Since the device in this example is basically similar to the method example, the description process of this example is relatively simple. Relevant details can be found in the description of Example 1. The embodiment of the cow posture recognition device provided by this invention is merely illustrative.

[0070] Specifically, the cow posture recognition device provided in this embodiment includes: The image acquisition unit is configured to acquire images of dairy cows; The initial pose recognition unit is configured to input cow images into a pose recognition model based on an improved YOLOv8 to obtain the target detection box, detection confidence, and initial pose category of the cow; The model segmentation unit is configured to perform segmentation call gating strategy and Top-K mechanism to filter detection boxes based on detection confidence, and use the filtered detection boxes as prompts to input the segmentation model SAM2 to obtain the mask with the highest score; The contour extraction unit is configured to convert the highest-scoring mask into a binary image and extract the contour, selecting the largest contour as the cow contour. The parameter extraction unit is configured to extract shape feature parameters based on the cow's outline; The matching scoring unit is configured to score the matching degree of the cow's preset posture based on the shape feature parameters, and obtain the shape prior confidence score and the corresponding shape prior posture. The decision fusion unit is configured to compare the shape prior confidence score with a set threshold, and select the shape prior pose or initial pose category as the final cow pose based on the comparison result and a fusion strategy.

[0071] Example 3: This example provides an electronic device corresponding to the cow posture recognition method provided in Example 1. The electronic device can be an electronic device for the client, such as a mobile phone, laptop, tablet computer, desktop computer, etc., to execute the method of Example 1.

[0072] like Figure 4As shown, the electronic device includes a processor, a memory, a communication interface, and a bus. The processor, memory, and communication interface are connected via the bus to enable communication between them. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the method of Embodiment 1. The implementation principle and technical effects are similar to those of Embodiment 1, and will not be repeated here. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computing device on which the present application is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0073] In a preferred embodiment, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), and optical discs.

[0074] In a preferred embodiment, the processor can be any type of general-purpose processor such as a central processing unit (CPU) or a digital signal processor (DSP), and is not limited thereto.

[0075] Example 4: This example provides a computer-readable storage medium for storing one or more programs, the one or more programs including computer instructions, which, when executed by a computer, cause the computer to perform the method provided in Example 1 above.

[0076] In a preferred embodiment, the computer-readable storage medium may be a tangible device for holding and storing instructions executable, such as, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof. The computer-readable storage medium stores computer program instructions that cause a computer to perform the method provided in Embodiment 1 above.

[0077] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0078] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0079] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0080] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In the description of this specification, the terms "a preferred embodiment," "furthermore," "specifically," "in this embodiment," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments in this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for recognizing the posture of dairy cows, characterized in that, include: Capture images of dairy cows; The cow images are input into a pose recognition model based on an improved YOLOv8 to obtain the target detection box, detection confidence, and initial pose category of the cows; Based on the detection confidence, a segmentation call gating strategy and a Top-K mechanism are used to filter the detection boxes. The filtered detection boxes are then used as prompts to input the segmentation model SAM2 to obtain the mask with the highest score. The mask with the highest score is converted into a binary image and its contour is extracted. The largest contour is selected as the cow contour. Shape feature parameters are extracted based on the cow silhouette; The matching degree score of the cow's preset posture is calculated based on the shape feature parameters to obtain the shape prior confidence score and the corresponding shape prior posture. The shape prior confidence score is compared with a set threshold, and the shape prior pose or initial pose category is selected as the final cow pose based on the comparison result and a fusion strategy.

2. The method for recognizing cow postures according to claim 1, characterized in that, Based on the improved YOLOv8, the target detection bounding box, detection confidence score, and initial pose class of the cow were obtained, including: The images of cows are input into the improved YOLOv8 backbone network, in which the C2f modules in layers 3-5 of CSPDarknet53 are replaced with SEAM_C2f modules, and the pose features are enhanced by channel recalibration. A MultiSEAM module is added before SPPF. The image output from the backbone network is fed into the neck network. The neck network extracts spatial features through a depthwise separable convolution, then performs channel fusion through a 1×1 convolution, and finally outputs a feature map by fusing multi-scale features through learnable weights. The feature map output from the neck network is fed into the detection head, which adopts a decoupled structure including a classification head and a regression head. The classification head uses a multi-scale attention mechanism to compress the input feature map through a 1×1 convolution, then enhances key pose features through a SEAM attention mechanism, and finally outputs the classification through a 1×1 convolution. The regression head uses a coordinate attention mechanism to compress the input feature map through a 1×1 convolution, then focuses on key positions through a coordinate attention mechanism, and finally outputs the bounding box regression through a 1×1 convolution. The final output image includes the target detection box, detection confidence, and initial pose category.

3. The method for recognizing cow postures according to claim 2, characterized in that, The detection boxes are filtered based on the segmentation call gating strategy and the Top-K mechanism, which are performed according to the detection confidence. When the detection confidence is lower than the set segmentation gate threshold, the segmentation model SAM2 is not invoked; When the detection confidence is higher than the set segmentation gate threshold, the segmentation model SAM2 is only called for the Top-K high-confidence boxes. The Top-K high-confidence boxes refer to the top K detection boxes selected from all detection boxes that pass the confidence gate, sorted from high to low confidence.

4. The method for recognizing cow postures according to claim 3, characterized in that, Using the selected detection boxes as prompts, the segmentation model SAM2 obtains the highest-scoring mask, including: SAM2 point hints are generated based on the center points of the selected detection boxes, and the SAM2 segmentation model is called. After receiving the image and point cues, the segmentation model SAM2 uses the Hiera image encoder to extract multi-scale features from the input image, generating cue features containing visual information at different levels. The mask decoder combines image features and cue features, and generates multiple candidate masks through multi-scale feature fusion and high-resolution skip connection mechanism. Each mask is accompanied by a segmentation score, and the mask with the highest score is selected as the best segmentation result. The mask with the highest score is converted into a binary image and the image contour is extracted. The largest contour is selected as the cow contour.

5. The method for recognizing cow postures according to claim 4, characterized in that, Shape feature parameters are extracted based on the cow silhouette, where the shape feature parameters include: Minimum aspect ratio of the bounding rectangle: ； in, AR The aspect ratio of the minimum bounding rectangle. w , h These are the length and width of the smallest bounding rectangle, respectively; Compactness: ； in, Comp i For the first i The compactness of the shape, i For index identifier, M i Indicates the first i A mask or region of a certain shape, A( M i ) is the first i The area of ​​a shape, P( M i ) is the first i The perimeter of the shape, It is a very small positive number.

6. The method for recognizing cow postures according to claim 5, characterized in that, Based on shape feature parameters, a matching degree score is calculated for the preset pose of the cow, resulting in a shape prior confidence score and a corresponding shape prior pose, specifically: Normalized matching degree is calculated for the minimum bounding rectangle aspect ratio threshold range of each preset posture. The degree of matching between the minimum bounding rectangle aspect ratio of the current target and the preset ideal feature range of the posture is calculated, and this matching degree is mapped to a score of [0,1]. If the feature value is within the ideal range, a set high score is directly assigned. If it is not within the range, a score of 0 is obtained or a set relatively low base score is assigned. Different postures are differentiated and scored based on compactness characteristics. Additional scores are given for the significance of compactness in resting postures, and additional negative scores are given for the significance of compactness in standing postures. The scores are weighted and summed to obtain the overall matching score for each posture.

7. The method for recognizing cow postures according to claim 6, characterized in that, The shape prior confidence score is compared with a set threshold. Based on the comparison result and using a fusion strategy, the shape prior pose or initial pose category is selected as the final cow pose, including: Compare the highest score of the shape rating results with the preset confidence threshold; When the highest shape score exceeds the confidence threshold and the shape prior pose is inconsistent with the initial pose category, the shape prior pose is used as the final pose classification result. When the highest shape score does not exceed the confidence threshold or the shape prior pose is consistent with the initial pose category, the initial pose category result is used as the final pose classification result. The final pose classification result, shape feature parameters, and confidence information are integrated and output.

8. A device for recognizing the posture of dairy cows, characterized in that, include: The image acquisition unit is configured to acquire images of dairy cows; The initial pose recognition unit is configured to input cow images into a pose recognition model based on an improved YOLOv8 to obtain the target detection box, detection confidence, and initial pose category of the cow; The model segmentation unit is configured to perform segmentation call gating strategy and Top-K mechanism to filter detection boxes based on detection confidence, and use the filtered detection boxes as prompts to input the segmentation model SAM2 to obtain the mask with the highest score; The contour extraction unit is configured to convert the highest-scoring mask into a binary image and extract the contour, selecting the largest contour as the cow contour. The parameter extraction unit is configured to extract shape feature parameters based on the cow's outline; The matching scoring unit is configured to score the matching degree of the cow's preset posture based on the shape feature parameters, and obtain the shape prior confidence score and the corresponding shape prior posture. The decision fusion unit is configured to compare the shape prior confidence score with a set threshold, and select the shape prior pose or initial pose category as the final cow pose based on the comparison result and a fusion strategy.

9. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the method according to any one of claims 1-7.

10. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include computer instructions for causing a computer to perform the method according to any one of claims 1-7.

Images