Dairy cow body shape and appearance judgment method based on key point detection and related equipment

Abstract

The invention discloses a dairy cow body shape and appearance judgment method based on key point detection and related equipment, and relates to the technical field of image processing. The method comprises the steps that each first image of a cow to be detected is acquired, and each first image comprises at least two continuous frames of images; inputting the first images into an appearance feature detection model, and obtaining key points output by the appearance feature detection model; correcting the key points in each first image to obtain a target key point based on an association relationship between the key points in each first image of at least two continuous frames; based on the pixel point distance between the target key points, determining the body shape and appearance of the to-be-detected dairy cow; wherein the appearance feature detection model is obtained by training a sample dairy cow image and a sample label, and the sample label is a pixel point corresponding to a key point representing the appearance feature of the dairy cow body type in the sample dairy cow image. By implementing the technical scheme provided by the invention, the problem of inaccurate key point positioning caused by the posture change of the dairy cow can be effectively solved, so that the accuracy of evaluating the body shape and appearance of the dairy cow is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for determining the body shape and appearance of a dairy cow based on key point detection and related equipment. Background Art

[0002] The body shape and appearance of dairy cows are not only closely related to their health and service life, but can also directly or indirectly affect the production performance of dairy cows. Evaluating the body shape and appearance of dairy cows is of great significance for dairy cow breeding, reproduction and scientific research. At present, the body shape and appearance scoring system used in my country mainly draws on the Canadian 9-point identification method, and the identification content includes body structure, limbs and hooves, udders and milk characteristics. The data collected through body shape and appearance evaluation can be used for pasture management and bull progeny determination, providing an important basis for improving the genetic level of dairy cow populations.

[0003] In the existing technology, the body shape and appearance of dairy cows are mainly evaluated by combining manual scoring and computer vision. Among them, some solutions collect images of dairy cows by setting up multiple cameras and extracting the body shape characteristics of dairy cows by using image processing technology. Other solutions achieve automatic scoring by establishing a standard image database and comparing and matching the images of the dairy cows to be tested with the standard images. Some solutions also use three-dimensional reconstruction technology to measure body shape parameters by obtaining a three-dimensional model of the dairy cow.

[0004] However, the above scheme has the technical problem of inaccurate key point positioning in practical applications. Due to the changeable postures of cows during movement and the differences in body shape between different individuals, it is difficult for existing image processing methods to accurately identify and locate the key points of the cow's body shape and appearance. Especially in non-standard postures, the positioning accuracy of key points is often difficult to meet the needs of practical applications, affecting the accuracy of the assessment of the cow's body shape and appearance. Summary of the invention

[0005] The present application provides a method and related equipment for determining the body shape and appearance of a dairy cow based on key point detection, which can effectively overcome the problem of inaccurate key point positioning caused by changes in the posture of the dairy cow, thereby improving the accuracy of the assessment of the body shape and appearance of the dairy cow.

[0006] In a first aspect of the present application, a method for determining the body shape and appearance of a dairy cow based on key point detection is provided, comprising: Acquire at least two consecutive first image frames of the cow to be tested; Input each of the first images into an appearance feature detection model, respectively, and obtain key points of each of the first images output by the appearance feature detection model, wherein the key points are pixel points representing the appearance features of the cow to be tested; Based on the correlation relationship between the key points of each of the first images, correcting the key points to obtain target key points; Determining the body shape and appearance of the cow to be tested based on the pixel distances between the target key points of each of the first images; The appearance feature detection model is trained by sample cow images and sample labels, and the sample labels are pixel points corresponding to key points in the sample cow images that characterize the cow's body shape and appearance features.

[0007] In a second aspect of the present application, a device for determining the body shape and appearance of a dairy cow based on key point detection is provided, comprising: A first image acquisition module, used to acquire first images of the cow to be tested, each of the first images including at least two consecutive frames of images; A key point determination module, used for inputting each of the first images into an appearance feature detection model, and obtaining key points output by the appearance feature detection model, wherein the key points are pixel points representing the appearance features of the cow to be tested; A target key point determination module, configured to correct the key points in each of the first images to obtain the target key point based on the correlation between the key points in at least two consecutive frames of the first images; A body shape and appearance determination module, used to determine the body shape and appearance of the cow to be tested based on the pixel point distance between the target key points; The appearance feature detection model is obtained by training key points, and the key points are positioning points that characterize the body appearance features of dairy cows.

[0008] In a second aspect of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for determining the body shape and appearance of a cow based on key point detection as described above is implemented.

[0009] In a third aspect of the present application, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for determining the body shape and appearance of a dairy cow based on key point detection as described in any one of the above is implemented.

[0010] In a fourth aspect of the present application, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method for determining the body shape and appearance of a dairy cow based on key point detection as described above is implemented.

[0011] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages: By acquiring each first image containing at least two consecutive frames of images, and using the appearance feature detection model obtained through key point training to output key points, and then correcting based on the correlation between the key points between consecutive frames, and finally determining the body shape and appearance of the cow according to the pixel point distance between the corrected target key points, the accuracy of key point positioning can be effectively improved. Compared with the prior art, the present invention uses the correlation between key points in consecutive frame images for correction, overcoming the defect that a single frame image is easily affected by the change of cow posture; at the same time, the feature detection model trained by key points can adapt to the body shape differences between different individuals, and can still maintain a high key point positioning accuracy under non-standard postures, which can effectively overcome the problem of inaccurate key point positioning caused by the change of cow posture, thereby improving the accuracy of the evaluation of cow body shape and appearance. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 It is a flow chart of a method for determining the body shape and appearance of a dairy cow based on key point detection provided in an embodiment of the present application; Figure 2 It is a schematic diagram of the structure of a typical characteristic collection device for the body shape and appearance of a dairy cow provided in an embodiment of the present application; Figure 3 It is a structural schematic diagram of an appearance feature detection model provided in an embodiment of the present application; Figure 4 It is a structural schematic diagram of a device for determining the body shape and appearance of a dairy cow based on key point detection provided in an embodiment of the present application.

[0014] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0016] Please refer to Figure 1 , Figure 1This is a flow chart of a method for determining the body shape and appearance of a dairy cow based on key point detection provided by an embodiment of the present application. The method can be implemented by a computer program, can be implemented by a single-chip microcomputer, or can be run on a dairy cow body shape and appearance determination system based on key point detection based on a von Neumann system. The computer program can be integrated into an application or run as an independent tool application. Specifically, the method can include the following steps: S101, obtaining at least two consecutive first image frames of a cow to be tested.

[0017] S102. Input each first image into an appearance feature detection model to obtain key points output by the appearance feature detection model, where the key points are pixel points that characterize the appearance features of the cow to be tested, wherein the appearance feature detection model is trained by sample cow images and sample labels, and the sample labels are pixel points corresponding to the key points that characterize the cow's body appearance features in the sample cow images.

[0018] S103: Based on the correlation relationship between the key points in the first images of at least two consecutive frames, correct the key points in the first images to obtain target key points.

[0019] S104: Determine the body shape and appearance of the cow to be tested based on the pixel distance between the target key points.

[0020] By acquiring each first image containing at least two consecutive frames of images, and using the appearance feature detection model obtained through key point training to output key points, and then correcting based on the correlation between the key points between consecutive frames, and finally determining the body shape and appearance of the cow according to the pixel point distance between the corrected target key points, the accuracy of key point positioning can be effectively improved. Compared with the prior art, the present invention uses the correlation between key points in consecutive frame images for correction, overcoming the defect that a single frame image is easily affected by the change of cow posture; at the same time, the feature detection model trained by key points can adapt to the body shape differences between different individuals, and can still maintain a high key point positioning accuracy under non-standard postures, which can effectively overcome the problem of inaccurate key point positioning caused by the change of cow posture, thereby improving the accuracy of the evaluation of cow body shape and appearance.

[0021] Optionally, in S101, each first image refers to a single-frame static image extracted from the video acquisition process of the cow's body shape and appearance. In the embodiment of the present application, it can be understood as a rear view image of high, medium and low angles obtained by a high-definition camera in the milking turntable area, and a side view image obtained in the cow return channel area. These images must meet the characteristics of relatively standard cow posture, small movement amplitude, clear imaging, etc. Each first image is used to input into the appearance feature detection model to identify and extract the key point position, and provide basic data for subsequent intelligent identification of body shape and appearance.

[0022] Specifically, in order to realize the intelligent identification of the body shape and appearance of the cow, it is necessary to first obtain the first images of the cow to be tested. Since the cow will have different degrees of body shaking and posture changes during milking and activities, a single frame image may not accurately reflect the real body shape and appearance characteristics of the cow. Therefore, this embodiment improves the accuracy of body shape and appearance identification by obtaining at least two frames of first images.

[0023] Optionally, in S101: acquiring each first image of the cow to be tested may further include the following steps: S201. Obtain a target video clip of the target cow to be tested, wherein the target video clip includes at least one of a video clip of the target cow to be tested in a high angle view, a middle angle view, and a low angle view of a milking turntable area, and a video clip of the target cow to be tested in a cow return channel area.

[0024] S202: Extract at least two consecutive frames of images from the target video segment as each of the first images.

[0025] Optional, see Figure 2 , Figure 2 The schematic diagram of the structure of a typical cow body shape and appearance acquisition device provided in the embodiment of the present application is mainly used to acquire the first images of the cow to be tested, and the device includes two main parts: a milking turntable area acquisition unit and a cow return channel acquisition unit.

[0026] In S201, the milking turntable area acquisition unit includes three groups of high-definition cameras distributed vertically, which are used to obtain rear-view images at three different viewing angles: high, medium, and low. The high camera is mainly used to collect upper body shape characteristics such as rear udder attachment height, rear udder attachment width, and rump width; the middle-height camera is mainly used to collect middle body shape characteristics such as chest width, rear view of hind limbs, bone texture, and central suspensory ligament; the low camera is mainly used to collect lower body shape characteristics such as front teat position and rear teat position. These three groups of cameras are fixedly installed in the milking turntable area through brackets to ensure the stability of the shooting angle.

[0027] The cow return channel collection unit is set up in the cow return channel area of ​​the milking parlor, including a high-definition camera with a side view, which is mainly used to collect the cow's body depth, hoof angle, hind limb side view, front udder attachment and angularity and other side body features. The unit is equipped with an isolation door to control the speed of the cow passing through to ensure that a clear static image can be obtained.

[0028] In order to realize individual cow identification and data association, the device also includes an identity recognition system, which consists of RFID ear tags and RFID readers. The RFID readers are installed in the milking turntable area and the cow return channel area to read the individual information of the cows wearing RFID ear tags. A ruler is equipped for image size calibration to ensure the accuracy of the measurement data.

[0029] The data transmission and storage system includes a hard disk recorder and a wireless network module. The hard disk recorder is used for local storage of video data, and the wireless network module transmits the data to the cloud platform. The collected data is processed and analyzed on the server deployed on the cloud platform to complete the identification and grade scoring of body shape and appearance trait parameters.

[0030] The working process of the device for collecting typical characteristics of the cow's body shape and appearance is as follows: first, multiple high-definition cameras are used to simultaneously collect rear-view video clips of the cow in the milking turntable area and side video clips of the cow return channel; then the individual ID information of the cow is read through the RFID system, and the videos collected from different perspectives are associated with the individual ID; finally, the collected video data is uploaded to the local server in real time for storage and processing.

[0031] Optionally, in S202, in order to select the most suitable image for body shape and appearance feature recognition from the acquired video clips, this embodiment performs frame extraction processing on the video clips. Specifically, the video clips collected from various perspectives are analyzed frame by frame through a video processing algorithm, and image frames in which the cow's four feet are on the ground and the body shaking is relatively small are selected as each first image. Since a single-frame image may be affected by factors such as shooting angle deviation, light changes, or temporary displacement of the cow's posture, at least two frames of images that meet the conditions are extracted from each video clip, and the accuracy and reliability of subsequent feature recognition are improved through the information complementarity of multiple frames of images.

[0032] Optionally, in S102, the appearance feature detection model refers to a feature detection network structure based on deep learning, which can be understood as an end-to-end detection model composed of a multi-resolution parallel convolution module, a multi-resolution fusion module, and a key point prediction module in the embodiment of the present application, and is used to accurately locate and identify key point positions corresponding to body appearance features such as rear udder attachment height, rear udder attachment width, rump width, chest width, hind limb rear view, bone texture, central suspensory ligament, front teat position, and rear teat position from the input cow body appearance image. The model achieves accurate positioning of key points of cow body appearance by maintaining high-resolution feature information and multi-scale feature fusion, providing basic data support for subsequent body appearance feature calculation and intelligent scoring.

[0033] Specifically, key points refer to the pixel coordinates that characterize specific target positions and attributes in the image. In the embodiments of the present application, they can be understood as key positions of the cow's body appearance identified and located by the appearance feature detection model, including anatomical landmarks such as the posterior udder attachment point, rump contour points, back contour points, leg joint points, and nipple position points. The key points are used to calculate body appearance parameters through the spatial distance relationship between these key points, so as to realize intelligent evaluation and quantitative analysis of the cow's body appearance.

[0034] Specifically, key points refer to characteristic position marks in the image that have clear anatomical significance. In the embodiment of the present application, they can be understood as anatomical positioning points required for marking the body shape and appearance scoring of cows, used to construct a skeletal structure model of the body shape and appearance of cows, and provide a standardized position reference for the extraction and scoring of body shape and appearance features.

[0035] Optionally, in S102, the key points include a plurality of key points characterizing the rear view and side view of the cow's body shape and appearance.

[0036] Specifically, in order to comprehensively evaluate the body shape and appearance characteristics of dairy cows, the embodiment of the present application uses key points collected from multiple perspectives for model training. Since the evaluation of the body shape and appearance characteristics of dairy cows requires comprehensive consideration of morphological characteristics from different angles, the key point information from a single perspective is difficult to fully describe the body shape and appearance characteristics of dairy cows. Therefore, the key point information from the rear view and side view is collected at the same time to obtain a more comprehensive body shape and appearance representation.

[0037] Specifically, the key points of the rear view body appearance include the hind udder attachment point (2 points), the hind udder bottom contour point (3 points), the rump contour point (4 points), the ischial tuberosity point (2 points), the hip contour point (4 points) and the hind limb joint point (6 points). These key points are mainly used to evaluate the hindquarters development, udder shape and hind limb structure of dairy cows. Side body appearance feature point dataset D2. The side feature points of dairy cows are calibrated, including 1 eye, 1 nose, 1 neck, 2 backs, 2 ischial joints, 1 tail root, 1 bottom abdomen, 9 legs, 16 hooves, etc., a total of 34 key points, which are used to evaluate the body depth, back strength and forelimb structure of dairy cows.

[0038] In the process of key point annotation, different annotation strategies are used for key points of different viewpoints and types. For obvious anatomical key points, precise annotation is performed directly; for contour key points, the key point positions are determined by equally spaced sampling. To ensure the consistency of annotation, the system provides standardized annotation guidelines and ensures the annotation quality through cross-validation by multiple experts. At the same time, considering the possible occlusion and posture changes under actual shooting conditions, the visibility information of the key points is also recorded during the annotation process for sample weight allocation during subsequent model training.

[0039] By collecting key points from multiple perspectives, not only the feature dimension of model training is expanded, but also the correspondence between key points from different perspectives is established. This multi-perspective key point information can help the model learn a more robust feature representation and improve the accuracy and generalization ability of key point detection. At the same time, multi-perspective key point information also provides more comprehensive data support for subsequent body shape and appearance feature calculations, which helps to achieve more accurate and reliable body shape and appearance assessments.

[0040] Furthermore, the appearance feature detection model is trained by sample cow images and sample labels, and the sample labels are pixel points corresponding to key points that characterize the body appearance features of the cows in the sample cow images.

[0041] The sample label refers to the manually labeled information of the key points of the body appearance features in the sample cow image. In the embodiment of the present application, it can be understood as a set of key point coordinates characterizing the cow's body appearance labeled by professionals, which is used to train the appearance feature detection model so that the model can accurately locate and identify the key point positions of the cow's body appearance.

[0042] Specifically, in order to build a high-precision appearance feature detection model, the embodiment of the present application adopts a supervised training method based on deep learning. Since the accurate positioning of the key points of the cow's body appearance directly affects the subsequent body appearance evaluation results, it is necessary to establish a large-scale and accurately labeled training data set, so that the model can master the detection rules of the key points through deep learning.

[0043] During the sample annotation stage, experienced professionals use specially developed annotation tools to annotate key points. The annotation content includes two parts: key points of rear view body shape and appearance and key points of side view body shape and appearance. For rear view key points, the key annotations include the rear udder attachment point, rear udder bottom contour point, rump contour point and other positions; for side view key points, the main annotations include back contour point, abdomen contour point, limb and hoof joint point and other positions. To ensure the quality of annotation, a multi-person cross-validation method is used to discuss and correct controversial annotation points.

[0044] In the dataset construction stage, the key point datasets D1 and D2 of the cow's rear view and side view were established respectively. For the side view, 34 key points including eyes, nose, neck, etc. were annotated, which cover the main characteristic positions of the cow's side view. For the rear view, key points such as the central suspensory ligament, the position of the front and rear nipples, the height and width of the rear udder attachment, and the width of the buttocks were annotated. The annotation information is stored in Json format, including information such as image path, key point coordinates, point type, and visibility. To ensure the generalization ability of the model, the dataset is divided into training set and test set in a ratio of 8:2, while ensuring that the cow samples with different body characteristics are evenly distributed in the training set and test set.

[0045] After the annotation is completed, the system preprocesses and screens the annotated data. First, the integrity and rationality of the annotation points are checked, and samples with incomplete or obviously wrong annotations are removed. Then, data enhancement processing is performed on the retained samples, including random rotation, scaling, brightness adjustment and other operations to expand the diversity of training data. At the same time, in order to deal with occlusion and posture change problems in practical applications, occlusion simulation and posture transformation are added to some samples to improve the robustness of the model.

[0046] During the model training process, the processed sample data is divided into a training set and a validation set in a ratio of 8:2. The training adopts a batch learning method, and each batch contains multiple sample images and their corresponding annotation information. The model optimizes parameters by minimizing the loss function between the predicted key point position and the annotated position. The loss function design takes into account the importance of different types of key points and gives higher weights to key points related to key traits.

[0047] In order to improve the generalization ability of the model, a multi-stage strategy is adopted in the training process. First, it quickly converges to a better parameter area at a larger learning rate, and then gradually reduces the learning rate for fine-tuning. At the same time, an early stopping strategy is introduced. When the performance indicators on the validation set have not been significantly improved for multiple consecutive rounds, the training process is stopped in time. In addition, by regularly saving model checkpoints, the model parameters with the best verification performance can be selected for actual deployment.

[0048] Through the above-mentioned model training method based on sample labels, the system successfully built a deep learning model that can accurately detect the key points of the cow's body shape and appearance. This data-driven method not only inherits the professional experience of manual annotation, but also obtains stronger generalization ability through large-scale data training. The model can adapt to individual cows with different shooting conditions and body characteristics, providing reliable key point positioning support for subsequent intelligent evaluation of body shape and appearance. At the same time, the standardized data collection and annotation process ensures the quality of the training data, making the model more practical and reliable.

[0049] Optional, see Figure 3 , Figure 3 A schematic diagram of the structure of an appearance feature detection model provided in an embodiment of the present application. The appearance feature detection model includes a multi-resolution parallel convolution module, a multi-resolution fusion module and a key point prediction module; Wherein, the multi-resolution parallel convolution module is used to extract high-resolution features and low-resolution features in each of the first images.

[0050] Wherein, the multi-resolution fusion module is used to generate a multi-resolution feature map based on the high-resolution features and the low-resolution features.

[0051] Wherein, the key point prediction module is used to determine the corresponding key points in the multi-resolution feature map.

[0052] Optionally, the multi-resolution parallel convolution module refers to a multi-branch parallel feature extraction network structure, which can be understood as a parallel convolution network composed of four processing stages in the embodiment of the present application, wherein each stage contains multiple parallel branches, and simultaneously processes input features through different downsampling rates, and gradually adds new low-resolution branches while maintaining the original resolution branches in each processing stage, so as to gradually generate new low-resolution feature branches while maintaining the high-resolution features of the original image, thereby simultaneously obtaining the detailed feature information and semantic feature information of the image.

[0053] Specifically, in order to simultaneously obtain detail information and semantic information in the cow body appearance image, this embodiment uses a multi-resolution parallel convolution module to extract features from each first image. The traditional single feature extraction network will lose a lot of detail information during the downsampling process, and the body appearance key point detection needs to rely on both local detail features and global semantic features. Therefore, a parallel multi-resolution processing structure is used to maintain and extract feature information of different scales.

[0054] Specifically, the multi-resolution parallel convolution module divides the feature extraction process into four processing stages. In the first processing stage, each first input image is processed through a convolution operation to maintain the original resolution and extract low-level features containing information such as color, texture, and edge. In the second processing stage, one branch continues to maintain high-resolution feature processing, and the other branch reduces the feature map size to 1 / 2 of the original through a convolution operation with a step size of 2, forming a new low-resolution feature branch. In the third processing stage, on the basis of maintaining the first two branches, a new branch is added to further reduce the feature map size to 1 / 4 of the original. In the fourth processing stage, a new branch is added again to reduce the feature map size to 1 / 8 of the original.

[0055] Each branch in each processing stage contains multiple convolutional layers, which are connected through batch normalization layers and activation functions. The high-resolution branch uses smaller convolution kernels and fewer channels to focus on extracting local features, while the low-resolution branch uses larger convolution kernels and more channels to focus on extracting features with larger receptive fields. The above design enables different branches to capture feature information of different scales: the high-resolution branch maintains rich spatial details, which is conducive to precise positioning; the low-resolution branch provides stronger semantic information, which is helpful for the recognition of key points.

[0056] The above parallel multi-resolution feature extraction method not only avoids the information loss problem in the traditional feature extraction network, but also realizes the effective extraction of multi-scale features of each first image. The high-resolution features retain the fine spatial information required to locate the key points, and the low-resolution features provide the semantic information required to understand the overall structure. The combination of the two provides a more complete and rich feature representation for subsequent feature fusion and key point detection.

[0057] Optionally, the multi-resolution fusion module refers to a feature fusion network structure, which can be understood in the embodiment of the present application as a densely connected network constructed by convolutional layers, batch normalization layers and upsampling operations, which interacts and fuses the information of feature branches of different resolutions at the end of each processing stage to achieve an effective combination of high-resolution features and low-resolution features, thereby ensuring that high-resolution detail information and low-resolution semantic information can be fully utilized, improving feature expression capabilities, and providing a more complete feature representation for subsequent key point predictions.

[0058] Specifically, in order to make full use of the different scale feature information extracted by the multi-resolution parallel convolution module, this embodiment realizes the effective fusion of high-resolution features and low-resolution features through a multi-resolution fusion module. Since the key point detection of the cow's body appearance requires both accurate position information and reliable semantic understanding, it is necessary to reasonably fuse the high-resolution features that retain the detail information and the low-resolution features that contain the semantic information to form a feature representation with multi-scale expression capabilities.

[0059] Specifically, the multi-resolution fusion module performs feature fusion operations at the end of each processing stage. First, the feature map of each resolution branch is adjusted through a 1×1 convolution layer to reduce the feature dimension and reduce the amount of calculation. Then, the features are standardized through a batch normalization layer to improve the expressiveness of the features and the convergence speed of the model. For the low-resolution feature branch, the upsampling operation of bilinear interpolation is used to adjust the size of the feature map to the same as the high-resolution feature, ensuring that the feature maps of different resolutions have the same spatial dimensions.

[0060] After feature alignment, the multi-resolution fusion module uses dense connections to achieve feature fusion. Specifically, the feature maps of all resolution levels are spliced ​​in the channel dimension to form a feature representation containing multi-scale information. In order to further enhance the expressiveness of the features, the fused features are reorganized through a 3×3 convolutional layer, which can learn the correlation between features of different resolutions and generate more discriminative feature representations. Finally, the original feature information is introduced into the fusion process through residual connections to avoid the gradient vanishing problem during information transmission and maintain the training stability of the model.

[0061] Through the above feature fusion method, the fine spatial information in the high-resolution features and the semantic information in the low-resolution features are fully interacted and fused. The generated multi-resolution feature map not only maintains the spatial details required for accurate positioning of key points, but also contains the semantic information required to understand the overall structure. This multi-scale feature representation greatly improves the model's efficiency in utilizing features of different scales, provides more comprehensive and reliable feature information for subsequent key point prediction, and effectively improves the accuracy and robustness of key point detection.

[0062] Optionally, the key point prediction module refers to a key point positioning structure based on a heat map, which can be understood in the embodiment of the present application as a prediction network that converts the fused multi-resolution feature map into a body shape and appearance key point heat map. The network adjusts the feature dimension to the same number of channels as the number of key points to be predicted through a convolutional layer, and applies a Gaussian kernel to each channel to generate a heat map distribution, which is used to represent and predict the key point positions in the form of a heat map, thereby achieving accurate positioning of the key points of the cow's body shape and appearance, and at the same time reflecting the confidence of the prediction through the probability distribution of the heat map, providing position coordinate information for subsequent body shape and appearance feature calculations and intelligent scoring.

[0063] Specifically, in order to convert the fused multi-resolution feature map into specific body shape and appearance key point position information, the embodiment of the present application uses a key point prediction module to perform the final position prediction. Since the key points of the cow's body shape and appearance have precise positioning requirements, the method of directly predicting the coordinate values ​​is easily affected by data noise. Therefore, a prediction method based on a heat map is used to "softly mark" the key point positions through Gaussian distribution to improve the fault tolerance and stability of the prediction.

[0064] Optionally, the key point prediction module is specifically used to generate a body shape and appearance key point heat map based on corresponding key points in the multi-resolution feature map, and obtain key points in the body shape and appearance key point heat map.

[0065] Among them, the body shape and appearance key point heat map refers to a key point position representation method based on probability distribution, which can be understood as a two-dimensional probability map generated by a key point prediction module in the embodiment of the present application, in which each channel corresponds to a body shape and appearance key point to be predicted, and the key point position is "softly labeled" through a Gaussian kernel, so that each pixel value in the heat map represents the confidence score of the position as a key point, which is used to expand the discrete key point coordinates into a continuous probability distribution representation, provide uncertainty information for position prediction, and thus enhance the model's fault tolerance and robustness for key point positioning.

[0066] Specifically, in order to accurately extract key point location information from the generated multi-resolution feature map and improve the robustness of positioning, the embodiment of the present application adopts a key point prediction method based on a heat map. Since predicting key point coordinates directly on the feature map is easily affected by noise and local interference, the discrete coordinate point representation is expanded into a continuous probability distribution by generating a heat map of body shape and appearance key points, thereby enhancing the model's tolerance to position deviations.

[0067] Specifically, the key point prediction module first adjusts the channel dimension of the input multi-resolution feature map, and converts the number of feature channels into the same dimension as the number of key points to be predicted through a 1×1 convolution layer. Each channel corresponds to a specific body shape and appearance key point. Then, a Gaussian kernel is applied to the feature map of each channel for convolution operation to generate a heat map representation of the corresponding key point. The standard deviation parameter of the Gaussian kernel determines the diffusion range of the heat map. A larger standard deviation will produce a smoother heat map distribution, which is conducive to processing small deviations in the key point position; a smaller standard deviation will generate a more concentrated heat map distribution, which is conducive to improving the accuracy of position prediction.

[0068] After generating the heat map of key points of body shape and appearance, the key point prediction module performs local maximum detection on the heat map by setting a sliding window. The selection of window size needs to consider the minimum spacing between key points and the resolution of the heat map. A window that is too large may miss adjacent key points, and a window that is too small may lead to repeated detection. In this embodiment, based on the distribution characteristics of the key points of the body shape and appearance of dairy cows, an adaptive window size is adopted, and different search ranges are adopted for different types of key points. For example, for key points with higher accuracy requirements such as nipple position, a smaller search window is used; for key points such as back contour, a larger search window is used.

[0069] This heat map-based key point detection method not only expands the pixel representation range of key points and improves the prediction tolerance, but also reflects the uncertainty of position prediction through heat map distribution. The value of each pixel position in the generated heat map represents the confidence score of the position as a key point. The peak position of the heat map corresponds to the most likely position of the key point, and the intensity distribution of the heat map reflects the reliability of the position prediction. This representation method provides more reference basis for subsequent key point positioning and screening, effectively improving the accuracy and stability of key point detection.

[0070] Optionally, in S103, the association relationship refers to a mathematical representation that describes the temporal and spatial correspondence and interdependence between different data elements, which can be understood in the embodiment of the present application as a temporal correspondence and motion constraint between key points in each first image of consecutive frames, including a matching relationship between key points in adjacent frames, continuity of motion trajectories, and a spatial constraint relationship based on a priori structure of the cow's body shape and appearance skeleton, which is used to establish an association mapping of key points in time and space dimensions, thereby providing a constraint basis for the optimal correction of the key point positions, ensuring that the corrected key point distribution conforms to both temporal continuity and biological structural characteristics.

[0071] Optionally, in S103: based on the correlation relationship between the key points in the first images of at least two consecutive frames, correcting the key points in the first images to obtain the target key points may further include the following steps: S301: constructing association relationships between key points in adjacent first images based on similarities of key points in adjacent first images.

[0072] S302. Based on the prior structure and association relationship of the cow's body appearance skeleton, coordinate correction is performed on the key points in the target image to obtain target key points, wherein the prior structure of the cow's body appearance skeleton is used to characterize at least one of the spatial position relationship, length ratio and angle range between the key points of the cow's body appearance.

[0073] Optionally, in S301, in order to accurately establish the correspondence between key points in consecutive frames, the embodiment of the present application adopts a feature matching method based on similarity measurement to construct an association relationship. Since the same key point will have a position shift between adjacent frames during the movement of the cow, but its local features and context information have a certain consistency, so a reliable matching relationship can be established by analyzing the image features of the area around the key point.

[0074] Specifically, the local feature descriptor is first extracted for each key point. The system extracts a fixed-size image block centered on the key point, encodes the image block through a pre-trained convolutional neural network, and generates a high-dimensional feature vector. This feature description method not only includes the local texture and edge information of the key point, but also integrates the contextual features of a larger receptive field through multi-layer convolution operations, improving the distinguishing ability and robustness of the feature representation.

[0075] Then, the similarity matrix between key points of adjacent frames is calculated. For each first image of the two adjacent frames, the cosine similarity between the feature descriptors of each pair of key points is calculated to construct a similarity score matrix. Considering the continuity characteristics of cow movement, the system sets a maximum displacement threshold to exclude matching candidates that exceed the reasonable range of movement. At the same time, in order to handle the missing key points due to occlusion or detection failure, key points are allowed to perform one-to-many or many-to-one soft matching, and the confidence score of each pair of matches is recorded.

[0076] After the initial establishment of the matching relationship, motion consistency constraints are introduced to optimize the matching. Based on the displacement vectors between the matching pairs, the motion pattern of the local area is estimated, and abnormal matches that are inconsistent with the main motion trend are detected. For matching pairs with low confidence or abnormal motion patterns, the system will re-evaluate their matching relationships and adjust the matching relationships if necessary. At the same time, by analyzing the motion trajectories of key points in multiple consecutive frames, a simple motion prediction model is established to verify whether the newly established matching relationship conforms to the expected motion trend.

[0077] Through the above similarity-based key point matching method, the system can effectively establish the association relationship between key points in consecutive frames. This matching method not only takes into account the local appearance characteristics of key points, but also combines motion continuity constraints, significantly improving the accuracy and reliability of matching. The established association relationship provides important timing constraint information for subsequent key point position correction, helps to suppress noise and errors in single-frame detection, and ensures that the final key point trajectory has good temporal continuity.

[0078] Optionally, in S302, the prior structure of the cow's body appearance skeleton refers to a bone structure constraint model established based on the anatomical characteristics of the cow, which can be understood in the embodiment of the present application as a mathematical representation of at least one of the spatial position relationship, length ratio and angle range between the key points of the cow's body appearance, including prior knowledge such as distance constraints between adjacent bone nodes, joint motion range limitations and local structural morphological characteristics, which is used to constrain the rationality of the key point detection results and ensure that the predicted key point distribution conforms to the biological characteristics of the cow.

[0079] In order to ensure that the key point positions are consistent with the biological characteristics of the cow's body shape and appearance, the embodiment of the present application combines the cow's body shape and appearance skeleton prior structure and the established association relationship to correct the coordinates of the key points. Since the detection results based solely on image features may not conform to anatomical characteristics, it is necessary to introduce skeleton prior knowledge as a constraint condition, and use time series association information to ensure the continuity of the key point positions.

[0080] Specifically, we first construct a priori structure of the cow's body appearance skeleton. By analyzing a large amount of labeled data, we extract the statistical characteristics of the bone structure, including the length ratio between adjacent bone nodes, the range of motion of the joints, and the morphological characteristics of the local structure. For example, the distance ratio between the rear udder attachment point and the ischial tuberosity, the angle range between the leg joints, and the smoothness of the spine line. These prior knowledge are encoded in the form of probability distribution to evaluate the rationality of the key point distribution.

[0081] Then, an optimization objective function is constructed based on the skeleton prior structure and association relationship. The objective function contains three main items: detection confidence item, skeleton constraint item and temporal consistency item. The detection confidence item reflects the degree of deviation of the key point from the original detection position; the skeleton constraint item measures the degree of violation of the current key point distribution to the skeleton prior structure; and the temporal consistency item evaluates the degree of conformity of the key point position with the association relationship established with adjacent frames. These three items are balanced by weight coefficients, and the weight setting takes into account the importance and reliability of different constraints.

[0082] During the optimization process, the coordinates of key points are updated by iterative solution. Each iteration first calculates the objective function value based on the current key point distribution, then calculates the gradient of the objective function for each key point coordinate, and updates the key point position according to the gradient information. In order to improve the optimization efficiency and avoid falling into the local optimal solution, a multi-scale optimization strategy is adopted, first making a rough adjustment on a larger spatial scale, and then gradually refining it to the precise position optimization. At the same time, a dynamic constraint weight mechanism is introduced to dynamically adjust the weights of each constraint according to the confidence of the key point and the degree of constraint violation during the optimization process.

[0083] Finally, the optimized key point positions are verified and fine-tuned. By calculating the deviation between the optimized results and the original detection results, as well as the conformity with the skeleton prior model, key points with possible problems are screened out. For these key points, the system will re-evaluate their positions and make local optimization adjustments if necessary to ensure that the final key point distribution conforms to both image features and skeleton constraints.

[0084] Through the above coordinate correction method based on prior structure and association relationship, the system can effectively optimize the spatial distribution of key points. The corrected key points not only maintain consistency with the image features, but also meet the biological characteristics of the cow's body shape and appearance, while having good temporal continuity. This multi-constraint optimization method significantly improves the accuracy and reliability of key point positioning, and provides more accurate location information for subsequent body shape and appearance feature calculations.

[0085] Optionally, S104, determining the body shape and appearance of the cow to be tested based on the pixel distance between the target key points.

[0086] Pixel point distance refers to the Euclidean distance between the coordinates of two key points in the image. In the embodiment of the present application, it can be understood as the spatial distance relationship between the target key points. It is used to quantify various trait parameters that characterize the body shape and appearance of dairy cows, including key measurement indicators such as body height, body depth, rump angle, and hoof angle, providing an objective numerical basis for the intelligent scoring of the body shape and appearance of dairy cows.

[0087] Specifically, in order to convert the detected and corrected target key point information into body shape and appearance parameters that can be used for evaluation, the embodiment of the present application calculates the relevant trait indicators based on the pixel distance between the key points. Since the traditional manual scoring method is easily affected by subjective factors and the measurement process is time-consuming and cumbersome, a mapping model of the key point spatial relationship to the body shape and appearance traits is established to obtain the angle formed by the target key points, and the body shape and appearance are determined based on the angle.

[0088] Specifically, first, a calculation model based on key points is established for each body shape and appearance trait. Please refer to Table 1. Table 1 is a standard table for calculating different shapes provided in an embodiment of the present application.

[0089] Table 1 Calculation standard table for different shapes

[0090] For linear distance traits, such as body depth, the vertical distance between the key point of the bottom abdomen and the key point of the lumbar spine at the last rib is calculated; for angle traits, such as rump angle, the relative height difference is calculated by extracting the coordinates of the ischial node and the hip node; for complex traits, such as front udder attachment, the angle between the front edge of the udder and the turning point of the abdominal wall of the trunk is calculated. To ensure the accuracy of the measurement, the system converts the pixel distance into actual units using the pre-calibrated reference size in the image.

[0091] After obtaining the quantitative indicators of each trait, the system maps these values ​​to the standard scoring system. By analyzing a large amount of expert scoring data, the corresponding relationship between trait values ​​and scoring levels is established. Considering the differences in scoring standards for different traits, the system designs a specific mapping function for each trait to convert continuous measurement values ​​into discrete scoring levels. At the same time, by introducing a smooth transition interval, the impact of measurement errors on the final scoring results is reduced.

[0092] In addition, the system also considers the relationship between different body shape and appearance traits. By analyzing the statistical correlation between traits, a comprehensive scoring model is established to ensure the overall consistency of the scores of various traits. For example, the scores of related traits such as udder depth and hind limb side view need to be considered in a coordinated manner to avoid scoring combinations that do not conform to physiological characteristics. This multi-trait coordinated scoring method not only improves the accuracy of the scoring, but also reflects the overall condition of the cow's body shape and appearance.

[0093] Through the above-mentioned method of determining body shape and appearance based on the distance of target key points, the system realizes the automated and standardized evaluation of the body shape and appearance of dairy cows. This method not only improves the efficiency and objectivity of scoring, but also saves detailed measurement data in a digital way, providing reliable data support for subsequent breeding selection and management decisions. At the same time, the scalability design of the system enables it to flexibly adjust the scoring criteria according to different breeding goals to meet the personalized needs of different ranches.

[0094] Please refer to Figure 4 , Figure 4 A schematic diagram of a structure of a device for determining the body shape and appearance of a dairy cow based on key point detection provided in an embodiment of the present application includes the following modules: A first image acquisition module, used to acquire first images of the cow to be tested, each of the first images including at least two consecutive frames of images; A key point determination module, used for inputting each of the first images into an appearance feature detection model, and obtaining key points output by the appearance feature detection model, wherein the key points are pixel points representing the appearance features of the cow to be tested; A target key point determination module, configured to correct the key points in each of the first images to obtain the target key point based on the correlation between the key points in at least two consecutive frames of the first images; A body shape and appearance determination module, used to determine the body shape and appearance of the cow to be tested based on the pixel point distance between the target key points; The appearance feature detection model is trained by sample cow images and sample labels, and the sample labels are pixel points corresponding to key points in the sample cow images that characterize the cow's body shape and appearance features.

[0095] Optionally, the key point determination module is also used to extract high-resolution features and low-resolution features in each of the first images through the multi-resolution parallel convolution module; generate a multi-resolution feature map based on the high-resolution features and the low-resolution features through the multi-resolution fusion module; and determine the corresponding key points in the multi-resolution feature map through the key point prediction module.

[0096] Optionally, the key point determination module is also used to generate a body shape and appearance key point heat map based on the corresponding key points in the multi-resolution feature map through the key point prediction module, and obtain the key points in the body shape and appearance key point heat map; wherein the body shape and appearance key point heat map is used to expand the pixel range of the key points in the multi-resolution feature map.

[0097] Optionally, the key points include multiple key points that characterize the rear view and side view of the cow's body shape and appearance.

[0098] Optionally, the first image acquisition module is further used to acquire a target video clip of the cow to be tested, the target video clip including at least one of the video clips of the target cow to be tested in a high perspective, a middle perspective and a low perspective of the milking turntable area, and in a cow return channel area; and extract at least two consecutive frames of images from the target video clip as each first image.

[0099] Optionally, the target key point determination module is also used to construct an association relationship between the key points in each adjacent first image based on the similarity of the key points in each first image; based on the prior structure of the cow's body appearance skeleton and the association relationship, the key points in the target image are coordinate corrected to obtain the target key points.

[0100] Please refer to Figure 5 , Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the method for determining the body shape and appearance of a dairy cow based on key point detection.

[0101] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0102] On the other hand, the present application also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for determining the body shape and appearance of a cow based on key point detection provided by the above methods.

[0103] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for determining the body shape and appearance of a cow based on key point detection provided by the above-mentioned methods.

[0104] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0105] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for determining the body shape and appearance of a dairy cow based on key point detection, characterized in that: include: Acquire at least two consecutive first image frames of the cow to be tested; Input each of the first images into an appearance feature detection model, respectively, and obtain key points of each of the first images output by the appearance feature detection model, wherein the key points are pixel points representing the appearance features of the cow to be tested; Based on the correlation relationship between the key points of each of the first images, correcting the key points to obtain target key points; Determining the body shape and appearance of the cow to be tested based on the pixel distances between the target key points of each of the first images; The appearance feature detection model is trained by sample cow images and sample labels, and the sample labels are pixel points corresponding to key points in the sample cow images that characterize the cow's body shape and appearance features.

2. The method for determining the body shape and appearance of a dairy cow based on key point detection according to claim 1, characterized in that: The appearance feature detection model includes a multi-resolution parallel convolution module, a multi-resolution fusion module and a key point prediction module; The multi-resolution parallel convolution module is used to extract high-resolution features and low-resolution features in each of the first images; The multi-resolution fusion module is used to generate a multi-resolution feature map based on the high-resolution features and the low-resolution features; The key point prediction module is used to determine the corresponding key points in the multi-resolution feature map.

3. The method for determining the body shape and appearance of a dairy cow based on key point detection according to claim 2, characterized in that: The key point prediction module is specifically used to generate a body shape and appearance key point heat map based on the corresponding key points in the multi-resolution feature map, and obtain the key points in the body shape and appearance key point heat map; The body shape and appearance key point heat map is used to expand the pixel range of the key points in the multi-resolution feature map.

4. The method for determining the body shape and appearance of a dairy cow based on key point detection according to claim 1, characterized in that: The key points include a plurality of key points characterizing the rear view and side view of the cow's body shape and appearance.

5. The method for determining the body shape and appearance of a dairy cow based on key point detection according to claim 1, characterized in that: The step of obtaining each first image of the cow to be tested comprises: Obtaining a target video clip of the cow to be tested, the target video clip comprising at least one of a video clip of the cow to be tested in a high angle of view, a middle angle of view, and a low angle of view in a milking turntable area, and a video clip of the cow in a cow return channel area; At least two consecutive frames of images are extracted from the target video segment as first images.

6. The method for determining the body shape and appearance of a dairy cow based on key point detection according to claim 1, characterized in that: The step of correcting the key points in each of the first images to obtain the target key points based on the correlation between the key points in each of the first images of at least two consecutive frames includes: Based on the similarity of the key points in the adjacent first images, constructing the association relationship between the key points in the first images; Based on the prior structure of the cow's body appearance skeleton and the association relationship, the key points in the target image are corrected in coordinates to obtain target key points, wherein the prior structure of the cow's body appearance skeleton is used to characterize at least one of the spatial position relationship, length ratio and angle range between the key points.

7. A device for determining the body shape and appearance of a dairy cow based on key point detection, characterized in that: include: A first image acquisition module, used to acquire each of the first images of the cow to be tested, each of the first images includes at least two consecutive frames of images; A key point detection module, used for inputting each of the first images into an appearance feature detection model, and obtaining key points output by the appearance feature detection model, wherein the key points are pixel points representing the appearance features of the cow to be tested; A target key point detection module, configured to correct the key points in each of the first images to obtain the target key points based on the correlation between the key points in at least two consecutive frames of the first images; A body shape and appearance identification module, used to determine the body shape and appearance of the cow to be tested based on the pixel distance between the target key points; The appearance feature detection model is obtained by training key points, and the key points are positioning points that characterize the body appearance features of dairy cows.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for determining the body shape and appearance of a dairy cow based on key point detection as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for external identification of a cow's body shape based on key point detection as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for determining the body shape and appearance of a dairy cow based on key point detection as described in any one of claims 1 to 6 is implemented.

Images