Multi-view beef cattle individual oestrus behavior monitoring method, system and equipment based on ReID and Yolov8

Abstract

The invention discloses a multi-view beef cattle individual oestrus behavior monitoring method, system and device based on ReID and Yolov8, and belongs to the technical field of deep learning target detection. Comprising the following steps: acquiring activity videos of beef cattle in different scenes, and labeling behavior data in the activity videos; preprocessing the marked behavior data to obtain a training data set; training a beef cattle re-identification model GCN-ReID based on the training data set; performing multi-view identity recognition on beef cattle individuals in a to-be-detected area through a beef cattle re-recognition model GCN-ReID to obtain beef cattle identity information; based on cattle identity information, through an estrus behavior detection and early warning model CowEstrNet based on Yolov8, behavior characteristics of beef cattle are extracted, estrus behaviors are identified, a monitoring result is output in real time, and timely early warning is provided. And the limitation in cross-view oestrus monitoring in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection based on deep learning, and more specifically to a method, system and device for monitoring estrus behavior of individual beef cattle from multiple perspectives based on ReID and Yolov8. Background Art

[0002] Accurate monitoring of estrus behavior of beef cattle is a key link in improving reproductive efficiency and production benefits. At present, there are many technical means for monitoring estrus in beef cattle, mainly focusing on behavioral monitoring, video image recognition and sensor detection. For example, by wearing sensors to monitor the activity level, standing time and other behavioral characteristics of cattle, video image recognition systems are used to capture typical behaviors during estrus.

[0003] However, wearing sensors (such as smart collars) increases equipment costs and maintenance difficulties. Reliance on a single device may lead to insufficient differentiation of certain behaviors, and using multiple devices will cause additional pressure on cows, affecting cost control and animal welfare. Video image recognition methods are mostly limited to single-perspective applications and have limited applicability in cross-perspective monitoring. Environmental changes (such as lighting, background) and changes in the cow's body condition can easily lead to a decrease in monitoring accuracy, especially in large-scale ranches or different regional scenarios. The adaptability and generalization capabilities of the technology need to be improved urgently.

[0004] Therefore, how to solve the limitations of the existing technology in cross-perspective estrus monitoring and provide a multi-perspective beef cattle individual estrus behavior monitoring method, system and equipment based on ReID and Yolov8 is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a multi-perspective beef cattle individual estrus behavior monitoring method, system and device based on ReID and Yolov8 to solve the limitations of the prior art in cross-perspective estrus monitoring.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A multi-view individual estrus behavior monitoring method for beef cattle based on ReID and Yolov8, comprising the following steps:

[0008] S100: Obtain activity videos of beef cattle in different scenarios, and annotate behavior data in the activity videos;

[0009] S200: preprocessing the labeled behavior data to obtain a training data set;

[0010] S300: Training the beef cattle re-identification model GCN-ReID based on the training dataset;

[0011] S400: Using the beef cattle re-identification model GCN-ReID to perform multi-view identification of beef cattle individuals in the test area to obtain cattle identity information;

[0012] S500: Based on the cattle identity information, the estrus behavior detection and early warning model CowEstrusNet based on Yolov8 is used to extract the behavioral characteristics of beef cattle, identify estrus behavior, and output the monitoring results in real time to provide timely early warning.

[0013] Furthermore, the S200 includes:

[0014] S210: Using adaptive histogram equalization CLAHE to handle uneven illumination in behavioral data;

[0015] S220: Use the Gaussian mixture model (GMM) to build a pasture background model to identify and remove static background in the behavior data;

[0016] S230: Use the Yolov8 model to automatically generate cow and cow face bounding boxes for images in the behavior data, wherein the data labels of the cow and cow face bounding boxes use the cow ear tag number as a unique identifier; at the same time, the key parts of the cow's body are marked to obtain a training data set.

[0017] Furthermore, the beef cattle re-identification model GCN-ReID includes: a posture estimation learning module, a Transformer learning module, a convolution learning module and a graph convolution module GCM;

[0018] The posture estimation learning module is used to estimate the posture information and joint positions of cattle and extract key point features of cattle;

[0019] The convolutional learning module extracts fine-grained local features of cattle based on the ResNet network;

[0020] The Transformer learning module is used to establish global dependencies between different local features to obtain global features;

[0021] The graph convolution module GCM fuses local features, global features and key point features of cattle to obtain cattle identity information.

[0022] Furthermore, the Yolov8-based estrus behavior detection and early warning model CowEstrusNet includes: a Yolov8 detection module, a body part detection module, an estrus behavior detection module and a behavior analysis module;

[0023] Wherein, based on the identity information of the cattle, the specific position of the cattle in the image in the video is determined using the Yolov8 detection module, and the specific position information is input into the body part detection module;

[0024] The body part detection module is used to locate key body parts of beef cattle and extract features related to estrus behavior by detecting the joint movement of multiple parts;

[0025] The estrus behavior detection module extracts the movement characteristics and coordinate characteristics of the cattle, identifies the body interactions related to the estrus behavior, and calculates the relative positions between multiple body parts based on the positions of the nearest neighbor cattle to capture the signs of estrus;

[0026] The behavior analysis module is used to draw the cumulative timeline of estrus events for each cow, identify estrus intervals and non-estrus intervals, and use confusion matrices to analyze the detection accuracy of each cow, and perform estrus discrimination and sorting of cows according to estrus frequency.

[0027] A multi-view individual estrus behavior monitoring system for beef cattle based on ReID and Yolov8, including:

[0028] Data acquisition module: obtains activity videos of beef cattle in different scenarios and annotates the behavioral data in the activity videos;

[0029] Data preprocessing module: preprocess the labeled behavior data to obtain the training data set;

[0030] Model training module: train the beef cattle re-identification model GCN-ReID based on the training dataset;

[0031] Cattle identification module: The beef cattle re-identification model GCN-ReID is used to perform multi-view identification of individual beef cattle in the test area to obtain cattle identity information;

[0032] Estrus behavior recognition module: Based on the cattle identity information, through the estrus behavior detection and early warning model CowEstrusNet based on Yolov8, the behavioral characteristics of beef cattle are extracted, estrus behavior is identified, and the monitoring results are output in real time to provide timely warnings.

[0033] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a multi-viewpoint individual estrus behavior monitoring method for beef cattle based on ReID and Yolov8 is implemented.

[0034] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a method, system and device for monitoring the estrus behavior of individual beef cattle from multiple perspectives based on ReID and Yolov8. The ReID technology is used to accurately identify individual beef cattle in different scenes to achieve multi-perspective identity tracking; the Yolov8 model is used to identify the specific behavioral characteristics of beef cattle during estrus, ensuring efficient and real-time estrus behavior monitoring. The specific beneficial effects are:

[0035] (1) In view of the limited applicability and poor generalization of beef cattle estrus behavior monitoring technology in multi-view monitoring, for the first time, beef cattle individual weight recognition technology, estrus behavior determination and early warning algorithm are combined and applied to multi-view monitoring of estrus behavior, breaking through the technical bottleneck caused by perspective differences. ReID technology is used to accurately identify beef cattle individuals in different scenes, and cross-scene identity tracking is achieved; based on Yolov8, which is used to identify the specific behavioral characteristics of beef cattle during estrus, the estrus behavior of beef cattle can be monitored efficiently and in real time, providing a more intelligent and automated technical idea for precision breeding of beef cattle.

[0036] (2) To address the problems of local occlusion, posture change and individual appearance similarity in the process of individual beef cattle recognition, a GCN-ReID model that includes posture estimation, local and global feature extraction and graph convolution fusion was constructed. By fusion of key point features and multi-layer features, the posture estimation learning module, Transformer learning module, convolution learning module and graph convolution module (GCM) were utilized to improve the accuracy and robustness of cattle re-identification under multiple perspectives.

[0037] (3) In order to solve the problem that estrus behavior of beef cattle is difficult to accurately identify and early warn, a beef cattle estrus detection and warning model CowEstrusNet based on Yolov8 is provided to solve the problem that estrus behavior is difficult to accurately identify and early warn. The model CowEstrusNet consists of three modules: the body part detection module accurately locates the key body parts of cattle (nose, body, tail root) and extracts estrus characteristics through the joint movement of multiple parts; the estrus behavior detection module combines the movement and relative position of cattle to capture typical estrus behaviors such as straddling and sniffing; the behavior analysis module analyzes the estrus frequency through timeline and confusion matrix to provide decision support for cattle management and reproduction. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0039] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0040] Figure 2 A schematic diagram of the structure of the beef cattle re-identification model GCN-ReID provided in an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of the structure of the estrus behavior detection and early warning model CowEstrusNet based on Yolov8 provided in an embodiment of the present invention;

[0042] Figure 4 It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] The purpose of the present invention is to provide a multi-perspective beef cattle individual estrus behavior monitoring method, system and equipment based on ReID and Yolov8. First, the activity video of beef cattle is collected in different scenes by a camera device, and the behavior data is annotated to train the algorithm. Secondly, multi-perspective identity recognition of beef cattle is performed based on ReID technology to ensure that the identity of the individual can be accurately confirmed no matter in which scene the individual appears. Finally, the estrus behavior detection and early warning model based on Yolov8 is used to extract the behavioral characteristics of beef cattle, identify estrus-related behaviors, and output the monitoring results in real time to provide timely early warnings, thereby improving reproductive efficiency. Provide a solution to the limitations of existing technologies in cross-perspective estrus monitoring.

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] See also Figure 1 The embodiment of the present invention discloses a multi-view individual estrus behavior monitoring method for beef cattle based on ReID and Yolov8, comprising the following steps:

[0047] S100: Obtain activity videos of beef cattle in different scenarios, and annotate behavior data in the activity videos;

[0048] S200: preprocessing the labeled behavior data to obtain a training data set;

[0049] S300: Training the beef cattle re-identification model GCN-ReID based on the training dataset;

[0050] S400: Using the beef cattle re-identification model GCN-ReID to perform multi-view identification of beef cattle individuals in the test area to obtain cattle identity information;

[0051] S500: Based on the cattle identity information, the estrus behavior detection and early warning model CowEstrusNet based on Yolov8 is used to extract the behavioral characteristics of beef cattle, identify estrus behavior, and output the monitoring results in real time to provide timely early warning.

[0052] In a specific embodiment, S100 includes:

[0053] According to the area of ​​the farm, multiple surveillance cameras are arranged in the site to ensure that there are no blind spots in the monitoring range. Focus on covering key areas where cattle move, such as feeding areas, water trough areas, rest areas and main passages. Reasonable overlapping areas are designed between the fields of view of each camera so that cattle can be re-identified across camera perspectives through the ReID algorithm. The installation height of the camera is uniformly set to 3 meters, which can not only overlook the entire herd and capture the full body posture of beef cattle, but also effectively avoid blind spots in the field of view. All surveillance cameras are synchronized through NTP (Network Time Protocol) to ensure that the surveillance data from different cameras are aligned on the same timeline. Video recording uses a frame rate of 30 frames per second to ensure continuous capture of cattle behavior.

[0054] Specifically, the facial and body feature data of cattle are collected manually, requiring the collection of the cattle's body features, face and ear tag numbers at the same time. The ear tag number is used as the unique data label for the cattle's facial and body features. Each camera needs to uniformly record information such as the number and shooting angle to provide support for subsequent scene data fusion and synchronization.

[0055] In a specific embodiment, S200 includes:

[0056] S210: Data denoising and enhancement. The adaptive histogram equalization (CLAHE) method is used to deal with uneven lighting problems, especially in the case of changing lighting in the morning and evening or on cloudy days, to ensure stable video quality. Image rotation, scaling, mirror flipping and other methods are used to perform data enhancement operations on the perspective of each cow, etc., to enhance the model's recognition ability in multi-camera scenarios.

[0057] S220: Background separation and ROI extraction. Use the Gaussian mixture model (GMM) to build a pasture background model, identify and remove static backgrounds, and focus on extracting dynamic cattle objects; divide the video frames into regions, extract the active areas containing cattle as the regions of interest, and ignore irrelevant areas to save storage and computing resources.

[0058] S230: Data annotation and label generation. Data annotation mainly includes the boundary of the cow and the cow face area. The Yolov8 model is used to automatically generate the boundary boxes of the cow and the cow face in the image to reduce the workload of manual annotation. The data labels of the cow and the cow face use the cow ear number as the unique identifier to provide training data for the subsequent cow individual recognition model. In addition, the key parts of the cow's body (including the nose, body and tail root) are marked to provide training data for cow posture estimation and body part detection.

[0059] See also Figure 2 In a specific embodiment, the present invention provides a beef cattle re-identification model GCN-ReID, which includes four key modules: a posture estimation learning module, a Transformer learning module, a convolutional learning module, and a graph convolutional module GCM. The posture estimation learning module is used to estimate the posture information and joint position of the cattle and extract key point features; the Transformer learning module is used to establish a global dependency relationship between different local features to enhance the robustness of the model to local occlusion and partial deformation; the convolutional learning module extracts fine-grained local features of cattle based on the ResNet network, such as fur texture, color and other subtle marks; the graph convolutional module (GCM) fuses local features, global features and body feature information (body feature information refers to the key point features of cattle extracted by the posture estimation learning module, including cattle posture information and joint position information.), and the fused features can be more effectively used for individual cattle identification, improving the recognition accuracy and robustness of the model.

[0060] In a specific embodiment, the posture estimation learning module can capture the unique movement patterns, body postures and contours of cattle, which are relatively stable in a short period of time and are suitable as auxiliary recognition features. The cattle posture information provides data on the positions of bones and joints, and provides fine-grained features for cattle with different postures and partial occlusions, thereby improving the performance of re-identification and effectively coping with various complex scenarios and challenges. Therefore, the present invention uses a cattle posture detection network to obtain the positions of key points and the connection relationships between bones to better support individual recognition tasks.

[0061] Specifically, the input image is first fed into the VGG-16 network to extract its deep feature representation and predict the confidence map and affinity field. This process can be represented by the following mathematical equation, where W, H, and 3 represent the width, height, and number of channels of the input image, respectively.

[0062] Specifically, the input image It is input into VGG-16 to extract deep feature representation The confidence map and affinity field are predicted simultaneously. The process is represented by the following mathematical equation, where W, H, and 3 represent the width, height, and channels of the input image, respectively.

[0063] F pe =VGG16(I)

[0064] In the pose estimation learning part, and They represent affinity field and confidence map respectively, where K represents the key point in the feature map. The confidence map provides the probability information of the existence of key points at each position, which helps to determine the most likely position of the key point, while the affinity field captures the relationship between different key points, indicating the degree of association between key point pairs (such as key point pairs connecting bones), and helps to build a complete skeleton structure. Therefore, when predicting the confidence map, combining the information of the affinity field can improve the accuracy of key point positioning, and the two work together to improve the overall posture estimation performance.

[0065] S=ρ(F pe ,L),

[0066] L=φ(F pe ,S),

[0067] Among them, ρ(F pe ,L) and φ(F pe , S) is a fully connected layer, receiving F pe , L, and S as input. Compared with the original OpenPose, the backbone network that generates L and S is weight-shared, thereby reducing the number of parameters by half. In addition, in order to improve computational efficiency and speed, the multi-stage iteration steps in OpenPose are removed. For complex pose changes and occlusion situations, these problems are solved by the subsequent global context module (GCM).

[0068] In order to obtain rich pose information, OpenPose is pre-trained on the constructed dataset, and the model can extract K = 10 key point heat maps and N = 9 part affinity fields from the input cow image. Each generated key point heat map S and the corresponding confidence score L contain the joint information of the cow, thus supporting more accurate pose estimation and subsequent recognition tasks.

[0069] In a specific embodiment, the convolutional learning module is based on a convolutional neural network (CNN). Each convolution kernel (filter) operates only in a local area and gradually traverses the entire image through a sliding window, so that CNN can effectively capture local features (such as edges, corners, and textures). In the task of cattle re-identification, local features are crucial for identifying and distinguishing different individuals, especially in complex backgrounds. Paying attention to local features can significantly enhance the robustness of the model. Therefore, the present invention uses ResNet-50 pre-trained on ImageNet as the backbone network.

[0070] In order to adapt to the task of cattle feature extraction, the final average pooling layer and fully connected layer of the original ResNet-50 were removed, and the stride of the convolutional layer in the first residual block of the fourth stage was set to 1, making it more suitable for fine-grained extraction of cattle features and improving the accuracy of individual recognition.

[0071] F Res =ResNet50(I),

[0072] in, Represents the output of ResNet-50.

[0073] In a specific embodiment, the Transformer learning module specifically includes: Due to partial occlusion of the cattle in the image, key parts are often missing, which makes accurate recognition difficult. However, the Transformer architecture provides a new way to solve this problem because of its ability to effectively capture global information and integrate scattered features. Therefore, the present invention proposes a Transformer-based feature extraction method that can more effectively understand the overall structure of cattle images and maintain efficient recognition performance.

[0074] First, the input cow image I is divided into N fixed-size p×p image patches. Then, a learnable projection matrix Each image block is linearly transformed to map the original pixel information into a feature space with a fixed dimension. This process not only helps to preserve the relative position relationship between image blocks, but also promotes the Transformer model to integrate information across blocks, thereby enhancing the ability to capture local and overall features of cattle.

[0075]

[0076] Where D represents the dimension of the feature vector. The feature vectors of all image patches are arranged in order and appended with a learnable classification token (denoted as f cls ) to integrate the features of all image blocks and generate a global feature representation. Add learnable position encoding information E to the feature vector of each image blockposi , to preserve the position information in the original image. Therefore, the final input feature sequence is For use by Transformer.

[0077]

[0078] Where N represents the number of image blocks, represents the kth image block. E represents an embedding matrix, which is used to map the input feature vector to the feature space. The input feature sequence is normalized by the layer to obtain the normalized feature sequence X N , and respectively with three weight matrices W Q ,W K ,and W V Multiply them together to get the query vector Q, key vector K and value vector V.

[0079] X N =LayerNorm(X f )

[0080] By utilizing the multi-head self-attention mechanism of the Transformer encoder, features from different image patches are weighted and fused to construct a global image feature representation. Subsequently, these fused features can be further input into the graph convolution module (GCM) to extract more discriminative features.

[0081] In a specific embodiment, in order to effectively integrate features from different branches, namely affinity fields and confidence maps in pose estimation, and feature maps extracted by ResNet and Transformer, the present invention provides a graph convolution module GCM to combine these features to generate a comprehensive feature representation for cattle re-identification.

[0082] First, for the output of pose estimation, the affinity field L and the confidence map S are used to construct the node and edge features of the graph. Among them, the node features are initialized using the confidence map S, each node represents a key point, and the node features represent the confidence score of the key point. For the feature maps provided by ResNet and Transformer, they are converted into node features adapted to graph neural network processing through a series of convolution operations. In this process, this embodiment not only initializes the node features, but also defines the edge features so that the graph convolution layer can effectively update the node features by considering the edge features.

[0083] Subsequently, the aggregated node features obtained after processing by the graph convolutional layer will be further fused with the features of the aforementioned branches to form the final fused features. The final fused features are input into the classifier for cattle re-identification to achieve accurate cattle identification.

[0084] In summary, the proposed graph convolution module effectively integrates features from different sources and fully utilizes the advantages of each feature to extract more discernible cattle features. This method not only improves the performance of the model in the cattle re-identification task, but also significantly enhances its robustness in various complex environments.

[0085] In a specific embodiment, in order to enhance the performance of the beef cattle re-identification model GCN-ReID, the pose estimation module, ResNet and Transformer learning modules are trained separately, and then the modules are combined. Each module uses a loss function customized for its specific task to achieve optimal learning effect and feature fusion.

[0086] In the pose estimation module, OpenPose generates an affinity field L and a confidence map S. In order to optimize the prediction of L and S, the pose estimation network adopts the L2 loss function, which is expressed as:

[0087]

[0088] Where L i and S i denote the predicted affinity field and confidence map respectively, and is the corresponding true value, and N is the number of key points. The L2 loss function ensures the accurate positioning and confidence score of key points by minimizing the square difference between the predicted value and the true value, thereby improving the accuracy of pose estimation.

[0089] ResNet is responsible for extracting deep features from images, using contrastive loss to encourage similar features to be close in feature space, while dissimilar features remain far away. The formula for the contrastive loss function is:

[0090]

[0091] Among them, f(x i )and are the feature vectors of the positive sample pair, f(x i )and are the feature vectors of negative sample pairs, y i For labels (if x i and is a positive sample pair, then y i =1, otherwise y i =0), d(·,·) is the Euclidean distance between feature vectors, m is the minimum distance threshold between negative sample pairs, and N is the number of sample pairs. Contrastive loss encourages the model to learn more discriminative features by reducing the distance between positive sample pairs and increasing the distance between negative sample pairs.

[0092] Transformer captures global context information through the self-attention mechanism and uses triplet loss for training. Its loss function is:

[0093]

[0094] Where: f(a i )、f(p i ) and f(n i ) represent the feature vectors of anchor samples, positive samples, and negative samples, respectively, d(·,·) is the Euclidean distance, α is the minimum interval, and N is the number of triplets. By minimizing the distance between anchor samples and positive samples and maximizing the distance between anchor samples and negative samples, the model can better distinguish different categories.

[0095] Finally, the trained features of each branch will be connected or fused into a unified representation:

[0096] F Combined =F pose +F trans +F res

[0097] Among them, F pose From the pose estimation feature, F res From ResNet features, F trans From Transformer features.

[0098] By optimizing each module separately to learn local features, contextual understanding, and global representation, the model is able to extract discriminative features more comprehensively. This learning method improves the accuracy and robustness of cattle re-identification tasks and achieves efficient recognition under multiple viewpoints.

[0099] In a specific embodiment, two cattle re-identification evaluation indicators, Rank-k and mAP, are used to measure the performance of the proposed beef cattle re-identification model GCN-ReID, specifically including:

[0100] Rank-k: This indicator indicates the probability of the target object appearing in the first k positions of the retrieval results. Rank-1, Rank-5 and Rank-10 are selected as the main evaluation indicators, among which Rank-1 is an important reference indicator. A higher Rank-1 value indicates that the probability of the target object being correctly identified in the first place is higher, which means that the model performance is better.

[0101] mAP: In order to more comprehensively evaluate the overall performance of the model, mAP is used as an evaluation criterion. mAP considers all image positions in the retrieval results that have the same ID as the query target and calculates their average precision. A higher mAP value indicates that the correct retrieval target ranks higher in the results, reflecting the average precision and performance of the model in different scenarios. For the GCN-ReID model, the higher the mAP value, the better the average accuracy and performance of the model.

[0102] In a specific embodiment, after completing the above individual weight identification of beef cattle, based on the cattle identity information, the estrus behavior detection and early warning model CowEstrusNet based on Yolov8 is used to extract the behavioral characteristics of beef cattle, identify estrus behavior, and output the monitoring results in real time to provide timely early warning.

[0103] Specifically, here the cattle are detected based on the cattle identity information extracted by the above module. Since the cattle identity information extraction model has successfully completed the identity recognition task, all beef cattle in each frame of the video have clear identity information. With the help of the Yolov8 model, the specific position of the cattle in the image can be accurately located. While obtaining its identity information, the regional position of the cattle in the picture can also be accurately located, thereby improving the accuracy of the detection of cattle body parts.

[0104] Specifically, Figure 3 As shown in the figure, the estrus behavior detection and early warning model CowEstrusNet based on Yolov8 mainly includes four important modules: Yolov8 detection module, body part detection module, estrus behavior detection module and behavior analysis module. Yolov8 is used to detect beef cattle in the video, and the detection results are input into the body part detection module. The role of the body part detection module is to accurately locate the key body parts of beef cattle (such as nose, body and tail root), extract the features related to estrus behavior by detecting the joint movement of multiple parts, and then provide basic data for subsequent behavior analysis. The role of the estrus behavior detection module is to accurately identify the body interaction related to estrus behavior by combining the movement characteristics and coordinate characteristics of cattle (the individual feature detection in the estrus behavior detection module includes movement characteristics and coordinate characteristics.), calculate the relative positions between multiple body parts based on the position of the nearest neighbor cattle, effectively capture the signs of estrus such as sniffing and climbing, and provide a basis for further estrus behavior analysis. The function of the behavior analysis module is to identify the estrus interval (peak) and non-estrus interval (flat part) by drawing the cumulative timeline of estrus events for each cow, and use the confusion matrix to analyze the detection accuracy of each cow, identify and sort the estrus of cattle according to the estrus frequency, and provide decision support for herd management and reproduction.

[0105] In a specific embodiment, considering that beef cattle often use their bodies to communicate with each other during estrus, it is crucial to accurately locate body parts for the analysis of estrus behavior. The body part detection module is responsible for detecting the key body parts of the identified cattle, including the nose, body and coccyx (base of the tail), which can be used to monitor the movement trajectory of individual cattle. In addition, cattle in estrus often interact through these body parts, such as climbing, sniffing, butting heads, and placing their chins on the buttocks of other cattle. Relying on a single part alone may not be enough to identify the visual signals of estrus, and the combined movement of multiple parts can more effectively analyze and identify estrus behavior. Therefore, this embodiment extracts discriminative features related to estrus behavior by detecting the combined movement of these parts. After the body parts are detected, the data is passed to the estrus behavior detection module to extract a set of discriminative features that can indicate estrus behavior.

[0106] To achieve accurate body part localization, a ResNet-50-based key point estimator (KPE) was designed based on a convolutional neural network to predict the coordinates (x, y) of each body part through a regression task. The model receives an image containing a cow as input and outputs six continuous variables, representing the position coordinates of key parts such as the nose, body, and tail root.

[0107] The output coordinates of the model are located in the cropped cow area (xcropped, ycropped), which can be converted to the coordinates in the original image (xoriginal, yoriginal) by a simple translation operation. The conversion formula is:

[0108] (xoriginal,yoriginal)=(xcropped+xoffset,ycropped+yoffset)

[0109] Among them, (xoffset, yoffset) represents the offset of the cropped area relative to the original image. In this way, the key point estimator can accurately detect the various body parts of the cow and convert the detection results from the cropped area back to the original image coordinates, providing accurate coordinate information for subsequent behavioral analysis.

[0110] In a specific embodiment, the estrus behavior detection module includes:

[0111] (1) Individual feature detection

[0112] Individual characteristics of cattle can be further divided into motion-based features and coordinate-based features. Among them, motion-based features are particularly important for detecting estrus behavior. When one or more body parts of a cow (especially the body and tail root) are blocked by another cow, these parts may not be detected, thereby triggering the following behaviors related to the current cow: 1) climbing behavior, another cow rides on the current cow; 2) sniffing event, another cow sniffs the current cow; 3) resting event, another cow puts its chin on the hips or head of the current cow. In view of the advantages and disadvantages of these two types of features, this embodiment combines these two types of features to improve detection accuracy. In each frame, this embodiment calculates the distance, speed and acceleration of each body part. The distance is calculated by the difference in the coordinates of the body parts of the current frame and the previous frame. The specific algorithm is shown in Table 1, and the Euclidean distance is used for calculation.

[0113] Table 1 is calculated using Euclidean distance

[0114]

[0115] (2) Nearest neighbor distance feature detection

[0116] The nearest neighbor feature is not only based on the body part of the current cow, but also refers to the position of the body part of the nearest neighbor cow. This feature is used to detect sniffing behavior. If the body parts of two cows are close enough, the current cow is likely to be sniffing the other cow, or being sniffed by the other cow. There are two main events: the first is that the current cow is sniffing the other cow, which usually occurs when the current cow's nose is close to the body or tail of the other cow; the second is that the other cow is sniffing the current cow, which usually occurs when the tail or body of the current cow is close to the nose of the other cow. Algorithm 2 describes in detail the calculation method of the nearest neighbor distance feature (n1-n9), such as n2 refers to the shortest distance between the current cow's nose and the tail of the other cow. If some body parts are not detected in a certain frame, the feature value will be set to -1. Through these nearest neighbor-based features, the interactive behaviors between cows, especially sniffing behaviors, can be captured and analyzed more accurately, thereby improving the accuracy of identifying estrus behaviors.

[0117] In a specific embodiment, the estrus detection results in the behavior analysis module are compared with the actual labels by drawing a cumulative timeline graph of consecutive estrus events for each cow. In this embodiment, the peak in the graph is regarded as the estrus interval, and the flat part is regarded as the non-estrus interval. The detection performance of each cow is evaluated through the confusion matrix, and the average performance of all cows is calculated at the same time. After the model detects the estrus interval of each cow, the cows are sorted from high to low based on the estrus frequency. According to the sorting result, it is determined which cow should be given priority for artificial insemination. The more estrus signs detected, the greater the possibility that the cow is in estrus.

[0118] On the other hand, see Figure 4 The embodiment of the present invention discloses a multi-view individual estrus behavior monitoring system for beef cattle based on ReID and Yolov8, comprising:

[0119] Data acquisition module: obtains activity videos of beef cattle in different scenarios and annotates the behavioral data in the activity videos;

[0120] Data preprocessing module: preprocess the labeled behavior data to obtain the training data set;

[0121] Model training module: train the beef cattle re-identification model GCN-ReID based on the training dataset;

[0122] Cattle identification module: The beef cattle re-identification model GCN-ReID is used to perform multi-view identification of individual beef cattle in the test area to obtain cattle identity information;

[0123] Estrus behavior recognition module: Based on the cattle identity information, through the estrus behavior detection and early warning model CowEstrusNet based on Yolov8, the behavioral characteristics of beef cattle are extracted, estrus behavior is identified, and the monitoring results are output in real time to provide timely warnings.

[0124] On the other hand, an embodiment of the present invention discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a multi-perspective beef cattle individual estrus behavior monitoring method based on ReID and Yolov8 is implemented.

[0125] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0126] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-view individual estrus behavior monitoring method for beef cattle based on ReID and Yolov8, characterized in that: The following steps are involved: S100: Obtain activity videos of beef cattle in different scenarios, and annotate behavior data in the activity videos; S200: preprocessing the labeled behavior data to obtain a training data set; S300: Training the beef cattle re-identification model GCN-ReID based on the training dataset; S400: Using the beef cattle re-identification model GCN-ReID to perform multi-view identification of beef cattle individuals in the test area to obtain cattle identity information; S5 00: Based on the cattle identity information, the estrus behavior detection and early warning model CowEstrusNet based on Yolov8 is used to extract the behavioral characteristics of beef cattle, identify estrus behavior, and output the monitoring results in real time to provide timely early warning.

2. According to claim 1, a multi-view beef cattle individual estrus behavior monitoring method based on ReID and Yolov8 is characterized in that: The S200 includes: S210: Using adaptive histogram equalization CLAHE to handle uneven illumination in behavioral data; S220: Use the Gaussian mixture model (GMM) to build a pasture background model to identify and remove static background in the behavior data; S230: Use the Yolov8 model to automatically generate cow and cow face bounding boxes for images in the behavior data, wherein the data labels of the cow and cow face bounding boxes use the cow ear tag number as a unique identifier; at the same time, the key parts of the cow's body are marked to obtain a training data set.

3. The multi-view beef cattle individual estrus behavior monitoring method based on ReID and Yolov8 according to claim 1, characterized in that: The beef cattle re-identification model GCN-ReID includes: a posture estimation learning module, a Transformer learning module, a convolution learning module and a graph convolution module GCM; The posture estimation learning module is used to estimate the posture information and joint positions of cattle and extract key point features of cattle; The convolutional learning module extracts fine-grained local features of cattle based on the ResNet network; The Transformer learning module is used to establish global dependencies between different local features to obtain global features; The graph convolution module GCM fuses local features, global features and key point features of cattle to obtain cattle identity information.

4. The multi-view beef cattle individual estrus behavior monitoring method based on ReID and Yolov8 according to claim 1, characterized in that: The estrus behavior detection and early warning model CowEstrusNet based on Yolov8 includes: a Yolov8 detection module, a body part detection module, an estrus behavior detection module and a behavior analysis module; Wherein, based on the identity information of the cattle, the specific position of the cattle in the image in the video is determined using the Yolov8 detection module, and the specific position information is input into the body part detection module; The body part detection module is used to locate key body parts of beef cattle and extract features related to estrus behavior by detecting the joint movement of multiple parts; The estrus behavior detection module extracts the movement characteristics and coordinate characteristics of the cattle, identifies the body interactions related to the estrus behavior, and calculates the relative positions between multiple body parts based on the positions of the nearest neighbor cattle to capture the signs of estrus; The behavior analysis module is used to draw the cumulative timeline of estrus events for each cow, identify estrus intervals and non-estrus intervals, and use confusion matrices to analyze the detection accuracy of each cow, and perform estrus discrimination and sorting of cows according to estrus frequency.

5. A multi-view individual estrus behavior monitoring system for beef cattle based on ReID and Yolov8 using the multi-view individual estrus behavior monitoring method for beef cattle based on ReID and Yolov8 according to any one of claims 1 to 4, characterized in that: include: Data acquisition module: obtains activity videos of beef cattle in different scenarios and annotates the behavioral data in the activity videos; Data preprocessing module: preprocess the labeled behavior data to obtain the training data set; Model training module: train the beef cattle re-identification model GCN-ReID based on the training dataset; Cattle identification module: The beef cattle re-identification model GCN-ReID is used to perform multi-view identification of individual beef cattle in the test area to obtain cattle identity information; Estrus behavior recognition module: Based on the cattle identity information, through the estrus behavior detection and early warning model CowEstrusNet based on Yolov8, the behavioral characteristics of beef cattle are extracted, estrus behavior is identified, and the monitoring results are output in real time to provide timely warnings.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the multi-perspective beef cattle individual estrus behavior monitoring method based on ReID and Yolov8 is implemented as described in any one of claims 1 to 4.

Images