A sow estrus monitoring method and system

Abstract

The present invention discloses a sow estrus monitoring method and system, and relates to the field of sow estrus monitoring. The method comprises the following steps: collecting sow color video data and sow infrared video data; obtaining whole pig color data segments, pig head color key segments, pig buttocks color key segments and pig buttocks infrared key segments; obtaining sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results and pig vulva-buttocks temperature difference reaching estrus state feature recognition results; obtaining sow estrus prediction results; obtaining sow actual estrus results; constructing a training data set, and using the training data set to fine-tune the weight parameters in the multi-value weighted prediction to predict and monitor the sow estrus. The present invention predicts sow estrus based on global features and local features, which can improve the accuracy of the prediction results of sow estrus behavior, and thus effectively monitor sow estrus.

Description

Technical Field

[0001] The present invention relates to the field of sow estrus monitoring, and in particular to a sow estrus monitoring method and system. Background Art

[0002] Estrus detection plays a key role in sow reproduction management and is essential for determining the best time for insemination and improving sow pregnancy rates. With the development of my country's pig farming industry, the transition from the traditional free-range model to the large-scale and intensive farming model, there are still many farms that rely on traditional manual monitoring methods to determine whether sows are in estrus, including observing mounting behavior, vulva swelling, and back pressure testing. However, this method has the problems of high labor intensity and low accuracy, especially in large-scale farms, which can easily lead to missed detection and inaccurate judgment of estrus time, thus missing the best breeding time for sows, adversely affecting reproductive performance, and directly affecting the economic benefits of pig farms.

[0003] The existing sow estrus monitoring systems are mainly divided into manual monitoring and intelligent monitoring methods. The traditional manual monitoring method is mainly to manually check the estrus of sows regularly, but manual estrus is time-consuming and labor-intensive, and it is difficult to detect sow estrus in time. The intelligent monitoring method includes two means. One is to monitor through contact sensors such as back clips, and use machine learning methods to process the individual information of sows collected by contact sensors to detect whether the sows are in estrus. However, the installation of contact sensors can easily cause stress reactions in sows, and it is also very easy to be damaged and fall off and unresponsive. The other is to monitor through non-contact sensors such as cameras and ultrasonic detectors, combined with computer vision technology to detect sow estrus in the captured video. Many scholars have used the latter method to conduct research. Liu Tonghai et al. used infrared thermal imaging and other technologies to extract the temperature of the sow's vulva to detect the estrus of the sow; Zhuchuan Shen et al. determined whether the sow was in estrus by measuring the frequency of the sow's standing up and lying down; Ju Tiezhu et al. determined whether the sow was in estrus by detecting the duration of the sow's standing posture; but the estrus characteristics of sows used in these methods are too single, which is easy to cause false detection and missed detection. Xue Sujin et al. performed weighted calculations on the obtained sow arch back sequence, vulva temperature sequence and sow sound intensity sequence, but the weight parameters of each feature used in this method are artificially predetermined, and the source of the parameters is unexplainable, and it is impossible to perform dynamic calculations based on the predicted results of the program.

[0004] In summary, the prior art has the following problems:

[0005] (1) The estrus characteristics of sows used in the prior art are too simple, which can easily lead to missed detection or misjudgment, thus missing the best breeding time for sows and affecting breeding production;

[0006] (2) In the prior art, the estrus of sows is predicted by feature weighting, wherein the weight parameters of each feature are manually set and cannot be dynamically adjusted according to the actual prediction situation;

[0007] (3) The sow estrus determination model used in the prior art has low transparency, which makes it difficult for pig farm managers to fully trust similar sow estrus determination systems;

[0008] (4) The existing technology cannot effectively utilize the difference between the actual results of sow estrus and the predicted results of the program to train the sow estrus determination model. Summary of the invention

[0009] In view of the above-mentioned deficiencies in the prior art, the present invention provides a sow estrus monitoring method and system, which can improve the accuracy of the sow estrus behavior prediction results based on the global features and local features, and thus effectively monitor the sow estrus.

[0010] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0011] A sow estrus monitoring method comprises the following steps:

[0012] S1, collecting sow color video data and sow infrared video data;

[0013] S2, performing individual segmentation and key frame extraction on the sow color video data and sow infrared video data in step S1, and obtaining whole pig color data segments, pig head color key segments, pig butt color key segments and pig butt infrared key segments;

[0014] S3, based on the whole pig color data segment, pig head color key segment, pig buttocks color key segment and pig buttocks infrared key segment in step S2, obtain the sow behavior feature recognition result, pig face dull expression feature recognition result, pig ear erection feature recognition result, pig vulva mucus flow feature recognition result and pig vulva buttocks temperature difference reaching estrus feature recognition result;

[0015] S4, performing single-value prediction or multi-value weighted prediction on the sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results, and pig vulva-buttock temperature difference reaching estrus state feature recognition results in step S3 to obtain the sow estrus segment prediction results, and obtain the sow estrus prediction results based on the sow estrus segment prediction results;

[0016] S5, based on the sow estrus prediction result in step S4, artificial estrus checking is performed on the sow to obtain the actual estrus result of the sow;

[0017] S6. Use the sow estrus prediction result in step S4 and the sow actual estrus result in step S5 to construct a training data set, use the training data set to fine-tune the weight parameters in the multi-value weighted prediction, and use the multi-value weighted prediction after the weight parameters are fine-tuned to predict and monitor the sow estrus.

[0018] Further, step S3 includes the following steps:

[0019] S31, based on the whole pig color data fragment in step S3, using the Two-Stream model to obtain the sow behavior feature recognition result;

[0020] S32, based on the pig head color key segment in step S3, using the classic multi-task cascade convolutional neural network, residual convolutional network, multi-attention convolutional network and fully connected layer connected in sequence to obtain the pig face dull expression feature recognition result;

[0021] S33, based on the pig head color key segment in step S3, using the resnet-50 network and SVM classifier connected in sequence to obtain the pig ear erect feature recognition result;

[0022] S34, based on the pig buttocks color key segment in step S3, using the yolov8 model, super-resolution model, resnet-50 network and SVM classifier connected in sequence to obtain the pig vulvar mucus feature recognition result;

[0023] S35. Based on the pig buttocks color key segment and the pig buttocks infrared key segment in step S3, the yolov8 model is used to obtain the feature recognition result of the pig's vulva and buttocks temperature difference reaching the estrus state.

[0024] Furthermore, in step S31, a score attention mechanism is added to the Two-Stream model, and the climbing behavior score, mounting behavior score, being mounted behavior score, standing still behavior score and other behavior scores in the sow's behavior characteristics are assigned weight parameters of 3:3:3:2:1 respectively.

[0025] Further, step S35 includes the following steps:

[0026] S351, based on the pig buttocks color key segment in step S3, using the yolov8 model to obtain the pig vulva position and the pig buttocks position in the color image;

[0027] S352, based on the infrared key segment of the pig buttocks in step S3, mapping the pig vulva position and the pig buttocks position in the color image in step S351 to the infrared temperature matrix, and obtaining the pig vulva temperature and the pig buttocks temperature from the infrared temperature matrix;

[0028] S353, according to the pig vulva temperature and pig buttocks temperature in step S352, calculate the probability that the pig vulva and buttocks temperature difference reaches the estrus state, and obtain the characteristic recognition result of the pig vulva and buttocks temperature difference reaching the estrus state, expressed as:

[0029]

[0030] in: is the probability that the temperature difference between the genitals and buttocks of the i-th sow in the k-th frame of the j-th data segment reaches the estrus state, min is the minimum symbol, max is the maximum symbol, T V is the temperature of the pig vulva, T G is the pig buttocks temperature, θ is the minimum threshold of the pig genital-buttocks temperature difference, λ is the confidence of the temperature information, and m is the maximum threshold of the pig genital-buttocks temperature difference.

[0031] Further, step S4 includes the following steps:

[0032] S41, normalizing the sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results, and pig vulva-buttocks temperature difference reaching estrus state feature recognition results in step S3;

[0033] S42, using the sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results and pig vulva buttocks temperature difference reaching estrus state feature recognition results after normalization processing in step S41, calculate the entropy value between the feature recognition results, expressed as:

[0034]

[0035] Where: H ij (x) is the entropy value between the feature recognition results of the i-th sow in the j-th data segment, q is the traversal sequence number, p(x q ) is the qth normalized feature recognition result;

[0036] S43, judging whether the entropy value between the feature recognition results in step S42 is greater than the multi-value weighted prediction threshold; if so, adopting multi-value weighted prediction to obtain the sow estrus prediction result, otherwise adopting single-value prediction to obtain the sow estrus prediction result;

[0037] S44, obtaining the sow estrus prediction result according to the sow estrus segment prediction result in step S43.

[0038] Further, in step S43, multi-value weighted prediction is used to obtain the prediction result of the sow estrus segment, specifically:

[0039] Based on the improved Kernel Shap algorithm, the sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results and pig vulva-buttock temperature difference reaching estrus state feature recognition results in step S3 are weighted by credibility to obtain the sow estrus segment prediction result, which is expressed as:

[0040]

[0041] Where: P ij is the prediction result of the estrus segment of the i-th sow in the j-th data segment, σ(·) is the first activation function, A is the weight parameter of the sow behavior prediction probability, is the sow behavior feature recognition result of the i-th sow in the j-th data segment, B is the weight parameter for the prediction probability of the pig face dull expression, is the pig face dull expression feature recognition result of the i-th sow in the j-th data segment, C is the weight parameter of the prediction probability of the pig ear erection, is the recognition result of the erect ear feature of the i-th sow in the j-th data segment, D is the weight parameter for the prediction probability of mucus flow from the pig vulva, is the recognition result of the mucus characteristics of the pig vulva flow of the i-th sow in the j-th data segment, E is the weight parameter of the predicted probability that the temperature difference between the pig vulva and buttocks reaches the estrus state, It is the feature recognition result of the i-th sow’s vulva-buttock temperature difference reaching the estrus state in the j-th data segment.

[0042] Further, in step S6, the weight parameters in the multi-value weighted prediction are fine-tuned using the training data set, including the following steps:

[0043] A1. Use the training data set to construct a prediction probability data set, and divide the prediction probability data set into a prediction probability training set and a prediction probability test set;

[0044] A2. Obtain the contribution of global features and local features based on the prediction probability training set in step A1 and the improved Kernel Shap algorithm;

[0045] A3. Based on the contribution of global features and local features in step A2, the updated prediction weight of the sub-model is obtained using the moving average method, which is expressed as:

[0046]

[0047] Where: Coef′ is the prediction weight of the sub-model after update, softmax(·) is the second activation function, Coef is the prediction weight of the sub-model, Coef=(A,B,C,D,E), A is the weight parameter of the prediction probability of sow behavior, B is the weight parameter of the prediction probability of dull expression on the pig's face, C is the weight parameter of the prediction probability of erect ears, D is the weight parameter of the prediction probability of mucus flow from the pig's vulva, E is the weight parameter of the prediction probability of the temperature difference between the pig's vulva and buttocks reaching the estrus state, β is the update weight, The contribution of global features and local features;

[0048] A4. Calculate the loss ratio of the sub-model based on the predicted probability test set in step A1 and the predicted weight updated in step A3;

[0049] A5. Update the sub-model loss using the loss proportion of the sub-model in step A4 to fine-tune the weight parameters of the system model, expressed as:

[0050]

[0051] Where: θ r is the updated weight parameter of the rth sub-model, argmin is the variable value that minimizes the objective function, is the loss ratio of the rth sub-model, is the original prediction loss of the rth sub-model.

[0052] Further, step A2 includes the following steps:

[0053] A21, based on the prediction probability training set in step A1, construct a simplified feature set of prediction probability training data;

[0054] A22. Construct a linear prediction function of sow estrus based on the improved Kernel Shap algorithm, and construct a mapping function of the prediction probability training data and an explanation function of sow estrus;

[0055] A23. According to the simplified feature group of the prediction probability training data in step A21, the linear prediction function of sow estrus in step A22, the mapping function of the prediction probability training data and the interpretation function of sow estrus, the contribution of the global features and the local features is obtained, which is expressed as:

[0056]

[0057] in: is the contribution of global features and local features, is the contribution of the predicted probability of sow behavior, is the contribution of the prediction probability of the pig face dull expression, is the contribution of the prediction probability of pig ears erect, is the contribution of the predicted probability of pig vulvar mucus discharge, is the contribution of the predicted probability of the pig's vulva-hip temperature difference reaching the estrus state, argmin is the variable value that minimizes the objective function, is the symbol for taking the average value, X j is the prediction result of sow estrus in the prediction probability dataset, X train is the predicted probability training set, z jo To simplify feature group Z j The oth simplified feature in Z j is the simplified feature of the predicted probability data of the jth video clip, F(·) is the linear prediction function of sow estrus, is the mapping function for predicting the probability training data, G(·) is the explanation function for sow estrus, Kernel weight function for training data for prediction probabilities.

[0058] Further, step A4 includes the following steps:

[0059] A41. Based on the predicted probability test set in step A1 and the predicted weight updated in step A3, the predicted probability of the predicted probability test set is calculated, which is expressed as:

[0060]

[0061] Where: P test is the predicted probability set of the predicted probability test set, σ(·) is the first activation function, Coef′ is the predicted weight after the sub-model is updated, is the prediction result X of sow estrus in the prediction probability test set i The transpose of is the predicted probability test set X test The transpose of

[0062] A42. Based on the predicted probability of the predicted probability test set in step A41, the binary classification loss of the sow estrus behavior is calculated, which is expressed as:

[0063]

[0064] in: is the binary loss of sow estrus behavior, is the symbol for taking the average value, p is the predicted probability of estrus of the sample data in the predicted probability test set, y is the actual estrus result of the sample data in the predicted probability test set, and Y test is the label corresponding to the predicted probability test set;

[0065] A43. According to the binary loss of the sow estrus behavior in step A42, the loss ratio of the sub-model is calculated, which is expressed as:

[0066]

[0067] in: is the loss ratio of the rth sub-model, Coef′ r is the updated prediction weight of the rth sub-model, Coef′ j is the updated prediction weight of the j-th sub-model.

[0068] A sow estrus monitoring system using the above method is characterized by comprising:

[0069] Data acquisition module, used to collect sow color video data and sow infrared video data;

[0070] The data preprocessing module is used to perform individual segmentation and key frame extraction on the sow color video data and the sow infrared video data, and obtain the whole pig color data segment, the pig head color key segment, the pig butt color key segment and the pig butt infrared key segment;

[0071] A data analysis module is used to obtain the recognition results of sow behavior characteristics, pig face dull expression characteristics, pig ear erection characteristics, pig vulva mucus flow characteristics, and pig vulva and buttocks temperature difference reaching estrus state based on the whole pig color data segment, pig head color key segment, pig buttocks color key segment, and pig buttocks infrared key segment;

[0072] A prediction module is used to perform single-value prediction or multi-value weighted prediction on the recognition results of sow behavior characteristics, pig face dull expression characteristics, pig ear erection characteristics, pig vulva mucus flow characteristics, and pig vulva-buttock temperature difference reaching estrus state characteristics, to obtain the prediction results of sow estrus segments, and obtain the sow estrus prediction results based on the sow estrus segment prediction results;

[0073] A feedback module is used to perform manual estrus check on the sows according to the estrus prediction results of the sows to obtain the actual estrus results of the sows;

[0074] The updating module is used to construct a training data set based on the sow estrus prediction results and the sow's actual estrus results, and use the training data set to fine-tune the weight parameters in the multi-value weighted prediction.

[0075] The present invention has the following beneficial effects:

[0076] (1) The present invention predicts sow estrus based on global features and local features, wherein the global features are sow behavior features, and the local features are sow appearance features and sow temperature features. The sow appearance features specifically include dull expression features of pig faces, erect ears, and mucus flow from pig vulva. The sow temperature features specifically include the feature that the temperature difference between the pig vulva and buttocks reaches the estrus state, thereby improving the accuracy of the prediction results of sow estrus behavior, and thus effectively monitoring sow estrus;

[0077] (2) The present invention adopts an improved Kernel Shap algorithm, which can dynamically adjust the weight parameters of the prediction probability of each feature of sow estrus according to the actual prediction situation. Specifically, the contribution of the recognition results of the five estrus feature sub-models of sows to the final model prediction results is calculated, and a larger weight parameter is assigned to the feature with a higher contribution, thereby providing an explanation for the size of the final weight parameter of the prediction probability of each feature;

[0078] (3) The present invention adds a fractional attention mechanism to the behavioral feature discrimination model for sow estrus, namely the Two-Stream model, which can give high weights to important behavioral features, so that the model pays more attention to those behavioral features that are obvious, thereby making the sow behavioral feature recognition results closer to the actual situation;

[0079] (4) The present invention provides a temperature characteristic discrimination model for sow estrus, namely the yolov8 model. The present invention proposes a method for calculating the probability of sow estrus based on the vulva-buttock temperature difference, which combines the pig buttock detection probability to calculate the probability that the vulva-buttock temperature difference of the sow reaches the estrus state, thereby improving the confidence of the prediction result of the sow temperature characteristic;

[0080] (5) Based on the characteristic probabilities of sow estrus, the present invention proposes a method for selecting a sow estrus prediction method, wherein the estrus prediction method includes a single-value prediction or a multi-value weighted prediction, which can comprehensively consider the relationship between the sizes of various characteristic probabilities, thereby improving the efficiency of sow estrus prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 A schematic diagram of a sow estrus monitoring method;

[0082] Figure 2 The figure is a schematic diagram of the structure of a sow estrus monitoring system. DETAILED DESCRIPTION

[0083] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0084] like Figure 1 As shown, a sow estrus monitoring method includes steps S1-S6, which are as follows:

[0085] S1. Collect sow color video data and sow infrared video data.

[0086] In an optional embodiment of the present invention, the present invention collects sow color video data and sow infrared video data through a data acquisition module. The data acquisition module mainly collects data through a estrus inspection robot. The robot is equipped with a 2D color camera with a frame rate of 25fps, an infrared thermal imaging camera with a frame rate of 25fps, a speaker and a boar odor sprayer, and moves along the track above the four sides of the pig pen at a fixed time and point.

[0087] S2. Perform individual segmentation and key frame extraction on the sow color video data and sow infrared video data in step S1 to obtain whole pig color data segments, pig head color key segments, pig butt color key segments and pig butt infrared key segments.

[0088] In an optional embodiment of the present invention, the present invention performs individual segmentation and key frame extraction on sow color video data and sow infrared video data through a data preprocessing module to obtain whole pig color data segments, pig head color key segments, pig butt color key segments and pig butt infrared key segments.

[0089] Specifically, the present invention uses the instance segmentation model Mask R-CNN in the data preprocessing module to perform individual segmentation on the sow in each frame of the color video, and then uses the tracking algorithm DeepSORT to track the sow, thereby obtaining the whole pig color data of the sow, and then divides all frames into 720 segments based on 100 frames as the division standard to obtain the whole pig color data segments. The present invention uses the position information of the sow in the color video obtained by Mask R-CNN to obtain the position information of the sow in the infrared video, thereby performing individual segmentation on the sow in each frame of the infrared video, and finally obtaining the whole pig infrared data of the sow, and then divides all frames into 720 segments based on 100 frames as the division standard to obtain the whole pig infrared data segments.

[0090] The present invention extracts key frames from the whole pig color data segments and the whole pig infrared data segments to obtain the pig head color key segments, the pig butt color key segments and the pig butt infrared key segments, including the following steps:

[0091] B1. Use the target detection model yolov8 to perform target detection on the pig head and pig butt in the whole pig color data segment, obtain the detection frame information of the pig head and the detection frame information of the pig butt, and obtain the pig head color key segment and the pig butt color key segment based on the detection frame information of the pig head and the detection frame information of the pig butt.

[0092] Specifically, the present invention obtains the detection frame information of the whole pig from the results of the above-mentioned Mask R-CNN, and then calculates the area intersection and union ratio of the detection frame of the pig head with the detection frame of the whole pig and the area intersection and union ratio of the detection frame of the pig butt with the detection frame of the whole pig. If the area intersection and union ratio of the detection frame of the pig head with the detection frame of the whole pig is greater than 0.5, the image is determined as a color key segment of the pig head. If the area intersection and union ratio of the detection frame of the pig butt with the detection frame of the whole pig is greater than 0.5, the image is determined as a color key segment of the pig butt.

[0093] B2. Based on the whole pig color data segment, the pig butt color key segment and the whole pig infrared data segment, obtain the pig butt infrared key segment.

[0094] Specifically, the present invention assumes that the first frame of color image in the pig butt color key segment is the pth frame in the whole pig color data segment, and then takes the pth frame of infrared image in the whole pig infrared data segment as the first frame of the pig butt infrared key segment, so as to obtain the pig butt infrared key segment.

[0095] S3. Based on the whole pig color data segment, pig head color key segment, pig butt color key segment and pig butt infrared key segment in step S2, obtain the sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results and pig vulva and buttocks temperature difference reaching estrus state feature recognition results.

[0096] In an optional embodiment of the present invention, the present invention obtains the sow behavior feature recognition results, the pig face dull expression feature recognition results, the pig ear erect feature recognition results, the pig vulva mucus flow feature recognition results and the feature recognition results of the pig vulva and buttocks temperature difference reaching the estrus state through a data analysis module based on the whole pig color data segment, the pig head color key segment, the pig buttocks color key segment and the pig buttocks infrared key segment.

[0097] Step S3 includes the following steps:

[0098] S31. Based on the whole pig color data segment in step S3, a Two-Stream model is used to obtain the sow behavior feature recognition result.

[0099] The present invention adds a score attention mechanism to the Two-Stream model, and assigns weight parameters of 3:3:3:2:1 to the climbing behavior score, mounting behavior score, being mounted behavior score, standing still behavior score and other behavior scores in the sow's behavioral characteristics, respectively.

[0100] The present invention is based on the whole pig color data fragment in step S3, obtains the climbing behavior score, climbing mounting behavior score, being mounted behavior score, standing still behavior score and other behavior scores after the weight parameters are assigned, and then normalizes the climbing behavior score, climbing mounting behavior score, being mounted behavior score, standing still behavior score and other behavior scores after the weight parameters are assigned, obtains the probabilities of climbing behavior, climbing mounting behavior, being mounted behavior, standing still behavior and other behaviors, and sets the maximum probability value among the probabilities of climbing behavior, climbing mounting behavior, being mounted behavior, standing still behavior and other behaviors as the sow behavior feature recognition result.

[0101] S32. Based on the pig head color key fragment in step S3, the classic multi-task cascade convolutional neural network, residual convolutional network, multi-attention convolutional network and fully connected layer connected in sequence are used to obtain the pig face dull expression feature recognition result.

[0102] Specifically, the present invention adopts a classic multi-task cascade convolutional neural network to detect and locate the pig face of the pig head color key segment, and then uses a residual convolutional network to extract the features of the pig face to obtain a pig face convolution feature map, and then uses a multi-attention convolutional network to extract the pig face refined attention feature map according to the pig face convolution feature map. Finally, the pig face refined attention feature map is extracted through a fully connected layer to obtain the pig face dull expression feature recognition result.

[0103] S33. Based on the key color segment of the pig head in step S3, a resnet-50 network and an SVM classifier connected in sequence are used to obtain the pig ear erect feature recognition result.

[0104] Specifically, the present invention adopts the resnet-50 network as the backbone network to extract the pig ear features from the key color segments of the pig head, and then adopts the SVM classifier to perform the pig ear erection recognition on the extracted pig ear features to obtain the pig ear erection feature recognition result.

[0105] S34. According to the key color fragment of the pig buttocks in step S3, the yolov8 model, super-resolution model, resnet-50 network and SVM classifier connected in sequence are used to obtain the feature recognition result of the pig vulvar mucus.

[0106] Specifically, the present invention detects the pig vulva based on the color key fragment of the pig buttocks using the yolov8 model, then uses the super-resolution model to improve the resolution of the pig vulva, and then uses the resnet-50 network to extract the pig vulva features of the pig vulva with improved resolution, and finally uses the SVM classifier to perform pig vulva flow mucus feature recognition on the pig vulva features to obtain the pig vulva flow mucus feature recognition result.

[0107] S35. Based on the pig buttocks color key segment and the pig buttocks infrared key segment in step S3, the yolov8 model is used to obtain the feature recognition result of the pig's vulva and buttocks temperature difference reaching the estrus state.

[0108] Step S35 includes the following steps:

[0109] S351. Based on the pig buttocks color key segment in step S3, the yolov8 model is used to obtain the pig vulva position and pig buttocks position in the color image.

[0110] S352. Based on the infrared key segment of the pig buttocks in step S3, the position of the pig vulva and the position of the pig buttocks in the color image in step S351 are mapped to the infrared temperature matrix, and the temperature of the pig vulva and the temperature of the pig buttocks are obtained from the infrared temperature matrix.

[0111] The present invention maps the pig vulva position and pig buttocks position in the color image to the infrared temperature matrix, which is expressed as:

[0112]

[0113] Where: x ′ is the horizontal coordinate of the position point in the infrared temperature matrix, y ′ is the ordinate of the position point in the infrared temperature matrix, M is the affine transformation for transforming between the color image and the infrared image, x is the abscissa of the position point in the color image, and y is the ordinate of the position point in the color image.

[0114] S353, according to the pig vulva temperature and pig buttocks temperature in step S352, calculate the probability that the pig vulva and buttocks temperature difference reaches the estrus state, and obtain the characteristic recognition result of the pig vulva and buttocks temperature difference reaching the estrus state, expressed as:

[0115]

[0116] in: is the probability that the temperature difference between the genitals and buttocks of the i-th sow in the k-th frame of the j-th data segment reaches the estrus state, min is the minimum symbol, max is the maximum symbol, T V is the temperature of the pig vulva, T Gis the pig buttocks temperature, θ is the minimum threshold of the pig genital-buttocks temperature difference, λ is the confidence of the temperature information, and m is the maximum threshold of the pig genital-buttocks temperature difference.

[0117] S4. Perform single-value prediction or multi-value weighted prediction on the sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results and pig vulva-buttocks temperature difference reaching estrus state feature recognition results in step S3 to obtain the sow estrus segment prediction results, and obtain the sow estrus prediction results based on the sow estrus segment prediction results.

[0118] In an optional embodiment of the present invention, the present invention uses a prediction module to perform single-value prediction or multi-value weighted prediction on the sow behavior feature recognition results, the pig face dull expression feature recognition results, the pig ear erect feature recognition results, the pig vulva mucus flow feature recognition results, and the pig vulva and buttocks temperature difference reaching the estrus state feature recognition results to obtain the sow estrus segment prediction results, and obtain the sow estrus prediction results based on the sow estrus segment prediction results.

[0119] Step S4 includes the following steps:

[0120] S41, normalizing the sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results, and pig vulva-buttocks temperature difference reaching estrus state feature recognition results in step S3.

[0121] S42, using the sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results and pig vulva buttocks temperature difference reaching estrus state feature recognition results after normalization processing in step S41, calculate the entropy value between the feature recognition results, expressed as:

[0122]

[0123] Where: H ij (x) is the entropy value between the feature recognition results of the i-th sow in the j-th data segment, q is the traversal sequence number, p(x q ) is the qth feature recognition result after normalization.

[0124] S43, judging whether the entropy value between the feature recognition results in step S42 is greater than the multi-value weighted prediction threshold; if so, adopting multi-value weighted prediction to obtain the prediction result of the sow estrus segment, otherwise adopting single-value prediction to obtain the prediction result of the sow estrus segment.

[0125] The multi-value weighted prediction threshold in the present invention is 0.25.

[0126] The present invention adopts multi-value weighted prediction to obtain the prediction results of sow estrus segment, specifically:

[0127] Based on the improved Kernel Shap algorithm, the sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results and pig vulva-buttock temperature difference reaching estrus state feature recognition results in step S3 are weighted by credibility to obtain the sow estrus segment prediction result, which is expressed as:

[0128]

[0129] Where: P ij is the prediction result of the estrus segment of the i-th sow in the j-th data segment, σ(·) is the first activation function, A is the weight parameter of the sow behavior prediction probability, is the sow behavior feature recognition result of the i-th sow in the j-th data segment, B is the weight parameter for the prediction probability of the pig face dull expression, is the pig face dull expression feature recognition result of the i-th sow in the j-th data segment, C is the weight parameter of the prediction probability of the pig ear erection, is the recognition result of the erect ear feature of the i-th sow in the j-th data segment, D is the weight parameter for the prediction probability of mucus flow from the pig vulva, is the recognition result of the mucus characteristics of the pig vulva flow of the i-th sow in the j-th data segment, E is the weight parameter of the predicted probability that the temperature difference between the pig vulva and buttocks reaches the estrus state, It is the feature recognition result of the i-th sow’s vulva-buttock temperature difference reaching the estrus state in the j-th data segment.

[0130] The present invention adopts single value prediction to obtain the prediction result of the sow estrus segment, specifically: judging the sow behavior feature recognition result in step S3 Pig face dull expression feature recognition results Pig ear erect feature recognition results Results of pig vulvar mucus characteristics identification The characteristic recognition results of the pig's vulva-buttock temperature difference reaching the estrus state Whether the highest probability in the above equation exceeds 0.85, if so, the corresponding prediction result P of the sow estrus segment is directly ij If the stork is in heat, it is judged as being in heat; otherwise, it is judged as being out of heat.

[0131] S44, obtaining the sow estrus prediction result according to the sow estrus segment prediction result in step S43.

[0132] If the prediction result of sow estrus segment P ij If the value is higher than 0.85, the present invention will predict the corresponding sow estrus segment P ij If sow i is in estrus, it is judged as in estrus, otherwise it is judged as not in estrus, and then the number n of estrus segments of the i-th sow is counted.

[0133] In the present invention, each sow has a total of 720 video clips, and the calculation formula for the sow estrus prediction result is:

[0134]

[0135] Where: P i is the estrus prediction result of the i-th sow, and n is the number of estrus segments of the i-th sow.

[0136] If the estrus prediction result of the i-th sow is P i If the value exceeds 0.85, the present invention will convert the corresponding sow estrus prediction result P i If the stork is in heat, it is judged as being in heat; otherwise, it is judged as being out of heat.

[0137] S5. Based on the sow estrus prediction result in step S4, artificial estrus checking is performed on the sow to obtain the actual estrus result of the sow.

[0138] In an optional embodiment of the present invention, the present invention receives the sow estrus prediction result transmitted by the prediction module through the feedback module, and based on the sow estrus prediction result, performs artificial estrus check on the sow to obtain the actual estrus result of the sow.

[0139] Specifically, once the program in the present invention is executed and a sow is found to be in estrus, 5 clear photos of the sow in estrus are immediately selected and stored in the feedback module, and the pig farm broadcast function is started to inform the management personnel to come and check; once the pig farm management personnel receive the broadcast feedback on the estrus of the sow, they go to the server of the inspection robot to check the photos of the sow in estrus, find the corresponding sow for manual estrus check, and obtain the actual estrus results of the sow.

[0140] S6. Use the sow estrus prediction result in step S4 and the sow actual estrus result in step S5 to construct a training data set, use the training data set to fine-tune the weight parameters in the multi-value weighted prediction, and use the multi-value weighted prediction after the weight parameters are fine-tuned to predict and monitor the sow estrus.

[0141] In an optional embodiment of the present invention, the present invention constructs a training data set using the sow estrus prediction results and the sow actual estrus results in the update module, uses the training data set to fine-tune the weight parameters in the multi-value weighted prediction, and uses the multi-value weighted prediction after the weight parameters are fine-tuned to predict and monitor the sow estrus.

[0142] The present invention uses a training data set to fine-tune the weight parameters of the system model, including the following steps:

[0143] A1. Use the training data set to construct a prediction probability data set, and divide the prediction probability data set into a prediction probability training set and a prediction probability test set.

[0144] If the sow estrus prediction result in the training data set is the same as the sow's actual estrus result, all video clips are marked as true estrus, otherwise they are marked as false estrus. A total of 6 rounds of data can be collected in one day, with 720 video clips in one round, and a total of 4320 video clips can be collected, each of which corresponds to a sow estrus prediction result. The composition of these video clips exists in the following three situations:

[0145] First, there is only real estrus data, no false estrus data.

[0146] The present invention sorts all true estrus data in descending order based on the probability values ​​corresponding to the sow estrus prediction results, assuming that there are w probabilities higher than 0.85 and f probabilities lower than 0.35. If w is greater than 500, the first 500 video clips are taken as the estrus data set V estrus , otherwise take the first w groups, corresponding to each video label Y estrus Set to 1. If f is greater than 100, then take the last 100 video clips as the non-estrus dataset V nonEstrus Otherwise, take the inverse f group, corresponding to each video label Y nonEstrus is 0.

[0147] Second, there is only fake estrus data, no real estrus data.

[0148] The present invention sorts all fake estrus data in descending order, assuming that there are w videos with a probability higher than 0.85 and f videos with a probability lower than 0.35. If w is greater than 100, the last 100 video clips are taken as the estrus data set V. estrus , otherwise take the inverse w group, the corresponding Y estrus is 1. If f is greater than 500, the first 500 groups of estrus probabilities are taken as the non-estrus data set V nonEstrus , otherwise take the first f groups, the corresponding Y nonEstrus is 0.

[0149] Third, there are both real estrus data and false estrus data.

[0150] The present invention sorts the estrus probabilities obtained from the true estrus data in descending order. Assuming that there are w probabilities higher than 0.85, if w is greater than 500, the first 500 video clips are taken as the estrus data set V estrus , otherwise take the first w groups, the corresponding Y estrus The present invention sorts the estrus probabilities obtained from the fake estrus data in descending order, assuming that there are f probabilities below 0.35. If f is greater than 500, the last 500 video clips are taken as the non-estrus data set V. nonEstrus , otherwise take the inverse f group, the corresponding Y nonEstrus is 0.

[0151] In summary, the three cases above are used to obtain the video data set of sows for training, which is expressed as:

[0152] V i =V estrus +V nonestrus

[0153] Where: V i is the video dataset of the i-th sow, V estrus is the estrus dataset, V nonestrus This is the non-estrus data set.

[0154] Assume that there are s sows in total, and repeat the above construction process s times to obtain the total video dataset of sows for training, which is expressed as:

[0155]

[0156] Where: V is the total video dataset of sows, s is the total number of sows, V i is the video dataset of the i-th sow, V j is the jth video clip, t is the total number of video clips, and video clip V j There are corresponding labels Y j , Y j A value of 1 means that the segment is in estrus, V j A value of 0 indicates that the segment is not in estrus. If the system is used for the first time, the construction of V requires manual data screening.

[0157] The present invention performs a video segment V in a total video data set of a sow. j Perform global feature and local feature prediction in step S3 to obtain video segment V j Corresponding sow behavior feature recognition results Pig face dull expression feature recognition results Pig ear erect feature recognition results Results of pig vulvar mucus characteristics identification The characteristic recognition results of the pig's vulva-buttock temperature difference reaching the estrus state The present invention determines these five feature recognition results as predicted probability data, expressed as:

[0158]

[0159] Where: X j Predict probability data for the jth video segment.

[0160] The present invention constructs a prediction probability data set based on the prediction probability data, which is expressed as:

[0161]

[0162] Where: X is the predicted probability dataset.

[0163] The present invention divides the prediction probability data set into a prediction probability training set X and a prediction probability training set X. train and the predicted probability test set X test .

[0164] A2. Based on the predicted probability training set in step A1 and the improved Kernel Shap algorithm, the contribution of global features and local features is obtained.

[0165] Step A2 includes the following steps:

[0166] A21. Based on the prediction probability training set in step A1, a simplified feature set of prediction probability training data is constructed.

[0167] The present invention constructs a simplified feature set of prediction probability training data based on the prediction probability training set in step A1, which is expressed as:

[0168]

[0169] Where: Z j The simplified features for predicting the probability data for the jth video segment, z jo To simplify feature group Z j The oth simplified feature in .

[0170] The simplified feature z in the present invention jk is a five-dimensional (0, 1) vector, i.e. z jk ∈{0,1} 5 .

[0171] A22. Based on the improved Kernel Shap algorithm, a linear prediction function for sow estrus is constructed, and a mapping function for prediction probability training data and an explanation function for sow estrus are constructed.

[0172] In the present invention, a linear prediction function of sow estrus is constructed based on the improved Kernel Shap algorithm, which is expressed as:

[0173]

[0174] Where: F(·) is the linear prediction function of sow estrus, Coef is the prediction weight of the sub-model, Coef=(A,B,C,D,E), A is the weight parameter of the prediction probability of sow behavior, B is the weight parameter of the prediction probability of dull expression on the pig's face, C is the weight parameter of the prediction probability of erect ears, D is the weight parameter of the prediction probability of mucus flow from the pig's vulva, E is the weight parameter of the prediction probability that the temperature difference between the pig's vulva and buttocks reaches the estrus state, and T is the transposed sign.

[0175] The explanation function of sow estrus constructed in the present invention is expressed as:

[0176]

[0177] Among them: G(·) is the explanation function of sow estrus, is the mean value calculated by the linear prediction function of sow estrus based on the predicted probability data of the jth video clip, is the contribution of the u-th local feature, z jou To simplify the feature z jo The uth fragment of .

[0178] A23. According to the simplified feature group of the prediction probability training data in step A21, the linear prediction function of sow estrus in step A22, the mapping function of the prediction probability training data and the interpretation function of sow estrus, the contribution of the global features and the local features is obtained, which is expressed as:

[0179]

[0180] in: is the contribution of global features and local features, is the contribution of the predicted probability of sow behavior, is the contribution of the prediction probability of the pig face dull expression, is the contribution of the prediction probability of pig ears erect, is the contribution of the predicted probability of pig vulvar mucus discharge, is the contribution of the predicted probability of the pig's vulva-hip temperature difference reaching the estrus state, argmin is the variable value that minimizes the objective function, is the symbol for taking the average value, X j is the prediction result of sow estrus in the prediction probability dataset, X train is the predicted probability training set, z jo To simplify feature group Z j The oth simplified feature in Z j is the simplified feature of the predicted probability data of the jth video clip, F(·) is the linear prediction function of sow estrus, is the mapping function for predicting the probability training data, G(·) is the explanation function for sow estrus, Kernel weight function for training data for prediction probabilities.

[0181] The present invention calculates the kernel weight function of the simplified features of the prediction probability training data, which is expressed as:

[0182]

[0183] in: Simplified features z of training data for predicting probability jk The kernel weight function is is the symbol for the combination number, specifically 5 choices | z jk | the number of times, |z jk | for z jk The number of non-zero elements in .

[0184] A3. Based on the contribution of global features and local features in step A2, the updated prediction weight of the sub-model is obtained using the moving average method, which is expressed as:

[0185]

[0186] Where: Coef′ is the prediction weight of the sub-model after update, softmax(·) is the second activation function, Coef is the prediction weight of the sub-model, Coef=(A,B,C,D,E), A is the weight parameter of the prediction probability of sow behavior, B is the weight parameter of the prediction probability of dull expression on the pig's face, C is the weight parameter of the prediction probability of erect ears, D is the weight parameter of the prediction probability of mucus flow from the pig's vulva, E is the weight parameter of the prediction probability of the temperature difference between the pig's vulva and buttocks reaching the estrus state, β is the update weight, It is the contribution of global features and local features.

[0187] A4. Calculate the loss ratio of the sub-model based on the predicted probability test set in step A1 and the updated prediction weights in step A3.

[0188] Step A4 includes the following steps:

[0189] A41. Based on the predicted probability test set in step A1 and the predicted weight updated in step A3, the predicted probability of the predicted probability test set is calculated, which is expressed as:

[0190]

[0191] Where: P test is the predicted probability set of the predicted probability test set, σ(·) is the first activation function, Coef′ is the predicted weight after the sub-model is updated, is the prediction result X of sow estrus in the prediction probability test set i The transpose of is the predicted probability test set X test The transpose of .

[0192] A42. Based on the predicted probability of the predicted probability test set in step A41, the binary classification loss of the sow estrus behavior is calculated, which is expressed as:

[0193]

[0194] in: is the binary loss of sow estrus behavior, is the symbol for taking the average value, p is the predicted probability of estrus of the sample data in the predicted probability test set, y is the actual estrus result of the sample data in the predicted probability test set, and Y test is the label corresponding to the predicted probability test set.

[0195] A43. According to the binary loss of the sow estrus behavior in step A42, the loss ratio of the sub-model is calculated, which is expressed as:

[0196]

[0197] in: is the loss ratio of the rth sub-model, Coef′ r is the updated prediction weight of the rth sub-model, Coef′ j is the updated prediction weight of the j-th sub-model.

[0198] A5. Update the sub-model loss using the loss proportion of the sub-model in step A4 to fine-tune the weight parameters of the system model, expressed as:

[0199]

[0200] Where: θ r is the updated weight parameter of the rth sub-model, argmin is the variable value that minimizes the objective function, is the loss ratio of the rth sub-model, is the original prediction loss of the rth sub-model.

[0201] like Figure 2 As shown, a sow estrus monitoring system using the above method comprises:

[0202] The data acquisition module is used to collect sow color video data and sow infrared video data.

[0203] The data preprocessing module is used to perform individual segmentation and key frame extraction on sow color video data and sow infrared video data, and obtain whole pig color data segments, pig head color key segments, pig butt color key segments and pig butt infrared key segments.

[0204] The data analysis module is used to obtain the sow behavior feature recognition results, the pig face dull expression feature recognition results, the pig ear erect feature recognition results, the pig vulva mucus flow feature recognition results and the feature recognition results of the pig vulva and buttocks temperature difference reaching the estrus state based on the whole pig color data segment, the pig head color key segment, the pig buttocks color key segment and the pig buttocks infrared key segment.

[0205] The prediction module is used to perform single-value prediction or multi-value weighted prediction on the recognition results of sow behavior characteristics, pig facial dull expression characteristics, pig ear erection characteristics, pig vulvar mucus flow characteristics and pig vulva-buttock temperature difference reaching estrus state characteristics, obtain the prediction results of sow estrus segment, and obtain the sow estrus prediction results based on the sow estrus segment prediction results.

[0206] The feedback module is used to perform artificial estrus check on the sows according to the estrus prediction results of the sows to obtain the actual estrus results of the sows.

[0207] The updating module is used to construct a training data set based on the sow estrus prediction results and the sow's actual estrus results, and use the training data set to fine-tune the weight parameters in the multi-value weighted prediction.

[0208] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0209] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0210] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0211] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0212] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A method for monitoring estrus of sows, characterized in that: The following steps are involved: S1, collecting sow color video data and sow infrared video data; S2, performing individual segmentation and key frame extraction on the sow color video data and sow infrared video data in step S1, and obtaining whole pig color data segments, pig head color key segments, pig butt color key segments and pig butt infrared key segments; S3, based on the whole pig color data segment, pig head color key segment, pig buttocks color key segment and pig buttocks infrared key segment in step S2, obtain the sow behavior feature recognition result, pig face dull expression feature recognition result, pig ear erection feature recognition result, pig vulva mucus flow feature recognition result and pig vulva buttocks temperature difference reaching estrus feature recognition result; S4, performing single-value prediction or multi-value weighted prediction on the sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results, and pig vulva-buttock temperature difference reaching estrus state feature recognition results in step S3 to obtain the sow estrus segment prediction results, and obtain the sow estrus prediction results based on the sow estrus segment prediction results; S5, based on the sow estrus prediction result in step S4, artificial estrus checking is performed on the sow to obtain the actual estrus result of the sow; S6, using the sow estrus prediction result in step S4 and the sow actual estrus result in step S5 to construct a training data set, using the training data set to fine-tune the weight parameters in the multi-value weighted prediction, and using the multi-value weighted prediction after the weight parameters are fine-tuned to predict and monitor the estrus of the sow; Using the training data set to fine-tune the weight parameters in the multi-value weighted prediction includes the following steps: A1. Use the training data set to construct a prediction probability data set, and divide the prediction probability data set into a prediction probability training set and a prediction probability test set; A2. Obtain the contribution of global features and local features based on the prediction probability training set in step A1 and the improved Kernel Shap algorithm; A3. Based on the contribution of global features and local features in step A2, the updated prediction weight of the sub-model is obtained using the moving average method, which is expressed as: Where: Coef′ is the prediction weight of the sub-model after update, softmax(·) is the second activation function, Coef is the prediction weight of the sub-model, Coef=(A,B,C,D,E), A is the weight parameter of the prediction probability of sow behavior, B is the weight parameter of the prediction probability of dull expression on the pig's face, C is the weight parameter of the prediction probability of erect ears, D is the weight parameter of the prediction probability of mucus flow from the pig's vulva, E is the weight parameter of the prediction probability of the temperature difference between the pig's vulva and buttocks reaching the estrus state, β is the update weight, The contribution of global features and local features; A4. Calculate the loss ratio of the sub-model based on the predicted probability test set in step A1 and the predicted weight updated in step A3; A5. Update the sub-model loss using the loss proportion of the sub-model in step A4 to fine-tune the weight parameters of the system model, expressed as: Where: θ r is the updated weight parameter of the rth sub-model, argmin is the variable value that minimizes the objective function, is the loss ratio of the rth sub-model, is the original prediction loss of the rth sub-model.

2. A sow estrus monitoring method according to claim 1, characterized in that: Step S3 includes the following steps: S31, based on the whole pig color data fragment in step S3, using the Two-Stream model to obtain the sow behavior feature recognition result; S32, based on the pig head color key segment in step S3, using the classic multi-task cascade convolutional neural network, residual convolutional network, multi-attention convolutional network and fully connected layer connected in sequence to obtain the pig face dull expression feature recognition result; S33, based on the pig head color key segment in step S3, using the resnet-50 network and SVM classifier connected in sequence to obtain the pig ear erect feature recognition result; S34, based on the pig buttocks color key segment in step S3, using the yolov8 model, super-resolution model, resnet-50 network and SVM classifier connected in sequence to obtain the pig vulvar mucus feature recognition result; S35. Based on the pig buttocks color key segment and the pig buttocks infrared key segment in step S3, the yolov8 model is used to obtain the feature recognition result of the pig's vulva and buttocks temperature difference reaching the estrus state.

3. A sow estrus monitoring method according to claim 2, characterized in that: In step S31, a score attention mechanism is added to the Two-Stream model, and the climbing behavior score, mounting behavior score, being mounted behavior score, standing still behavior score and other behavior scores in the sow's behavior characteristics are assigned weight parameters of 3:3:3:2:1 respectively.

4. A sow estrus monitoring method according to claim 2, characterized in that: Step S35 includes the following steps: S351, based on the pig buttocks color key segment in step S3, using the yolov8 model to obtain the pig vulva position and the pig buttocks position in the color image; S352, based on the infrared key segment of the pig buttocks in step S3, mapping the pig vulva position and the pig buttocks position in the color image in step S351 to the infrared temperature matrix, and obtaining the pig vulva temperature and the pig buttocks temperature from the infrared temperature matrix; S353, according to the pig vulva temperature and pig buttocks temperature in step S352, calculate the probability that the pig vulva and buttocks temperature difference reaches the estrus state, and obtain the characteristic recognition result of the pig vulva and buttocks temperature difference reaching the estrus state, expressed as: in: is the probability that the temperature difference between the genitals and buttocks of the i-th sow in the k-th frame of the j-th data segment reaches the estrus state, min is the minimum symbol, max is the maximum symbol, T V is the temperature of the pig vulva, T G is the pig buttocks temperature, θ is the minimum threshold of the pig genital-buttocks temperature difference, λ is the confidence of the temperature information, and m is the maximum threshold of the pig genital-buttocks temperature difference.

5. A sow estrus monitoring method according to claim 1, characterized in that: Step S4 includes the following steps: S41, normalizing the sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results, and pig vulva-buttocks temperature difference reaching estrus state feature recognition results in step S3; S42, using the sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results and pig vulva buttocks temperature difference reaching estrus state feature recognition results after normalization processing in step S41, calculate the entropy value between the feature recognition results, expressed as: Where: H ij (x) is the entropy value between the feature recognition results of the i-th sow in the j-th data segment, q is the traversal sequence number, p(x q ) is the qth feature recognition result after normalization; S43, judging whether the entropy value between the feature recognition results in step S42 is greater than the multi-value weighted prediction threshold; if so, adopting multi-value weighted prediction to obtain the prediction result of the sow estrus segment, otherwise adopting single-value prediction to obtain the prediction result of the sow estrus segment; S44, obtaining the sow estrus prediction result according to the sow estrus segment prediction result in step S43.

6. A sow estrus monitoring method according to claim 5, characterized in that: In step S43, multi-value weighted prediction is used to obtain the prediction result of the sow estrus segment, specifically: Based on the improved Kernel Shap algorithm, the sow behavior feature recognition results, pig face dull expression feature recognition results, pig ear erection feature recognition results, pig vulva mucus flow feature recognition results and pig vulva-buttock temperature difference reaching estrus state feature recognition results in step S3 are weighted by credibility to obtain the sow estrus segment prediction result, which is expressed as: Where: P ij is the prediction result of the estrus segment of the i-th sow in the j-th data segment, σ(·) is the first activation function, A is the weight parameter of the sow behavior prediction probability, is the sow behavior feature recognition result of the i-th sow in the j-th data segment, B is the weight parameter for the prediction probability of the pig face dull expression, is the pig face dull expression feature recognition result of the i-th sow in the j-th data segment, C is the weight parameter of the prediction probability of the pig ear erection, is the recognition result of the erect ear feature of the i-th sow in the j-th data segment, D is the weight parameter for the prediction probability of mucus flow from the pig vulva, is the recognition result of the mucus characteristics of the pig vulva flow of the i-th sow in the j-th data segment, E is the weight parameter of the predicted probability that the temperature difference between the pig vulva and buttocks reaches the estrus state, It is the feature recognition result of the i-th sow’s vulva-buttock temperature difference reaching the estrus state in the j-th data segment.

7. A sow estrus monitoring method according to claim 1, characterized in that: Step A2 includes the following steps: A21, based on the prediction probability training set in step A1, construct a simplified feature set of prediction probability training data; A22. Construct a linear prediction function of sow estrus based on the improved Kernel Shap algorithm, and construct a mapping function of the prediction probability training data and an explanation function of sow estrus; A23. According to the simplified feature group of the prediction probability training data in step A21, the linear prediction function of sow estrus in step A22, the mapping function of the prediction probability training data and the interpretation function of sow estrus, the contribution of the global features and the local features is obtained, which is expressed as: in: is the contribution of global features and local features, is the contribution of the predicted probability of sow behavior, is the contribution of the prediction probability of the pig face dull expression, is the contribution of the prediction probability of pig ears erect, is the contribution of the predicted probability of pig vulvar mucus discharge, is the contribution of the predicted probability of the pig's vulva-hip temperature difference reaching the estrus state, argmin is the variable value that minimizes the objective function, is the symbol for taking the average value, X j is the prediction result of sow estrus in the prediction probability dataset, X train is the predicted probability training set, z jo To simplify feature group Z j The oth simplified feature in Z j is the simplified feature of the predicted probability data of the jth video clip, F(·) is the linear prediction function of sow estrus, is the mapping function for predicting the probability training data, G(·) is the explanation function for sow estrus, Kernel weight function for training data for prediction probabilities.

8. A sow estrus monitoring method according to claim 1, characterized in that: Step A4 includes the following steps: A41. Based on the predicted probability test set in step A1 and the predicted weight updated in step A3, the predicted probability of the predicted probability test set is calculated, which is expressed as: Where: P test is the predicted probability set of the predicted probability test set, σ(·) is the first activation function, Coef′ is the predicted weight after the sub-model is updated, is the prediction result X of sow estrus in the prediction probability test set i The transpose of is the predicted probability test set X test The transpose of A42. Based on the predicted probability of the predicted probability test set in step A41, the binary classification loss of the sow estrus behavior is calculated, which is expressed as: in: is the binary loss of sow estrus behavior, is the symbol for taking the average value, p is the predicted probability of estrus of the sample data in the predicted probability test set, y is the actual estrus result of the sample data in the predicted probability test set, and Y test is the label corresponding to the predicted probability test set; A43. According to the binary loss of the sow estrus behavior in step A42, the loss ratio of the sub-model is calculated, which is expressed as: in: is the loss ratio of the rth sub-model, Coef′ r is the updated prediction weight of the rth sub-model, Coef′ j is the updated prediction weight of the j-th sub-model.

9. A sow estrus monitoring system using the method according to any one of claims 1 to 8, characterized in that: include: Data acquisition module, used to collect sow color video data and sow infrared video data; The data preprocessing module is used to perform individual segmentation and key frame extraction on the sow color video data and the sow infrared video data, and obtain the whole pig color data segment, the pig head color key segment, the pig butt color key segment and the pig butt infrared key segment; A data analysis module is used to obtain the recognition results of sow behavior characteristics, pig face dull expression characteristics, pig ear erection characteristics, pig vulva mucus flow characteristics, and pig vulva and buttocks temperature difference reaching estrus state based on the whole pig color data segment, pig head color key segment, pig buttocks color key segment, and pig buttocks infrared key segment; A prediction module is used to perform single-value prediction or multi-value weighted prediction on the recognition results of sow behavior characteristics, pig face dull expression characteristics, pig ear erection characteristics, pig vulva mucus flow characteristics, and pig vulva-buttock temperature difference reaching estrus state characteristics, to obtain the prediction results of sow estrus segments, and obtain the sow estrus prediction results based on the sow estrus segment prediction results; A feedback module is used to perform manual estrus check on the sows according to the estrus prediction results of the sows to obtain the actual estrus results of the sows; The updating module is used to construct a training data set based on the sow estrus prediction results and the sow's actual estrus results, and use the training data set to fine-tune the weight parameters in the multi-value weighted prediction.

Images