Sow oestrus detection method and device based on thermal imaging and deep learning

Abstract

The invention discloses a sow oestrus detection method and device based on thermal imaging and deep learning, and the method comprises the steps: S1, collecting the infrared thermogram sequences of an ear root region and a vulva region of a target sow, and generating dual-region time sequence thermal imaging data; s2, performing temperature correction and key point positioning on the double-region time sequence thermal imaging data, and generating a region temperature characteristic spectrum through improved YOLOv8n model processing; and S3, inputting the regional temperature characteristic spectrum into a bidirectional long short-term memory network, extracting a temperature change trend, generating an estrus characteristic vector, and determining the estrus state and stage of the sow. The method can accurately and efficiently detect the oestrus state of the sow, and provides a scientific basis for agricultural production.

Description

Technical Field

[0001] This invention relates to the field of sow estrus detection technology, and in particular to a method and apparatus for sow estrus detection based on thermal imaging and deep learning. Background Technology

[0002] Estrus detection in sows is a crucial step in modern pig farming, directly impacting mating success rate, farrowing rate, and overall profitability. Traditional methods for detecting estrus in sows primarily rely on manual observation and physical stimulation, including observing vulvar congestion and swelling, testing the standing reflex, and observing behavioral changes.

[0003] Current common estrus detection technologies for sows have the following shortcomings: Manual observation is highly subjective, has low accuracy, requires experienced farmers, is labor-intensive, and inefficient; electronic olfactory sensor detection methods are greatly affected by environmental odors, prone to false positives, and have high equipment costs; automated monitoring systems based on single behavioral characteristics are highly dependent on the environment and difficult to adapt to the actual conditions of different farms; existing body surface temperature monitoring methods lack effective compensation for environmental factors, resulting in unstable measurement data and an inability to correct for individual sow differences; existing detection systems mostly use a single data source, failing to effectively integrate multimodal information and making it difficult to accurately determine the specific stage of estrus. These technical deficiencies lead to low accuracy and high false positive rates in estrus detection, affecting the accuracy of determining the optimal mating time for sows and ultimately impacting farming efficiency.

[0004] Therefore, there is an urgent need for a method and device for detecting estrus in sows based on thermal imaging and deep learning. Summary of the Invention

[0005] This invention provides a method and device for detecting estrus in sows based on thermal imaging and deep learning, in order to solve the above-mentioned problems existing in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for detecting estrus in sows based on thermal imaging and deep learning, comprising:

[0008] S1: Collect infrared thermal image sequences of the ear root region and vulva region of the target sow to generate dual-region time-series thermal imaging data;

[0009] S2: Perform temperature correction and key point localization on dual-region time-series thermal imaging data, and generate regional temperature feature maps by improving the YOLOv8n model.

[0010] S3: Input the regional temperature feature map into the bidirectional long short-term memory network, extract the temperature change trend, generate the estrus feature vector, and determine the estrus status and stage of the sow.

[0011] Step S1 includes:

[0012] S11: Set the infrared thermal imager acquisition parameters, continuously acquire images at fixed intervals in the ear root area of ​​the target sow, and generate an ear root infrared thermal image sequence;

[0013] S12: Keeping the acquisition parameters unchanged, continuously acquire images of the vulva area of ​​the target sow at the same interval to generate a vulva infrared thermogram sequence;

[0014] S13: Align and combine the ear root infrared thermal image sequence and the vulva infrared thermal image sequence according to the timestamp to form dual-region time-series thermal imaging data.

[0015] Step S1 further includes:

[0016] S14: Insert a thermometer into the rectum of the target sow to measure the temperature and record it as a baseline reference value for body temperature.

[0017] S15: Collect the activity frequency and intensity of the target sow using a wearable accelerometer to generate time-series activity data;

[0018] S16: Record the temperature, humidity, wind speed and wind temperature values ​​of the detected environment through environmental sensors to generate an environmental parameter set.

[0019] Step S2 includes:

[0020] S21: Combine rectal temperature with environmental parameters to establish a temperature compensation model, perform pixel-level temperature correction on the infrared thermogram, and generate a corrected thermogram;

[0021] S22: The HRNet lightweight key point detection network is used to locate temperature-sensitive points in the ear root and vulva region, and to extract a fixed-size target region image.

[0022] S23: Input the target region image from the corrected heat map into the improved YOLOv8n model to extract multi-level regional temperature feature maps.

[0023] Step S21 includes:

[0024] S211: Calculate the difference between rectal temperature and infrared thermogram surface temperature, establish a body surface temperature difference mapping function, and generate a preliminary correction factor;

[0025] S212: Analyze the influence weight of environmental parameters on the temperature distribution of infrared thermograms, construct an environmental factor compensation function, and generate environmental correction factors;

[0026] S213: Combine the body surface temperature difference mapping function with the environmental factor compensation function to construct a correction model and perform temperature numerical correction on the original infrared thermogram.

[0027] The improved structure of the YOLOv8n model includes:

[0028] S61: Replace the original YOLOv8n backbone network with the MobileNetV3 network designed for thermal imaging characteristics, and adopt a depthwise separable convolutional structure;

[0029] S62: Add a BiFPN bidirectional feature pyramid structure to the neck network to build bidirectional connections between feature levels and form a multi-scale feature fusion network;

[0030] S63: The original path aggregation network is replaced by a weighted bidirectional feature pyramid network. By learning the weight coefficients, the heat features of the ear root and vulva are fused to generate a comprehensive feature representation.

[0031] Step S3 includes:

[0032] S31: Organize the temperature feature maps of consecutive frames in chronological order to construct a temporal feature sequence with a fixed window size;

[0033] S32: Process time-series feature sequences through a bidirectional long short-term memory network, extract temperature change trend features, and fuse them with activity data to generate a multimodal estrus feature vector;

[0034] S33: Input the estrus feature vector into a fully connected neural network classifier, calculate the probability value of each estrus stage, and determine the estrus state and stage.

[0035] Step S32 includes:

[0036] S321: Construct a long short-term memory unit network structure that includes a forget gate, an input gate, and an output gate, and establish a bidirectional long short-term memory network in both forward and backward directions;

[0037] S322: Input the time-series feature sequence into a bidirectional long short-term memory network, extract short-term and long-term temperature change trend features through a gating mechanism, and generate a time-series feature representation;

[0038] S323: An attention mechanism is used to fuse temporal feature representations with activity data at the feature level, generating an estrus feature vector containing spatiotemporal multimodal information.

[0039] Step S33 includes:

[0040] S331: Construct a fully connected neural network with two hidden layers to process estrus feature vectors and calculate the probability values ​​of four states: non-estrus, proestrus, peak estrus, and mesestrus.

[0041] S332: Normalize the probability values ​​of the four states using the Softmax function, select the state with the highest probability as the result of estrus judgment, and record the probability value as the confidence level of the judgment;

[0042] S333: Uses gradient-weighted class activation mapping technology to generate a heat map of regional temperature changes, marks the temperature-sensitive areas on which the judgment is based, and provides a visual interpretation of the results.

[0043] The device for detecting estrus in sows based on thermal imaging and deep learning includes:

[0044] The dual-area thermal imaging acquisition module consists of a first infrared thermal imaging sensor and a second infrared thermal imaging sensor. The first sensor is installed directly in front of the sow's ear root area, and the second sensor is installed diagonally above the sow's vulva area.

[0045] The environmental monitoring module includes a temperature sensor, a humidity sensor, a wind speed sensor, and a wind temperature sensor that are connected to the data acquisition terminal.

[0046] The temperature correction module includes a calculation unit for the storage surface temperature difference mapping function and an evaluation unit for the storage environment parameter compensation matrix.

[0047] The key point localization module includes an image processor running an HRNet convolutional neural network, which outputs the coordinates of key points in the ear root and vulva areas.

[0048] The feature extraction module includes a computing device for deploying the improved YOLOv8n model, which features a MobileNetV3 backbone network and a bidirectional feature pyramid structure.

[0049] The time series analysis module includes a bidirectional long short-term memory network processor and a fully connected neural network classifier.

[0050] Compared with the prior art, the present invention has the following advantages:

[0051] A method for detecting estrus in sows based on thermal imaging and deep learning includes: S1: acquiring infrared thermogram sequences of the ear base and vulva regions of the target sow to generate dual-region time-series thermal imaging data; S2: performing temperature correction and key point localization on the dual-region time-series thermal imaging data, and generating regional temperature feature maps through an improved YOLOv8n model; S3: inputting the regional temperature feature maps into a bidirectional long short-term memory network to extract temperature change trends, generate estrus feature vectors, and determine the estrus state and stage of the sow. By acquiring infrared thermogram sequences of the ear base and vulva regions of the sow, performing temperature correction and key point localization, extracting regional temperature feature maps using an improved YOLOv8n model, and inputting them into a bidirectional long short-term memory network to extract temperature change trends and generate estrus feature vectors, the method can accurately and efficiently detect the estrus state of sows, providing a scientific basis for agricultural production.

[0052] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention.

[0053] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0054] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0055] Figure 1 This is a flowchart of a sow estrus detection method based on thermal imaging and deep learning in an embodiment of the present invention;

[0056] Figure 2 This is a structural diagram of a sow estrus detection device based on thermal imaging and deep learning, as described in an embodiment of the present invention. Detailed Implementation

[0057] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0058] This invention provides a method for detecting estrus in sows based on thermal imaging and deep learning, comprising:

[0059] S1: Collect infrared thermal image sequences of the ear root region and vulva region of the target sow to generate dual-region time-series thermal imaging data;

[0060] S2: Perform temperature correction and key point localization on dual-region time-series thermal imaging data, and generate regional temperature feature maps by improving the YOLOv8n model.

[0061] S3: Input the regional temperature feature map into the bidirectional long short-term memory network, extract the temperature change trend, generate the estrus feature vector, and determine the estrus status and stage of the sow.

[0062] The working principle and beneficial effects of the above technical solution are as follows: Step S1: Farm technicians conduct infrared thermal imaging data collection on the target sows in the pen. The staff uses a handheld thermal imager to first scan the base of the sow's ears, and then scan the vulva area, acquiring thermal image sequences for both areas. These two areas are key sites for detecting estrus, and their temperature changes are closely related to the sow's physiological cycle. The base of the ears represents the overall blood circulation status, while the vulva area directly reflects changes in the reproductive organs. During the data collection process, the operator maintains a fixed distance of 80 cm from the sow to ensure that the temperature distribution in the area captured by the thermal imager is clearly discernible.

[0063] Step S2: The system preprocesses and extracts features from the acquired dual-region time-series thermal imaging data. First, a temperature correction model is established using measured body temperature and environmental parameters to correct the original thermal map. Next, an improved HRNet localization network is used to determine key points in the region and define accurate regions of interest. The improved YOLOv8n model replaces the standard backbone network with MobileNetV3 and adds a temperature sensing layer to extract temperature gradients, hotspot distributions, and boundary features within the region, generating temperature feature maps. These feature maps contain multi-level information, including both macroscopic temperature distributions and microscopic hotspot distributions.

[0064] Step S3: The system feeds the extracted regional temperature feature map into a bidirectional long short-term memory network for time-series analysis. This network consists of a 128-unit input layer, two bidirectional LSTM layers (64 units each), and a 32-unit fully connected layer. The network structure can capture temperature change trends and periodic patterns, extracting key feature vectors representing estrus status. Finally, a three-classifier converts the feature vectors into estrus status judgment results: non-estrus, proestrus, and peak estrus. The system also outputs confidence scores to assist aquaculture personnel in decision-making.

[0065] In another embodiment, step S1 includes:

[0066] S11: Set the infrared thermal imager acquisition parameters, continuously acquire images at fixed intervals in the ear root area of ​​the target sow, and generate an ear root infrared thermal image sequence;

[0067] S12: Keeping the acquisition parameters unchanged, continuously acquire images of the vulva area of ​​the target sow at the same interval to generate a vulva infrared thermogram sequence;

[0068] S13: Align and combine the ear root infrared thermal image sequence and the vulva infrared thermal image sequence according to the timestamp to form dual-region time-series thermal imaging data.

[0069] The working principle and beneficial effects of the above technical solution are as follows: Step S11: The farm operator sets the acquisition parameters of the infrared thermal imager. The parameters include: imaging temperature range set to 30-45℃, sensitivity adjusted to 0.05℃, emissivity calibration to 0.95 (close to the emissivity of pig skin), distance compensation set to 80 cm, and sampling frequency of 2 frames per minute. The operator aims the thermal imager lens at the ear root area of ​​the target sow, maintains a stable posture, and continuously acquires data for 15 minutes, generating a total of 30 frames of ear root infrared thermal image sequence. Each frame has a resolution of 256×192 pixels and includes a temperature data matrix and a corresponding timestamp.

[0070] Step S12: After completing the ear root area data acquisition, the operator immediately turns the thermal imager to the vulva area of ​​the same sow, keeping the thermal imager parameter settings unchanged to ensure that the data acquisition conditions for both sets are consistent. Continue acquiring data for 15 minutes to generate a vulvar infrared thermal image sequence. During the acquisition process, care should be taken to avoid image blurring caused by the pig's vigorous movement; if necessary, the acquisition can be paused until the pig returns to a calm state before continuing. Due to its special physiological structure, the vulva area has a relatively concentrated and significantly changing temperature distribution.

[0071] Step S13: After acquisition, the system aligns the ear root infrared thermal image sequence with the vulva infrared thermal image sequence based on the timestamp information embedded in the images. For frames that do not completely correspond in time, the system uses linear interpolation to fill in the missing frames, ensuring that the two sequences correspond one-to-one in the time dimension. The aligned data forms a dual-region temporal thermal imaging dataset, with each time point containing thermal images of both the ear root and vulva regions, constituting complete detection input data.

[0072] In another embodiment, step S1 further includes:

[0073] S14: Insert a thermometer into the rectum of the target sow to measure the temperature and record it as a baseline reference value for body temperature.

[0074] S15: Collect the activity frequency and intensity of the target sow using a wearable accelerometer to generate time-series activity data;

[0075] S16: Record the temperature, humidity, wind speed and wind temperature values ​​of the detected environment through environmental sensors to generate an environmental parameter set.

[0076] The working principle and beneficial effects of the above technical solution are as follows: Step S14: While performing thermal imaging acquisition, the veterinarian or technician uses an electronic thermometer to measure the rectal temperature of the target sow. The operator first covers the electronic thermometer with a disposable protective sleeve, applies an appropriate amount of lubricant, and then gently inserts the thermometer into the sow's rectum about 10 cm deep, close to the rectal wall. After the reading stabilizes, the displayed temperature is recorded, usually between 38.0-39.5℃. This temperature serves as a benchmark reference value for the sow's actual body temperature, used to correct for surface temperature measurement deviations in infrared thermal imaging.

[0077] Step S15: Technicians install a custom-made wearable accelerometer on the neck or ear of the target sow. This device includes a triaxial accelerometer with a sampling frequency of 20Hz and an acceleration range of ±8g. The accelerometer has a built-in data processing chip that automatically identifies different activity patterns of the sow, such as standing, walking, and running, through signal filtering and threshold analysis. The device records the frequency and intensity changes of activity over 24 hours, generating time-series activity data in 10-minute time windows, including two indicators: movement frequency and average intensity. These data are highly correlated with the increased activity during estrus.

[0078] Step S16: Install a multi-parameter environmental monitoring device in the detection environment, including a temperature sensor, humidity sensor, wind speed sensor, and wind temperature sensor. The temperature sensor is a PT100 platinum resistance thermometer with an accuracy of ±0.1℃; the humidity sensor is capacitive with an accuracy of ±3%; the wind speed sensor is a thermal anemometer with a range of 0-10m / s; the wind temperature sensor is integrated with the wind speed sensor. The system collects environmental parameters every 5 minutes, recording the ambient temperature, relative humidity, wind speed, and wind temperature values ​​during the detection period, generating an environmental parameter set. These parameters are used for subsequent infrared thermal image correction calculations.

[0079] In another embodiment, step S2 includes:

[0080] S21: Combine rectal temperature with environmental parameters to establish a temperature compensation model, perform pixel-level temperature correction on the infrared thermogram, and generate a corrected thermogram;

[0081] S22: The HRNet lightweight key point detection network is used to locate temperature-sensitive points in the ear root and vulva region, and to extract a fixed-size target region image.

[0082] S23: Input the target region image from the corrected heat map into the improved YOLOv8n model to extract multi-level regional temperature feature maps.

[0083] The working principle and beneficial effects of the above technical solution are as follows: Step S21: Technicians establish a temperature compensation model for correction based on the collected rectal temperature and environmental parameters. First, a linear regression equation is constructed: Tc = a·Tr + b·Te + c·H + d·V + e, where Tc is the correction temperature, Tr is the original infrared temperature, Te is the ambient temperature, H is the humidity, V is the wind speed, and a to e are regression coefficients. The coefficient values ​​are calculated using the least squares method to obtain the correction equation. The system applies this equation to each pixel of the original thermal image to correct the temperature value, generating a corrected thermal image. The correction process eliminates the interference of environmental factors on surface temperature measurement.

[0084] Step S22: The system calls the pre-trained HRNet lightweight keypoint detection network to locate temperature-sensitive points. This network uses MobileNetV3 as the feature extraction backbone, and halves the number of output channels to improve computation speed. Keypoint localization includes: the base of the ear, the bifurcation point of the ear vessels, and the lateral carotid vein in the ear root region; and the upper edge of the vulva, the center point of the vulva, and the lower edge of the vulva in the vulvar region. The network marks the coordinates of these keypoints on each heatmap, and then extracts a fixed-size target region image of 100×100 pixels centered on the keypoints to ensure the consistency of the subsequent analysis area.

[0085] Step S23: Input the target region image from the corrected heatmap into the improved YOLOv8n model. Improvements include: replacing CSPDarknet with the lightweight MobileNetV3 backbone network; adding a temperature-sensitive attention module to highlight the features of high temperature difference regions; and modifying the detector head structure to add a temperature gradient branch. The model extracts feature maps at three scales: downsampled by 4x, 8x, and 16x, capturing detailed temperature changes, intermediate-scale temperature patterns, and global temperature distribution, respectively. These multi-scale feature maps are fused to form a regional temperature feature map, containing rich temperature spatial information.

[0086] In another embodiment, step S21 includes:

[0087] S211: Calculate the difference between rectal temperature and infrared thermogram surface temperature, establish a body surface temperature difference mapping function, and generate a preliminary correction factor;

[0088] S212: Analyze the influence weight of environmental parameters on the temperature distribution of infrared thermograms, construct an environmental factor compensation function, and generate environmental correction factors;

[0089] S213: Combine the body surface temperature difference mapping function with the environmental factor compensation function to construct a correction model and perform temperature numerical correction on the original infrared thermogram.

[0090] The working principle and beneficial effects of the above technical solution are as follows: Step S211: Technicians compare the rectal temperature with the surface temperature of multiple preset reference points on the infrared thermogram. The base of the ear, the middle of the auricle, the side of the neck, and the center of the vulva are selected as reference points. The difference between the surface temperature and the rectal temperature at each point is calculated, resulting in a difference sequence D = d1, d2, ..., dn. A body surface temperature difference mapping function is fitted based on experimental data. This function is used to generate a preliminary correction factor matrix, assigning a corresponding correction weight to each pixel.

[0091] Step S212: Systematically analyze the influence of environmental parameters on the temperature distribution of infrared thermograms. By comparing thermogram data of the same sow under different environmental conditions, an environmental impact model is established. The calculation formula is: E(T,H,V,W)=α·(T-T0)+β·(H-H0)+γ·(V-V0)+δ·(W-W0), where T is the ambient temperature, H is the humidity, V is the wind speed, W is the wind temperature, T0, H0, V0, and W0 are standard environmental parameters, and α, β, γ, and δ are weighting coefficients. Multivariate regression analysis is used to determine the values ​​of each coefficient, generating an environmental correction factor matrix to quantify the degree of influence of environmental conditions on temperature measurements in each area.

[0092] Step S213: The system combines the body surface temperature difference mapping function with the environmental factor compensation function to construct a comprehensive correction model. The correction formula is: Tc(x,y)=Tr(x,y)+F(x,y,Tr(x,y))+E(T,H,V,W)·M(x,y), where Tc is the corrected temperature, Tr is the original temperature, and M is the location weight matrix, considering the differences in sensitivity to environmental factors in different regions. The system applies this formula to each pixel of the original infrared thermogram for temperature numerical correction, generating the final corrected thermogram. The temperature distribution of the corrected thermogram more accurately reflects the actual body surface temperature of the sow, laying the foundation for subsequent estrus feature extraction.

[0093] In another embodiment, the structure of the improved YOLOv8n model includes:

[0094] S61: Replace the original YOLOv8n backbone network with the MobileNetV3 network designed for thermal imaging characteristics, and adopt a depthwise separable convolutional structure;

[0095] S62: Add a BiFPN bidirectional feature pyramid structure to the neck network to build bidirectional connections between feature levels and form a multi-scale feature fusion network;

[0096] S63: The original path aggregation network is replaced by a weighted bidirectional feature pyramid network. By learning the weight coefficients, the heat features of the ear root and vulva are fused to generate a comprehensive feature representation.

[0097] The working principle and beneficial effects of the above technical solution are as follows: Step S61: System developers modify the original YOLOv8n network structure and replace its backbone network. Technicians first remove the original CSPDarknet backbone and import a MobileNetV3 network structure specifically designed for thermal imaging. This structure uses depthwise separable convolutional units instead of traditional convolution, decomposing the convolution operation into two steps: depthwise convolution and pointwise convolution. In depthwise convolution, each convolutional kernel operates on only one input channel, and the number of filters is the same as the number of input channels; pointwise convolution uses a 1×1 convolutional kernel to integrate channel information. This structure significantly reduces the number of model parameters and is suitable for deployment on edge computing devices in aquaculture farms. Improvements in thermal imaging characteristics include: adjusting the input channels to single-channel temperature data, increasing the initial convolutional layer filter size from 3×3 to 5×5, and capturing larger temperature gradient regions.

[0098] Step S62: Engineers added a BiFPN structure to the network neckline to enhance multi-scale feature extraction capabilities. Traditional feature pyramids transmit information unidirectionally from high to low layers, while BiFPN constructs bidirectional connections between layers. Engineers first designed a top-down path to transmit semantically strong high-level features to low layers; then they designed a bottom-up path to transmit detailed low-level information to high layers. Each BiFPN module contains 5 nodes, corresponding to feature layers P3 to P7. The connections between nodes use a combination of skip connections and residual connections. The BiFPN modules are stacked three times in the network to deepen feature interaction. Each feature fusion uses addition rather than concatenation to maintain the feature dimension and reduce computational overhead.

[0099] Step S63: Developers construct a weighted bidirectional feature pyramid network to replace the original path aggregation network. The weighted BiFPN structure introduces learnable weight coefficients to control the fusion ratio of different feature layers. During implementation, engineers add weight parameters to each feature fusion node, initialized with a uniform distribution. The fusion calculation formula is: O = ∑(wi·Ii) / ∑wi, where O is the output feature, Ii is each input feature, and wi is the corresponding weight. To ensure the weights are non-negative, the system uses ReLU activation to process the weight values. For the characteristics of thermal imaging, a temperature gradient branch is added to the fusion node to extract and retain features from areas with significant temperature changes. The weight initialization value for the ear root region is higher than that for the edge region, and the weight for the vulva center region is higher than that for the surrounding region, guiding the network to focus on the temperature characteristics of physiologically sensitive areas.

[0100] In another embodiment, step S3 includes:

[0101] S31: Organize the temperature feature maps of consecutive frames in chronological order to construct a temporal feature sequence with a fixed window size;

[0102] S32: Process time-series feature sequences through a bidirectional long short-term memory network, extract temperature change trend features, and fuse them with activity data to generate a multimodal estrus feature vector;

[0103] S33: Input the estrus feature vector into a fully connected neural network classifier, calculate the probability value of each estrus stage, and determine the estrus state and stage.

[0104] The working principle and beneficial effects of the above technical solution are as follows: Step S31: The data processing system organizes the collected regional temperature feature maps in a time series. Technicians set the sliding window size to 12 frames, corresponding to a 6-minute observation period, with a step size of 4 frames and a window overlap rate of two-thirds. The program reads the temperature feature maps stored in the system in chronological order, checking the timestamps to ensure sequence continuity. For occasional missing frames, the system generates missing frames through linear interpolation of adjacent frames. Each feature map has a dimension of 32×32×64, containing temperature feature information of the ear root region and the vulva region. The system organizes the 12 consecutive feature maps within each window into a four-dimensional data structure [12,32,32,64], forming a time series feature sequence with a fixed window size, which serves as the input unit for subsequent time series analysis.

[0105] Step S32: The system calls the trained bidirectional long short-term memory network to extract temperature change trends. This network adopts a stacked structure, containing two bidirectional LSTM layers with 128 and 64 units respectively. The network first compresses the spatial dimension of the input features, converting the 32×32×64 feature map into a 2048-dimensional vector, and then inputs it into the LSTM for processing. The forward LSTM captures the temperature increase trend, and the backward LSTM captures the temperature decrease feature, together forming a complete temperature change pattern. At the same time, the system reads the activity data collected by the accelerometer and constructs an activity feature vector of the same length as the temperature sequence. The temperature change features and activity features are fused through an attention mechanism, and key time point features are weighted to generate a multimodal estrus feature vector with a dimension of 256.

[0106] Step S33: The system inputs the estrus feature vector into a fully connected neural network classifier to determine the estrus state. This classifier contains a three-layer fully connected structure with 128, 64, and 4 nodes respectively, and uses ReLU activation. The last layer uses Softmax activation and outputs four values, representing the probability values ​​of the sow being in four states: "not in estrus," "proestrus," "peak estrus," and "metestrus." The system selects the state with the highest probability as the judgment result and records its probability value as the confidence level. When the confidence level is lower than the preset threshold of 0.7, the system marks it as a "pending confirmation" state, and it is recommended that the farm personnel conduct manual inspection and confirmation. The system also generates a heat map of the temperature-sensitive area to visually display the area on which the judgment is based.

[0107] In another embodiment, step S32 includes:

[0108] S321: Construct a long short-term memory unit network structure that includes a forget gate, an input gate, and an output gate, and establish a bidirectional long short-term memory network in both forward and backward directions;

[0109] S322: Input the time-series feature sequence into a bidirectional long short-term memory network, extract short-term and long-term temperature change trend features through a gating mechanism, and generate a time-series feature representation;

[0110] S323: An attention mechanism is used to fuse temporal feature representations with activity data at the feature level, generating an estrus feature vector containing spatiotemporal multimodal information.

[0111] The working principle and beneficial effects of the above technical solution are as follows: Step S321, algorithm engineers write code to construct a Long Short-Term Memory (LSTM) network. Each LSTM unit contains three gating components: the forget gate controls the proportion of historical information retained, the input gate determines the degree of current information adoption, and the output gate manages the intensity of information output. The forget gate structure is a fully connected layer followed by a Sigmoid function, calculated as ft=σ(Wf·[ht-1,xt]+bf), where ht-1 is the hidden state at the previous time step, xt is the current input, and Wf and bf are the weights and biases. The input gate consists of two parts: it=σ(Wi·[ht-1,xt]+bi) determines which values ​​are updated. Create candidate states. The cell state update formula is as follows: The output is calculated as ht = ot * tanh(Ct), where ot is the output gate value. The engineers built a bidirectional structure containing two sets of LSTM units, one for forward and one for backward propagation, to process the forward and reverse timing information respectively.

[0112] Step S322: The system runs an LSTM network to process the time-series feature sequence. Forward processing captures the rising trend of body temperature from frame 1 to frame 12; backward processing captures the decreasing trend of body temperature from frame 12 to frame 1. The specific working process of the network gating mechanism is as follows: when the temperature change between two adjacent frames is less than 0.1℃, the forgetting threshold is close to 1, and the historical state is retained; when the temperature change is significant, the input threshold is increased, and the weights of the current observation information are updated. At each time step, the LSTM computation unit updates the formula. ft and it represent the forgetting ratio and the input ratio, respectively. When the temperature in the ear root region rises by more than 0.3℃ or the temperature in the vulva region rises by more than 0.4℃, the algorithm assigns high input weights to capture possible estrus signals. After complete sequence processing, the system extracts the final hidden states of the forward and backward LSTMs and concatenates them to form a 128-dimensional temporal feature representation.

[0113] Step S323: Technicians implement an attention mechanism to fuse multimodal features. First, a self-attention module is constructed to process temperature sequence features and calculate the correlation matrix at each time point within the sequence. The self-attention output is V·A, where V is the value matrix. Activity data is also processed through self-attention to highlight key behavioral changes. Then, a cross-modal attention mechanism is implemented: using temperature features as the query and activity features as the key and value, Across is calculated to align the two modalities. Finally, the algorithm connects the self-attention output and the cross-modal attention output, compressing them into a 256-dimensional vector through a fully connected layer to form an estrus feature vector containing information on temperature changes and activity patterns, which serves as the basis for judgment.

[0114] In another embodiment, step S33 includes:

[0115] S331: Construct a fully connected neural network with two hidden layers to process estrus feature vectors and calculate the probability values ​​of four states: non-estrus, proestrus, peak estrus, and mesestrus.

[0116] S332: Normalize the probability values ​​of the four states using the Softmax function, select the state with the highest probability as the result of estrus judgment, and record the probability value as the confidence level of the judgment;

[0117] S333: Uses gradient-weighted class activation mapping technology to generate a heat map of regional temperature changes, marks the temperature-sensitive areas on which the judgment is based, and provides a visual interpretation of the results.

[0118] The working principle and beneficial effects of the above technical solution are as follows: Step S331: Engineers construct a fully connected neural network to determine estrus status. The network includes an input layer, two hidden layers, and an output layer. The input layer receives a 256-dimensional estrus feature vector; the first hidden layer contains 128 neurons, uses the ReLU activation function, and adds a weight regularization term with an L2 regularization coefficient of 0.01 to prevent overfitting; the second hidden layer contains 64 neurons, also using ReLU activation and applying regularization; the output layer contains 4 neurons, corresponding to the four states of non-estrus, proestrus, peak estrus, and mesestrus. The formula for calculating the full connection between layers is: z = W·x + b, where W is the weight matrix, b is the bias vector, x is the input feature, and z is the output value. The total number of parameters in the fully connected network is approximately 45,000, which is convenient for running on ordinary computing platforms. The network optimizes the cross-entropy loss function using gradient descent, with a learning rate set to 0.001, and uses the Adam optimizer.

[0119] Step S332: The system applies the Softmax function to normalize the network output values. The Softmax function is calculated as: pi = exp(zi) / ∑exp(zj), where zi is the original output value of the i-th state, and pi is the transformed probability value. Four state probability values ​​are calculated, summing to 1, representing the likelihood of the sow being in each state. The system selects the state with the highest probability value as the final judgment result and records this probability value as a confidence index. A decision threshold of 0.7 is set; when the highest probability is lower than this threshold, the system marks the judgment result as "uncertain" and suggests manual review by the farmer. To evaluate model performance, the system records the confusion matrix, calculates the accuracy, recall, and F1 score for each category, and monitors the model's judgment ability at different estrus stages.

[0120] Step S333: Technicians implement gradient-weighted class activation mapping (CAM) technology to provide a visual basis for judgment. The system first records the feature map output F of the last convolutional layer of the network, with dimensions [h, w, c]; then calculates the gradient G of the target class with respect to the feature map, also with dimensions [h, w, c]; performs global average pooling on the gradient to obtain the weight α of each channel; finally, calculates the weighted feature map. The system merges the CAM with the original heatmap to generate a heatmap, with red areas representing the temperature-sensitive areas that contribute the most to the judgment result. Typically, the heatmap during proestrus shows a slight increase in temperature in the ear root area; during peak estrus, the ear root and vulva areas show significantly high temperatures simultaneously.

[0121] In another embodiment, a sow estrus detection device based on thermal imaging and deep learning includes:

[0122] The dual-area thermal imaging acquisition module consists of a first infrared thermal imaging sensor and a second infrared thermal imaging sensor. The first sensor is installed directly in front of the sow's ear root area, and the second sensor is installed diagonally above the sow's vulva area.

[0123] The environmental monitoring module includes a temperature sensor, a humidity sensor, a wind speed sensor, and a wind temperature sensor that are connected to the data acquisition terminal.

[0124] The temperature correction module includes a calculation unit for the storage surface temperature difference mapping function and an evaluation unit for the storage environment parameter compensation matrix.

[0125] The key point localization module includes an image processor running an HRNet convolutional neural network, which outputs the coordinates of key points in the ear root and vulva areas.

[0126] The feature extraction module includes a computing device for deploying the improved YOLOv8n model, which features a MobileNetV3 backbone network and a bidirectional feature pyramid structure.

[0127] The time series analysis module includes a bidirectional long short-term memory network processor and a fully connected neural network classifier.

[0128] The working principle and beneficial effects of the above technical solution are as follows: The dual-zone thermal imaging acquisition module consists of two infrared thermal imaging sensors, which are respectively installed at the base of the sow's ear and the vulva area. The first sensor is located in front of the base of the ear, while the second sensor is positioned diagonally above and aimed at the vulva to ensure thermal imaging data acquisition covering the target area. After the operator starts the system, the sensors simultaneously acquire infrared thermal images of the base of the ear and the vulva area and transmit them to the data processing center in real time.

[0129] The environmental monitoring module integrates multiple sensors, including temperature, humidity, wind speed, and wind temperature sensors, to collect ambient environmental data in real time. This data is transmitted wirelessly to the central processing unit, providing necessary environmental parameters for subsequent temperature correction and analysis, ensuring the accuracy of the thermal imaging data.

[0130] The core function of the temperature correction module is to compensate for the temperature of the original thermal image to eliminate the influence of environmental factors on the measurement results. This module utilizes a stored body surface temperature difference mapping function and an environmental parameter compensation matrix to calculate and adjust the pixel values ​​of the thermal image. Technicians can adjust the compensation parameters according to the specific conditions of the breeding environment to ensure that the final output thermal image accurately reflects the sow's body temperature status.

[0131] The keypoint localization module uses the HRNet convolutional neural network for image processing to identify key points in the ear root and vulva areas. The system outputs keypoint coordinates through the processed feature map, providing basic data for subsequent feature extraction and fusion, ensuring accurate localization of key areas.

[0132] The feature extraction module analyzes the acquired thermal imaging data in real time using an improved YOLOv8n model to extract temperature features from the ear root and vulva. The model utilizes a MobileNetV3 backbone network and a bidirectional feature pyramid structure to process multi-scale features simultaneously, ensuring high accuracy and real-time performance of the extracted temperature information.

[0133] The time-series analysis module uses a bidirectional long short-term memory network (Bi-LSTM) and a fully connected neural network to perform in-depth analysis of the extracted features. The module first analyzes the time-series data to extract temperature change trends and activity data, and then outputs the estrus status of the sows through a classifier.

[0134] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from the spirit and scope of this invention.

Claims

1. A method for detecting estrus in sows based on thermal imaging and deep learning, characterized in that, include: S1: Collect infrared thermal image sequences of the ear root region and vulva region of the target sow to generate dual-region time-series thermal imaging data; S2: Perform temperature correction and key point localization on dual-region time-series thermal imaging data, and generate regional temperature feature maps by improving the YOLOv8n model. S3: Input the regional temperature feature map into the bidirectional long short-term memory network, extract the temperature change trend, generate the estrus feature vector, and determine the estrus status and stage of the sow.

2. The method for detecting estrus in sows based on thermal imaging and deep learning according to claim 1, characterized in that, Step S1 includes: S11: Set the infrared thermal imager acquisition parameters, continuously acquire images at fixed intervals in the ear root area of ​​the target sow, and generate an ear root infrared thermal image sequence; S12: Keeping the acquisition parameters unchanged, continuously acquire images of the vulva area of ​​the target sow at the same interval to generate a vulva infrared thermogram sequence; S13: Align and combine the ear root infrared thermal image sequence and the vulva infrared thermal image sequence according to the timestamp to form dual-region time-series thermal imaging data.

3. The method for detecting estrus in sows based on thermal imaging and deep learning according to claim 1, characterized in that, Step S1 also includes: S14: Insert a thermometer into the rectum of the target sow to measure the temperature and record it as a baseline reference value for body temperature. S15: Collect the activity frequency and intensity of the target sow using a wearable accelerometer to generate time-series activity data; S16: Record the temperature, humidity, wind speed and wind temperature values ​​of the detected environment through environmental sensors to generate an environmental parameter set.

4. The method for detecting estrus in sows based on thermal imaging and deep learning according to claim 1, characterized in that, Step S2 includes: S21: Combine rectal temperature with environmental parameters to establish a temperature compensation model, perform pixel-level temperature correction on the infrared thermogram, and generate a corrected thermogram; S22: The HRNet lightweight key point detection network is used to locate temperature-sensitive points in the ear root and vulva region, and to extract a fixed-size target region image. S23: Input the target region image from the corrected heat map into the improved YOLOv8n model to extract multi-level regional temperature feature maps.

5. The method for detecting estrus in sows based on thermal imaging and deep learning according to claim 4, characterized in that, Step S21 includes: S211: Calculate the difference between rectal temperature and infrared thermogram surface temperature, establish a body surface temperature difference mapping function, and generate a preliminary correction factor; S212: Analyze the influence weight of environmental parameters on the temperature distribution of infrared thermograms, construct an environmental factor compensation function, and generate environmental correction factors; S213: Combine the body surface temperature difference mapping function with the environmental factor compensation function to construct a correction model and perform temperature numerical correction on the original infrared thermogram.

6. The method for detecting estrus in sows based on thermal imaging and deep learning according to claim 1, characterized in that, The improved structure of the YOLOv8n model includes: S61: Replace the original YOLOv8n backbone network with the MobileNetV3 network designed for thermal imaging characteristics, and adopt a depthwise separable convolutional structure; S62: Add a BiFPN bidirectional feature pyramid structure to the neck network to build bidirectional connections between feature levels and form a multi-scale feature fusion network; S63: The original path aggregation network is replaced by a weighted bidirectional feature pyramid network. By learning the weight coefficients, the heat features of the ear root and vulva are fused to generate a comprehensive feature representation.

7. The method for detecting estrus in sows based on thermal imaging and deep learning according to claim 1, characterized in that, Step S3 includes: S31: Organize the temperature feature maps of consecutive frames in chronological order to construct a temporal feature sequence with a fixed window size; S32: Process time-series feature sequences through a bidirectional long short-term memory network, extract temperature change trend features, and fuse them with activity data to generate a multimodal estrus feature vector; S33: Input the estrus feature vector into a fully connected neural network classifier, calculate the probability value of each estrus stage, and determine the estrus state and stage.

8. The method for detecting estrus in sows based on thermal imaging and deep learning according to claim 7, characterized in that, Step S32 includes: S321: Construct a long short-term memory unit network structure that includes a forget gate, an input gate, and an output gate, and establish a bidirectional long short-term memory network in both forward and backward directions; S322: Input the time-series feature sequence into a bidirectional long short-term memory network, extract short-term and long-term temperature change trend features through a gating mechanism, and generate a time-series feature representation; S323: An attention mechanism is used to fuse temporal feature representations with activity data at the feature level, generating an estrus feature vector containing spatiotemporal multimodal information.

9. The method for detecting estrus in sows based on thermal imaging and deep learning according to claim 7, characterized in that, Step S33 includes: S331: Construct a fully connected neural network with two hidden layers to process estrus feature vectors and calculate the probability values ​​of four states: non-estrus, proestrus, peak estrus, and mesestrus. S332: Normalize the probability values ​​of the four states using the Softmax function, select the state with the highest probability as the result of estrus judgment, and record the probability value as the confidence level of the judgment; S333: Uses gradient-weighted class activation mapping technology to generate a heat map of regional temperature changes, marks the temperature-sensitive areas on which the judgment is based, and provides a visual interpretation of the results.

10. An apparatus for detecting estrus in sows based on thermal imaging and deep learning as described in any one of claims 1-9, characterized in that, include: The dual-area thermal imaging acquisition module consists of a first infrared thermal imaging sensor and a second infrared thermal imaging sensor. The first sensor is installed directly in front of the sow's ear root area, and the second sensor is installed diagonally above the sow's vulva area. The environmental monitoring module includes a temperature sensor, a humidity sensor, a wind speed sensor, and a wind temperature sensor that are connected to the data acquisition terminal. The temperature correction module includes a calculation unit for the storage surface temperature difference mapping function and an evaluation unit for the storage environment parameter compensation matrix. The key point localization module includes an image processor running an HRNet convolutional neural network, which outputs the coordinates of key points in the ear root and vulva areas. The feature extraction module includes a computing device for deploying the improved YOLOv8n model, which features a MobileNetV3 backbone network and a bidirectional feature pyramid structure. The time series analysis module includes a bidirectional long short-term memory network processor and a fully connected neural network classifier.

Images