Sow oestrus detection method based on improved YOLOv5s

Abstract

The invention discloses a sow oestrus detection method based on improved YOLOv5s, and the method comprises the steps: S1, obtaining time sequence pudendum image sequence data and auxiliary sensing data of a target sow, and generating multi-modal input data; s2, inputting the multi-modal input data into an improved YOLOv5s target detection model, extracting spatial features, and then performing sequential network processing to generate an estrus feature vector; and S3, applying a classifier and a time sequence smoothing algorithm to the oestrus feature vector, and outputting an oestrus state detection result of the target sow. By improving a YOLOv5s target detection algorithm, the accuracy of sow oestrus detection is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of estrus detection technology in animal husbandry, and in particular to a method for detecting estrus in sows based on an improved YOLOv5s. Background Technology

[0002] Sow reproductive performance is a core aspect of pig production, directly impacting the economic benefits and sustainable development of pig farms. Accurate identification of the sow's estrus cycle is crucial for seizing the optimal mating time. Missing this optimal mating time leads to decreased conception rates, increased non-pregnant days, and fewer litters per year, resulting in significant economic losses for pig farms. Currently, large-scale pig farms primarily rely on manual observation for estrus detection. Farm workers determine estrus status by observing vulvar swelling, testing back pressure responses, and observing behavioral changes. While this manual observation method is intuitive, it requires a large number of experienced technicians, and the process is time-consuming and labor-intensive, especially in large-scale farms where timeliness and accuracy are difficult to guarantee.

[0003] However, existing estrus detection methods have significant drawbacks. First, manual observation is heavily influenced by subjective factors, and the judgments of different technicians can vary considerably, especially for sows in estrus or those with subtle signs, leading to a high misjudgment rate. Second, relying solely on visual observation cannot capture the gradual changes during estrus, easily missing the optimal mating time. Third, conventional computer vision methods are based solely on single-frame static image analysis, ignoring temporal state changes and making it difficult to accurately distinguish between similar states. Fourth, existing algorithms often depend on a single data source, failing to comprehensively utilize physiological indicators related to estrus, such as temperature changes and activity levels, resulting in insufficient detection reliability. Finally, traditional target detection algorithms are poorly adaptable to complex interference factors in the breeding environment; common problems such as changes in lighting and obstruction by dirt significantly reduce detection accuracy.

[0004] These shortcomings severely limit the practical application of automatic estrus detection technology in sows, and there is an urgent need for a sow estrus detection method based on improved YOLOv5s to solve these problems. Summary of the Invention

[0005] This invention provides a method for detecting estrus in sows based on an improved YOLOv5s, in order to solve the aforementioned problems in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for detecting estrus in sows based on an improved YOLOv5s method includes:

[0008] S1: Acquire time-series vulvar image sequence data and auxiliary sensor data of the target sow, and generate multimodal input data;

[0009] S2: Input the multimodal input data into the improved YOLOv5s target detection model, extract spatial features, and then process them through a temporal network to generate an estrus feature vector;

[0010] S3: Apply a classifier and a temporal smoothing algorithm to the estrus feature vector to output the estrus status detection results of the target sow.

[0011] Step S1 includes:

[0012] S11: Collect vulva image sequences of the target sow over a continuous time period to obtain raw image sequence data;

[0013] S12: Synchronously collect temperature data and activity data of the vulva area of ​​the target sow to obtain environmental adaptive auxiliary sensing data;

[0014] S13: Align and integrate the original image sequence data and auxiliary sensor data according to timestamps to generate multimodal input data.

[0015] Step S12 includes:

[0016] S121: Install an infrared temperature sensor in the activity area of ​​the target sow to collect temperature change data in the vulva area and obtain temperature time series data;

[0017] S122: Install a wearable activity sensor on the body surface of the target sow to collect data on changes in activity intensity and obtain time-series data on activity.

[0018] S123: Perform time synchronization and numerical normalization processing on temperature time series data and activity time series data to generate standardized auxiliary sensing data.

[0019] Step S2 includes:

[0020] S21: Apply an improved YOLOv5s object detection network to each vulva image to extract image spatial features containing vulva morphology and swelling characteristics;

[0021] S22: Construct a feature fusion neural network to integrate image spatial features with auxiliary sensing data to generate multimodal features containing multi-source information;

[0022] S23: Design a time-dimensional feature extraction network to process multimodal features on time series, extract state change features, and generate a comprehensive estrus feature vector.

[0023] Step S21 includes:

[0024] S211: The input image is cropped, scaled, and its pixel values ​​are normalized to convert it into standardized image data that conforms to the network input specifications;

[0025] S212: Input standardized image data into the backbone network with CA channel attention mechanism, extract and enhance key features of the vulva, and generate attention-weighted feature map;

[0026] S213: The attention-weighted feature map is fed into the CARAFE upsampling module. After content-aware feature reconstruction and upsampling, a high-resolution image spatial feature that retains more detailed information is generated.

[0027] Step S23 includes:

[0028] S231: Arrange the multimodal features of continuous time frames in the acquisition order to construct a sequence data structure, forming a feature sequence that reflects time changes;

[0029] S232: Construct a recurrent neural network containing LSTM memory units to process feature sequences and capture the changes in vaginal state over time, generating time-dimensional encoded features;

[0030] S233: Design a multi-head self-attention mechanism to calculate the importance distribution of time-dimensional encoded features, perform feature aggregation according to weights, and generate an estrus feature vector containing temporal information.

[0031] Step S3 includes:

[0032] S31: Construct a multilayer perceptron classifier to process the estrus feature vector, calculate the estrus probability value, and generate a preliminary estrus status judgment result;

[0033] S32: Set a sliding time window to include historical detection results, apply an exponential weighted average algorithm to smooth the initial estrus state judgment results, eliminate instantaneous fluctuations, and generate a stable corrected estrus state;

[0034] S33: Based on the preset threshold, perform binary classification on the corrected estrus status, determine and output the final estrus status detection result of the target sow.

[0035] Among them, the training methods for improving the YOLOv5s object detection model include:

[0036] S81: Collect temporal vulva images and sensor data of different breeds of sows at different estrus stages from multiple pig house environments to construct raw training data covering diverse conditions;

[0037] S82: The original training data is annotated by experts, the dataset is divided according to pigs and environmental conditions, and targeted data augmentation for farming scenarios is applied to generate a high-quality multimodal training dataset.

[0038] S83: Design a multi-input multi-task learning framework to train an improved YOLOv5s object detection model, and apply a model update strategy to generate a robust estrus detection model that adapts to different environmental conditions.

[0039] The data augmentation process in step S82 includes:

[0040] S821: Apply noise addition, brightness adjustment, angle rotation, region cropping, and position translation transformations with random parameters to the original image sequence to generate a basic augmented dataset;

[0041] S822: Add mud and dirt occlusion layers, uneven lighting gradients, cast shadow effects, and simulated lens blur to the basic augmented dataset to generate interference augmented data that simulates complex environmental conditions.

[0042] S823: Morphological transformations are applied to the vulva area to simulate different degrees of redness and swelling, changes in opening angle, and surface moisture characteristics, generating a morphologically enhanced dataset that reflects the physiological changes of estrus.

[0043] Step S83 includes:

[0044] S831: Construct a multi-branch network structure, including a convolutional branch for processing image data, a fully connected branch for processing sensor data, and a feature interaction module containing an attention gating mechanism, forming an end-to-end multimodal model architecture;

[0045] S832: Design a composite loss function that combines the EIoU loss of object detection, the perceptual loss of vulva texture similarity, and the smoothness consistency loss of temporal prediction in a weighted manner to establish an optimization objective that takes into account multi-dimensional feature learning.

[0046] S833: Employs a phased training strategy, enabling the model to learn basic features from standard environmental data and then transfer learning to specific pigsty environmental conditions, generating a multimodal estrus detection model with environmental adaptability.

[0047] Compared with the prior art, the present invention has the following advantages:

[0048] A method for detecting estrus in sows based on an improved YOLOv5s algorithm includes: S1: acquiring temporal vulvar image sequence data and auxiliary sensor data of the target sow to generate multimodal input data; S2: inputting the multimodal input data into the improved YOLOv5s target detection model, extracting spatial features, and then processing them through a temporal network to generate an estrus feature vector; S3: applying a classifier and a temporal smoothing algorithm to the estrus feature vector to output the estrus status detection result of the target sow. By improving the YOLOv5s target detection algorithm, the accuracy of estrus detection in sows is significantly improved.

[0049] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention.

[0050] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0051] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0052] Figure 1 This is a flowchart of a sow estrus detection method based on an improved YOLOv5s according to an embodiment of the present invention;

[0053] Figure 2 This is a flowchart illustrating the generation of estrus feature vectors in an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of data acquisition in an embodiment of the present invention;

[0055] Figure 4(a) is a schematic diagram of vulva image sample data of sows in estrus in an embodiment of the present invention;

[0056] Figure 4(b) is a schematic diagram of vulvar image sample data of sows not in estrus in an embodiment of the present invention. Detailed Implementation

[0057] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0058] The embodiments of the present invention provide, as follows Figure 1 As shown, a method for detecting estrus in sows based on an improved YOLOv5s includes:

[0059] S1: Acquire time-series vulvar image sequence data and auxiliary sensor data of the target sow, and generate multimodal input data;

[0060] S2: Input the multimodal input data into the improved YOLOv5s target detection model, extract spatial features, and then process them through a temporal network to generate an estrus feature vector;

[0061] S3: Apply a classifier and a temporal smoothing algorithm to the estrus feature vector to output the estrus status detection results of the target sow.

[0062] The working principle and beneficial effects of the above technical solution are as follows: In step S1, the system first acquires a continuous sequence of images of the vulva area of ​​the sow using equipment installed in the pigsty. These images are captured by an Intel RealSense D435 camera at a frame rate of one per second. Simultaneously, an infrared thermometer measures the surface temperature change of the vulva area, and an activity sensor worn on the sow's body records the activity frequency. The acquisition device transmits this heterogeneous data to the processing terminal via wired or wireless network. The terminal matches the image data with the temperature and activity data based on timestamps, forming an input data packet containing multi-dimensional information.

[0063] In step S2, after receiving multimodal input data, the processing terminal first feeds each frame of image into an improved YOLOv5s network. This improved network incorporates a CA channel attention mechanism in its backbone, enabling it to actively highlight the features of the vulvar swelling area. Simultaneously, the original interpolation upsampling is replaced by a CARAFE content-aware feature reconstruction module. This module effectively preserves detailed features by predicting the upsampling kernel and reconstructing the feature map. After image feature extraction, the system merges it with auxiliary sensor data through a feature fusion layer, and then inputs it into a temporal network containing LSTM memory units. This network can capture the temporal changes in the morphology, temperature, and activity patterns of the sow's vulva, ultimately generating a high-dimensional feature vector.

[0064] In step S3, the system inputs the generated feature vector into a binary classifier, which uses a fully connected layer and a softmax function to calculate the estrus probability value. To eliminate the influence of instantaneous fluctuations, the system introduces a temporal smoothing algorithm. This algorithm suppresses the instability of a single detection through weighted averaging and trend analysis of historical detection results. Finally, the system outputs the smoothed result as the sow's estrus status judgment result for reference by farmers.

[0065] In another embodiment, step S1 includes:

[0066] S11: Collect vulva image sequences of the target sow over a continuous time period to obtain raw image sequence data;

[0067] S12: Synchronously collect temperature data and activity data of the vulva area of ​​the target sow to obtain environmental adaptive auxiliary sensing data;

[0068] S13: Align and integrate the original image sequence data and auxiliary sensor data according to timestamps to generate multimodal input data.

[0069] The working principle and beneficial effects of the above technical solution are as follows: In step S11, an image acquisition device with a height of 1.1 meters is installed in the pigsty aisle. The device is equipped with an Intel RealSense D435 depth camera and is positioned approximately 0.3 meters away from the target vulva area. During the acquisition process, the camera continuously records images of the sow's vulva area at a frequency of one frame per second. The resolution of each frame is set to 1280×720 pixels, and the acquisition duration is determined according to the sow's activity cycle, typically 5-10 minutes, forming a continuous image sequence containing 300-600 frames.

[0070] In step S12, while acquiring images, an infrared thermometer installed in the target area records the vulvar surface temperature data. The sensor's sampling frequency is synchronized with image acquisition, recording temperature changes in real time. Simultaneously, an activity sensor pre-installed on the sow's body continuously collects activity intensity data; this sensor quantifies the sow's movement state using triaxial accelerometer data. This auxiliary sensor data is transmitted in real time to the data processing unit via a wireless transmission module, forming an environmental adaptation dataset reflecting the sow's physiological state.

[0071] In step S13, after receiving data from different sources, the data processing unit first checks the timestamps of each data stream and pairs image frames from the same time point (with an error of no more than 50 milliseconds) with their corresponding temperature and activity data. If a certain type of data is missing at a certain time point, the system supplements the missing value using a linear interpolation method. After completing the time alignment, the system organizes the three types of data into a unified format multimodal data packet, with each data packet containing one image frame and its corresponding temperature and activity values.

[0072] like Figure 3 As shown, the data used in this embodiment was collected from a large-scale pig farm in City A from April to May 2024. An image acquisition device was installed in the pigsty aisle. The device consisted of an Intel RealSense D435 depth camera, a support frame, and a mobile cart. The camera was 1.1m above the ground and approximately 0.3m from the vulva, capturing one image per second. The data collected included the vulvas of 30 estrus sows and 30 non-estrus sows of the same breed, housed in gestation crates. Estrus status was determined by experienced breeders through visual inspection combined with farrowing records. A total of 700 valid images were collected, including 350 images of estrus and 350 images of non-estrus sows. Each image was 1280×720 pixels. Key morphological characteristics: the vulva of estrus sows was red and full; the vulva of non-estrus sows was pink and shriveled. A comparison of the vulva images of estrus and non-estrus sows is shown in Figures 4(a) and 4(b).

[0073] In this embodiment, the LabelImg annotation tool is used to annotate the collected images. The annotation types include "estrus" and "non-estrus," and the annotation format is VOC. After selecting the target to be annotated and naming it, the annotation information is automatically generated into an XML file. The annotated images are stored in the JPEGImages folder, and the labels are stored in the Annotations folder. A VOCdevkit folder is then created to store both the JPEGImages and Annotations folders.

[0074] After labeling the dataset, it needs to be partitioned into a training set, a validation set, and a test set in an 8:1:1 ratio. The training set is used to train the model, allowing it to learn the relationship between the features of the data and the labels. The validation set is used to evaluate the model's ability to generalize to new data during training to avoid overfitting. The test set is used to evaluate the model's performance on completely unseen data to simulate the model's performance in the real world.

[0075] In object detection model training, the quality of the dataset has a significant impact on the training results. Sufficient samples are beneficial for training high-quality models and enhancing generalization ability. However, when faced with insufficient or poor-quality samples, data augmentation becomes crucial. Simple operations such as translation, scaling, and color transformation, while not changing the image category, can enrich the training set and effectively improve model training performance. These augmented samples can help the model better learn and understand image features, improving the model's generalization ability and accuracy.

[0076] In another embodiment, step S12 includes:

[0077] S121: Install an infrared temperature sensor in the activity area of ​​the target sow to collect temperature change data in the vulva area and obtain temperature time series data;

[0078] S122: Install a wearable activity sensor on the body surface of the target sow to collect data on changes in activity intensity and obtain time-series data on activity.

[0079] S123: Perform time synchronization and numerical normalization processing on temperature time series data and activity time series data to generate standardized auxiliary sensing data.

[0080] The working principle and beneficial effects of the above technical solution are as follows: In step S121, the worker fixes the infrared temperature sensor above the pigpen's restraint fence and adjusts the sensor angle to accurately align it with the vulva area of ​​the target sow. This sensor uses an MLX90614 non-contact infrared temperature sensor element with a temperature range of 0-100℃ and an accuracy of ±0.5℃. The sensor sampling frequency is set to 1Hz, synchronized with image acquisition. During temperature measurement, the sensor collects the vulva surface temperature in real time and transmits it to the data acquisition unit via an RS485 interface, forming temperature time-series data containing temperature values ​​and acquisition timestamps.

[0081] In step S122, technicians install a pre-packaged wearable activity sensor on the back of the sow's ear. This sensor incorporates an MPU6050 triaxial accelerometer and a Bluetooth transmission module. Weighing less than 15 grams, it does not interfere with the sow's normal activity. The sensor is secured with adhesive medical tape, and the sampling frequency is set to 10Hz to monitor the frequency and amplitude of the sow's head movements. The motion data captured by the accelerometer is transmitted in real-time to the receiving terminal via Bluetooth. The system records the raw acceleration values ​​and their composite values, forming a time-series data stream reflecting the intensity of the sow's activity.

[0082] In step S123, the data processing unit first performs noise reduction on the temperature time-series data, using a sliding window midpoint filtering method (window width of 5 sample points) to eliminate outliers. Then, it extracts features from the activity time-series data, calculating the standard deviation of acceleration per second as an indicator of activity intensity. During the standardization phase, the system maps the temperature and activity data to the [-1, 1] interval according to the formula x' = (x - μ) / σ, where μ and σ are the pre-calculated mean and standard deviation of each indicator, respectively. The standardized data is then resampled to a 1Hz frequency on the time axis to ensure alignment with the time points of the image sequence, ultimately forming a standardized auxiliary sensing data stream.

[0083] In another embodiment, such as Figure 2 As shown, step S2 includes:

[0084] S21: Apply an improved YOLOv5s object detection network to each vulva image to extract image spatial features containing vulva morphology and swelling characteristics;

[0085] S22: Construct a feature fusion neural network to integrate image spatial features with auxiliary sensing data to generate multimodal features containing multi-source information;

[0086] S23: Design a time-dimensional feature extraction network to process multimodal features on time series, extract state change features, and generate a comprehensive estrus feature vector.

[0087] The working principle and beneficial effects of the above technical solution are as follows: In step S21, the processing unit sequentially retrieves each vulva image from the data queue and inputs it into the improved YOLOv5s detection network. The network first extracts multi-level features through the CSPDarknet53 backbone network with SiLU activation function. Unlike the original YOLOv5s, the improved model adds a CA attention layer after the C3 module. This layer highlights the morphological and color change features of the vulva region by calculating the attention distribution of the feature map in the height and width directions. The network extracts features for the two key estrus indicators, vulvar redness and swelling degree and opening degree. The resulting feature map is then processed by spatial pyramid pooling to form a feature vector rich in target morphological information.

[0088] In step S22, the system employs a specially designed multi-path feature fusion network to integrate image spatial features and auxiliary sensor data. This network consists of three parallel branches: the main branch processes image features, auxiliary branch one processes temperature data, and auxiliary branch two processes activity data. Temperature data is first mapped to a high-dimensional feature space through two fully connected layers (16 and 32 nodes), while activity data undergoes a one-dimensional convolutional layer (kernel size 3, stride 1) to extract temporal patterns. The three feature paths are fused using an attention-weighted mechanism. The system calculates the importance weights of each branch's features and performs a weighted summation based on the weight ratios to form a comprehensive feature representation containing multi-source information.

[0089] In step S23, the system constructs a temporal feature extraction network to analyze the changing patterns of multi-frame multimodal features. This network employs a bidirectional LSTM structure, with each LSTM unit containing 64 hidden nodes, and uses a gating mechanism to selectively retain long short-term memory information. The network input is a sequence of multimodal features spanning eight consecutive frames. Through temporal modeling in both forward and backward directions, it captures the temporal evolution features of vulvar morphology, temperature, and activity patterns. The LSTM layer output is weighted by a self-attention module, focusing on key changes in the temporal features. The final network output is a 128-dimensional feature vector, which encodes the dynamic changes in the sow's vulvar state, providing a comprehensive basis for estrus status assessment.

[0090] In another embodiment, step S21 includes:

[0091] S211: The input image is cropped, scaled, and its pixel values ​​are normalized to convert it into standardized image data that conforms to the network input specifications;

[0092] S212: Input standardized image data into the backbone network with CA channel attention mechanism, extract and enhance key features of the vulva, and generate attention-weighted feature map;

[0093] S213: The attention-weighted feature map is fed into the CARAFE upsampling module. After content-aware feature reconstruction and upsampling, a high-resolution image spatial feature that retains more detailed information is generated.

[0094] The working principle and beneficial effects of the above technical solution are as follows: In step S211, the system first preprocesses the input image to adapt to the network input requirements. Preprocessing includes three main operations: cropping, scaling, and normalization. The cropping operation is based on a preset Region of Interest (ROI) template, extracting a 640×480 pixel area from the center of the image; the scaling operation uniformly adjusts the cropped image to a size of 416×416 pixels, using bicubic interpolation to ensure image quality; the normalization process divides the pixel values ​​by 255 to map them to the [0,1] interval, and then subtracts the mean (0.485, 0.456, 0.406) of the pre-training dataset from each RGB channel, and then divides by the standard deviation (0.229, 0.224, 0.225). The preprocessed image is then input into the network as a tensor with a size of [B, 3, 416, 416], where B is the batch size.

[0095] In step S212, a backbone network with a CA attention mechanism is added to the standardized image input for feature extraction. This network first extracts multi-scale features through five convolutional blocks, each containing multiple convolutional layers, batch normalization layers, and activation functions. At the output of each convolutional block, a CA attention module is inserted to enhance feature representation capabilities. The CA module operates as follows: first, the input feature map (C×H×W) is globally averaged in both the width and height directions to obtain feature vectors C×1×H and C×1×W; the two vectors are concatenated and transposed to C×1×(H+W), then compressed through a 2×2 convolution to C / r×1×(H+W); subsequently, the height and width features are separated, and each is restored to its channel count through a 1×1 convolution before being processed by a sigmoid function to generate attention weights; these weights are multiplied element-wise with the original feature map to enhance key areas of the vulva. Through this mechanism, the network can highlight the features of the vulvar swelling area, improving sensitivity to estrus states.

[0096] In step S213, the attention-weighted feature map enters the CARAFE upsampling module for feature reconstruction and upsampling. The CARAFE module consists of two parts: kernel prediction and content-aware reconstruction. In the kernel prediction stage, for a feature map of size H×W×C, the number of channels is first compressed to Cm (Cm=64) through a 1×1 convolution, and then a reconstruction kernel is generated through a 3×3 convolution, with an output size of H×W×(σ 2 ×K 2 up), where σ = 2 is the upsampling factor and Kup = 5 is the upsampling kernel size. The generated recombined kernel is rearranged into σH × σW × K after a Pixel Shuffle operation.2 The system upsamples the image shape and normalizes it using the Softmax function. During the content-aware reconstruction stage, for each location, the system extracts a local region feature of size Kup×Kup from the original feature map, and weights this feature with the corresponding reconstruction kernel to generate an upsampled feature value. Compared to traditional interpolation methods, CARAFE can adaptively generate upsampling weights based on image content, preserving more morphological and texture details, and outputting high-resolution (2H×2W×C) image spatial features.

[0097] In another embodiment, step S23 includes:

[0098] S231: Arrange the multimodal features of continuous time frames in the acquisition order to construct a sequence data structure, forming a feature sequence that reflects time changes;

[0099] S232: Construct a recurrent neural network containing LSTM memory units to process feature sequences and capture the changes in vaginal state over time, generating time-dimensional encoded features;

[0100] S233: Design a multi-head self-attention mechanism to calculate the importance distribution of time-dimensional encoded features, perform feature aggregation according to weights, and generate an estrus feature vector containing temporal information.

[0101] The working principle and beneficial effects of the above technical solution are as follows: In step S231, the processor receives multimodal feature data from the feature fusion network and arranges the features with time as the main axis. During operation, the program first assigns a time stamp to each frame of feature data, and then arranges the feature data according to the time stamp order. The system adopts a sliding window mechanism to extract 8 consecutive frames of features as basic analysis units, forming a feature sequence matrix of shape [8, F], where F represents the feature dimension. The window sliding step size is 1 each time to ensure that the feature sequence has sufficient temporal continuity. This operation enables the system to capture the sequence information of the vaginal state changing over time, providing a data foundation for subsequent time series analysis.

[0102] In step S232, the developers construct a specific temporal network structure to process the aforementioned feature sequences. This network uses a bidirectional LSTM as its core, and its construction process is as follows: First, the input layer is defined to receive feature sequences of shape [8, F]. Then, an LSTM layer containing 64 hidden units is designed. Each LSTM unit contains three control units: an input gate, a forget gate, and an output gate. Through a gating mechanism, temporal information is selectively remembered or forgotten. In implementation, the forget gate uses the sigmoid function to process the connection features between the previous hidden state h(t-1) and the current input x(t), controlling the degree of retention of the previous cell state. The input gate and candidate memory units jointly determine the amount of current information updated. Finally, the output gate controls the output ratio of the updated cell state. This structure enables the network to capture the gradual changes in vulvar morphology and color over time, effectively identifying the trend of redness and swelling during estrus.

[0103] In step S233, the temporal features output by the LSTM are weighted using a multi-head self-attention mechanism. This mechanism first transforms the input features into three representation spaces: query, key, and value. Four attention heads are then used to compute the attention weights in parallel. Each head calculates the similarity between the query and key using a scaled dot product formula, obtains the weight coefficients through softmax normalization, and multiplies the weights by the value vector to obtain the weighted features. In implementation, the program first maps the features to a 512-dimensional space, performs multi-head self-attention computation, and then reduces the dimensionality to a 128-dimensional vector using a feedforward network. This step effectively highlights the key temporal signals for estrus detection, suppresses noise and interference, and forms a feature vector that integrates information on vulvar morphological changes, temperature fluctuations, and activity patterns.

[0104] In another embodiment, step S3 includes:

[0105] S31: Construct a multilayer perceptron classifier to process the estrus feature vector, calculate the estrus probability value, and generate a preliminary estrus status judgment result;

[0106] S32: Set a sliding time window to include historical detection results, apply an exponential weighted average algorithm to smooth the initial estrus state judgment results, eliminate instantaneous fluctuations, and generate a stable corrected estrus state;

[0107] S33: Based on the preset threshold, perform binary classification on the corrected estrus status, determine and output the final estrus status detection result of the target sow.

[0108] The working principle and beneficial effects of the above technical solution are as follows: In step S31, the system uses a multilayer perceptron to classify the estrus feature vector. This classifier consists of a three-layer fully connected network: the input layer receives a 128-dimensional feature vector; the hidden layer contains 64 neurons, which use the ReLU activation function to extract nonlinear features; the output layer contains 2 neurons, corresponding to the "estrus" and "non-estrus" states respectively. During the calculation process, the input features are first linearly transformed and activated by the weight matrix W1 (128×64) and the bias term b1 to form the hidden layer representation; the hidden layer representation is then linearly transformed by the weight matrix W2 (64×2) and the bias term b2, and converted into a probability distribution by the Softmax function. The system extracts the class with the higher probability value as the preliminary judgment result and records the probability difference between the two classes as a confidence index.

[0109] In step S32, the program sets up a sliding window containing the 10 most recent detection results to perform time-series smoothing on the preliminary judgment results. In implementation, the system first assigns a weight to each historical result within the window, with the weight coefficient decreasing exponentially by ω. i =α(1-α) (n-i) The calculation is performed, where α is set to 0.3, n is the window size, and i is the result index (the closer the result, the greater its weight). Smoothing is executed in two stages: state voting and confidence correction. In the state voting stage, the system calculates the weighted voting values ​​for the estrus and non-estrus states within the window. In the confidence correction stage, the system comprehensively considers the consistency between the current confidence level and historical states, adjusting the confidence level of the final judgment. This step effectively eliminates single-frame misjudgments and temporary fluctuations, improving the stability of the judgment.

[0110] In step S33, the system determines the final estrus status based on the corrected state and confidence level. During processing, the program compares the corrected estrus probability value with a preset threshold (0.65). A value higher than the threshold is considered estrus; otherwise, it is considered not in estrus. If the corrected probability value falls within the uncertainty range (0.55–0.65), the system will invoke stricter rules for judgment, including analyzing temperature change trends and abnormal activity indices. The final result is displayed to the operator through the software interface and simultaneously recorded in the database for subsequent feeding management. Through dynamic thresholds and multiple confirmation mechanisms, this step balances detection sensitivity and specificity, reducing the false judgment rate in the breeding field.

[0111] In another embodiment, the improved training method for the YOLOv5s object detection model includes:

[0112] S81: Collect temporal vulva images and sensor data of different breeds of sows at different estrus stages from multiple pig house environments to construct raw training data covering diverse conditions;

[0113] S82: The original training data is annotated by experts, the dataset is divided according to pigs and environmental conditions, and targeted data augmentation for farming scenarios is applied to generate a high-quality multimodal training dataset.

[0114] S83: Design a multi-input multi-task learning framework to train an improved YOLOv5s object detection model, and apply a model update strategy to generate a robust estrus detection model that adapts to different environmental conditions.

[0115] The working principle and beneficial effects of the above technical solution are as follows: In step S81, researchers organized a multi-site collaborative data collection scheme to collect sample data from farms in three regions: North China, Central China, and Northeast China. The collection process first screened sows of different breeds (Large White, Landrace, Duroc, and their hybrids) through their breeding records, covering different estrus stages (proestrus, peak estrus, mesestrus, and anestrus). To ensure data quality, each site was equipped with standardized image acquisition equipment, including an Intel RealSense D435 camera with fixed parameters and a uniformly calibrated infrared thermometer. For each target sow, a 5-minute vulvar image sequence (1fps), body surface temperature, and activity level data were continuously collected, while environmental parameters (temperature, humidity, and light intensity) were recorded. Data from each site was uploaded to a central server using dedicated software. After format standardization and quality checks, an initial database was formed, containing multimodal time-series data from 120 sows.

[0116] In step S82, the data processing team performed multi-stage data processing. First, three breeding experts with over 10 years of experience were invited to use a customized annotation tool to assess the condition of each sow, including whether they were in estrus, the degree of redness and swelling (level four), and morphological descriptions. Samples with differing opinions among the experts were collectively discussed to determine the final labels. Subsequently, stratified sampling was conducted according to pig breed and environmental conditions to ensure that the training, validation, and test sets were evenly distributed and independent. In the data augmentation phase, in addition to conventional transformations, a dedicated augmentation module was developed specifically for the piggery environment to simulate various interference factors such as uneven lighting and obstruction by dirt. The processed dataset expanded to approximately 6500 multimodal sequence records, effectively alleviating the problem of insufficient samples.

[0117] In step S83, the algorithm team designed and implemented a learning framework suitable for multimodal inputs. This framework comprises three main branches: an image branch based on improved YOLOv5s for processing visual features; a temperature branch using a one-dimensional convolutional network to analyze temperature change curves; and an activity level branch employing a recurrent neural network to extract activity patterns. These three branches integrate features through a dynamic weighted fusion mechanism, with the output layer simultaneously performing estrus detection and redness assessment. Training employs a three-stage strategy: the first stage pre-trains each branch network separately; the second stage freezes the backbone, training only the fusion module; and the third stage applies a small learning rate to fine-tune each network. To enhance model adaptability, the development team implemented a model update mechanism, allowing the system to continuously collect feedback data from new environments after deployment and periodically adjust model parameters, ensuring stable detection performance in different pigsty environments.

[0118] In another embodiment, the data augmentation process in step S82 includes:

[0119] S821: Apply noise addition, brightness adjustment, angle rotation, region cropping, and position translation transformations with random parameters to the original image sequence to generate a basic augmented dataset;

[0120] S822: Add mud and dirt occlusion layers, uneven lighting gradients, cast shadow effects, and simulated lens blur to the basic augmented dataset to generate interference augmented data that simulates complex environmental conditions.

[0121] S823: Morphological transformations are applied to the vulva area to simulate different degrees of redness and swelling, changes in opening angle, and surface moisture characteristics, generating a morphologically enhanced dataset that reflects the physiological changes of estrus.

[0122] The working principle and beneficial effects of the above technical solution are as follows: In step S821, the data augmentation team first performs standard enhancement transformation on the original image sequence. During the operation, noise is added using two methods: Gaussian noise (mean 0, standard deviation 0.01-0.05, randomly set) and salt-and-pepper noise (density 0.001-0.01); brightness adjustment changes the overall brightness of the image through multiplication coefficients (range 0.7-1.3); angle rotation is randomly selected within the range of [-15°, 15°]; region cropping retains 80%-95% of the original image's central region and adjusts it back to its original size; position translation is performed horizontally and vertically within ±10% of the original image size. A specially designed program ensures that the above transformation parameters maintain a consistent trend within the same sequence, avoiding abrupt changes that produce unrealistic temporal characteristics. Each original sequence generates 4-6 variant sequences after enhancement, expanding the diversity of the training samples.

[0123] In step S822, the developers designed a simulated interference processing workflow specifically for the unique environment of the farm. The mud occlusion effect was achieved by synthesizing a semi-transparent mud patch overlay. Each mud patch used brown (RGB:103,67,45) as the base color, had an irregular shape, and a transparency of 30%–70%, covering 5%–15% of the image area. Uneven lighting was simulated using a gradient lighting mask, making one side of the image brighter than the other, with the degree of lighting difference randomly set within the range of 10%–30%. The cast shadow effect was achieved using a linear or radial gradient mask of black (60%–85% transparency), covering the edges or corners of the image. Local blurring was achieved using a Gaussian blur with a radius of 25 pixels, randomly applied to 10%–20% of the image area to simulate lens blemishes or moisture effects. These interference processes were smoothly transitioned between consecutive frames, maintaining temporal consistency.

[0124] In step S823, image processing experts perform specific morphological transformations on the vulva area to simulate the physiological changes during estrus. First, the vulva area is precisely located using a pre-trained segmentation network. Then, three types of morphological transformations are applied: changes in redness and swelling are achieved through color mapping, adjusting the saturation (±20%) and brightness (±15%) of the red channel in the HSV color space to simulate different levels of congestion; changes in opening angle are achieved through deformable mesh operations, maintaining the natural shape of the vulva while adjusting the degree of opening (±15%); and surface wetness features are simulated by adjusting specular reflection and local contrast enhancement to increase the visual effect of the wet area. These transformations exhibit gradual characteristics over time, simulating the physiological changes during real estrus and significantly improving the model's ability to identify edge cases.

[0125] In another embodiment, step S83 includes:

[0126] S831: Construct a multi-branch network structure, including a convolutional branch for processing image data, a fully connected branch for processing sensor data, and a feature interaction module containing an attention gating mechanism, forming an end-to-end multimodal model architecture;

[0127] S832: Design a composite loss function that combines the EIoU loss of object detection, the perceptual loss of vulva texture similarity, and the smoothness consistency loss of temporal prediction in a weighted manner to establish an optimization objective that takes into account multi-dimensional feature learning.

[0128] S833: Employs a phased training strategy, enabling the model to learn basic features from standard environmental data and then transfer learning to specific pigsty environmental conditions, generating a multimodal estrus detection model with environmental adaptability.

[0129] The working principle and beneficial effects of the above technical solution are as follows: In step S831, the network architect designed a network structure specifically for multi-source data processing. The image branch uses an improved YOLOv5s backbone network, with input consisting of eight consecutive frames of 416×416 pixel vulva images. After feature extraction through five convolutional blocks and self-attention enhancement, a 1024-dimensional feature vector is output. The sensor branch uses a two-layer fully connected network to process temperature and activity time-series data, outputting 128-dimensional feature representations respectively. Subsequently, a feature interaction module incorporating an attention gating mechanism is designed. This module enhances the complementary utilization of multimodal information by weighted fusion of image and sensor branch features. Finally, the fused features are input into the multi-task head to achieve joint prediction of estrus state detection and related physiological indicators, completing the end-to-end multimodal model construction.

[0130] In step S832, a composite loss function is used during training to accommodate multi-dimensional learning objectives. Specifically, the EIoU loss from object detection is used to optimize localization accuracy, the vulva texture similarity perception loss is used to improve the ability to capture detailed features, and the temporal prediction smoothness consistency loss is used to ensure the smooth changes in the estrus state time series. The total loss function is formed by linearly combining the various loss terms according to their weight coefficients, and the weights are determined through grid search optimization. During training, the loss function gradually guides the model to achieve a balance between spatial localization and temporal coherence, ensuring that the estrus detection results are both accurate and stable.

[0131] In step S833, a phased training strategy is employed to enhance the model's generalization ability. In the first phase, the model is trained on a basic dataset collected in a standard environment, learning fundamental visual and sensor features. In the second phase, some network layers are frozen, and only the feature fusion and output layers are fine-tuned to adapt to the diversity of specific pigsty environments. In the third phase, incremental fine-tuning is performed using a small number of representative new environmental samples to prevent overfitting and improve robustness. This strategy enables the model to adaptively adjust under different environmental conditions, meeting the application needs of the changing environments in actual pig farms.

[0132] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from the spirit and scope of this invention.

Claims

1. A method for detecting estrus in sows based on an improved YOLOv5s, characterized in that, include: S1: Acquire time-series vulvar image sequence data and auxiliary sensor data of the target sow, and generate multimodal input data; S2: Input the multimodal input data into the improved YOLOv5s target detection model, extract spatial features, and then process them through a temporal network to generate an estrus feature vector; S3: Apply a classifier and a temporal smoothing algorithm to the estrus feature vector to output the estrus status detection results of the target sow.

2. The method for detecting estrus in sows based on improved YOLOv5s according to claim 1, characterized in that, Step S1 includes: S11: Collect vulva image sequences of the target sow over a continuous time period to obtain raw image sequence data; S12: Synchronously collect temperature data and activity data of the vulva area of ​​the target sow to obtain environmental adaptive auxiliary sensing data; S13: Align and integrate the original image sequence data and auxiliary sensor data according to timestamps to generate multimodal input data.

3. The method for detecting estrus in sows based on the improved YOLOv5s according to claim 2, characterized in that, Step S12 includes: S121: Install an infrared temperature sensor in the activity area of ​​the target sow to collect temperature change data in the vulva area and obtain temperature time series data; S122: Install a wearable activity sensor on the body surface of the target sow to collect data on changes in activity intensity and obtain time-series data on activity. S123: Perform time synchronization and numerical normalization processing on temperature time series data and activity time series data to generate standardized auxiliary sensing data.

4. The method for detecting estrus in sows based on improved YOLOv5s according to claim 1, characterized in that, Step S2 includes: S21: Apply an improved YOLOv5s object detection network to each vulva image to extract image spatial features containing vulva morphology and swelling characteristics; S22: Construct a feature fusion neural network to integrate image spatial features with auxiliary sensing data to generate multimodal features containing multi-source information; S23: Design a time-dimensional feature extraction network to process multimodal features on time series, extract state change features, and generate a comprehensive estrus feature vector.

5. The method for detecting estrus in sows based on the improved YOLOv5s according to claim 4, characterized in that, Step S21 includes: S211: The input image is cropped, scaled, and its pixel values ​​are normalized to convert it into standardized image data that conforms to the network input specifications; S212: Input standardized image data into the backbone network with CA channel attention mechanism, extract and enhance key features of the vulva, and generate attention-weighted feature map; S213: The attention-weighted feature map is fed into the CARAFE upsampling module. After content-aware feature reconstruction and upsampling, a high-resolution image spatial feature that retains more detailed information is generated.

6. The method for detecting estrus in sows based on the improved YOLOv5s according to claim 4, characterized in that, Step S23 includes: S231: Arrange the multimodal features of continuous time frames in the acquisition order to construct a sequence data structure, forming a feature sequence that reflects time changes; S232: Construct a recurrent neural network containing LSTM memory units to process feature sequences and capture the changes in vaginal state over time, generating time-dimensional encoded features; S233: Design a multi-head self-attention mechanism to calculate the importance distribution of time-dimensional encoded features, perform feature aggregation according to weights, and generate an estrus feature vector containing temporal information.

7. The method for detecting estrus in sows based on improved YOLOv5s according to claim 1, characterized in that, Step S3 includes: S31: Construct a multilayer perceptron classifier to process the estrus feature vector, calculate the estrus probability value, and generate a preliminary estrus status judgment result; S32: Set a sliding time window to include historical detection results, apply an exponential weighted average algorithm to smooth the initial estrus state judgment results, eliminate instantaneous fluctuations, and generate a stable corrected estrus state; S33: Based on the preset threshold, perform binary classification on the corrected estrus status, determine and output the final estrus status detection result of the target sow.

8. The method for detecting estrus in sows based on improved YOLOv5s according to claim 1, characterized in that, Improved training methods for the YOLOv5s object detection model include: S81: Collect temporal vulva images and sensor data of different breeds of sows at different estrus stages from multiple pig house environments to construct raw training data covering diverse conditions; S82: The original training data is annotated by experts, the dataset is divided according to pigs and environmental conditions, and targeted data augmentation for farming scenarios is applied to generate a high-quality multimodal training dataset. S83: Design a multi-input multi-task learning framework to train an improved YOLOv5s object detection model, and apply a model update strategy to generate a robust estrus detection model that adapts to different environmental conditions.

9. The method for detecting estrus in sows based on the improved YOLOv5s according to claim 8, characterized in that, The data augmentation process in step S82 includes: S821: Apply noise addition, brightness adjustment, angle rotation, region cropping, and position translation transformations with random parameters to the original image sequence to generate a basic augmented dataset; S822: Add mud and dirt occlusion layers, uneven lighting gradients, cast shadow effects, and simulated lens blur to the basic augmented dataset to generate interference augmented data that simulates complex environmental conditions. S823: Morphological transformations are applied to the vulva area to simulate different degrees of redness and swelling, changes in opening angle, and surface moisture characteristics, generating a morphologically enhanced dataset that reflects the physiological changes of estrus.

10. The method for detecting estrus in sows based on the improved YOLOv5s according to claim 8, characterized in that, Step S83 includes: S831: Construct a multi-branch network structure, including a convolutional branch for processing image data, a fully connected branch for processing sensor data, and a feature interaction module containing an attention gating mechanism, forming an end-to-end multimodal model architecture; S832: Design a composite loss function that combines the EIoU loss of object detection, the perceptual loss of vulva texture similarity, and the smoothness consistency loss of temporal prediction in a weighted manner to establish an optimization objective that takes into account multi-dimensional feature learning. S833: Employs a phased training strategy, enabling the model to learn basic features from standard environmental data and then transfer learning to specific pigsty environmental conditions, generating a multimodal estrus detection model with environmental adaptability.

Images