A sow farrowing status monitoring method and apparatus

Abstract

The application discloses a sow delivery state monitoring method and device, the method comprises the following steps: obtaining a sow delivery video, the sow delivery video comprises a sow delivery process; inputting the sow delivery video into a preset sow delivery state classification network model to obtain a sow delivery state and record piglet information; wherein, the sow delivery state classification network model comprises a YOLO V5 classification network model, and a SAC module is added to the YOLO V5 classification network model. Through the method, accurate information about the birth time of piglets and the duration of the sow delivery process can be recorded, labor can be reduced, and production management efficiency can be improved. Moreover, the sow delivery state classification network model using the SAC module can extract more detailed features, so that the network model has better performance, and the accuracy of the monitoring result of sow delivery is improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and device for monitoring the farrowing status of sows. Background Technology

[0002] my country is a major livestock and poultry farming country, and pig farming is an important component of its livestock industry, providing a vital driving force for the development of my country's animal husbandry. The farrowing process is a crucial stage in pig production, and the birth of piglets requires close monitoring to ensure the health and safety of the sow and her offspring.

[0003] In traditional pig farming, close manual observation of the health of sows and piglets is necessary to promptly detect and address abnormalities such as dystocia, abortion, stillbirth, and retained placenta. Furthermore, on large-scale farm production lines, manual recording of farrowing information for each sow in each pen is typically required. However, manually recording the time and duration of farrowing is impractical; this manual observation method is both time-consuming and labor-intensive, and prone to omissions and errors in data entry.

[0004] Therefore, an automated method is needed to accurately and effectively monitor the farrowing process. This method would automatically detect and record the number of piglets born, the time of birth, and the duration of the farrowing process, providing precise information about the timing of birth and the duration of the process. This information can be used to improve the health and safety of sows and their offspring, reduce labor costs, and increase productivity. Summary of the Invention

[0005] To overcome the problems existing in related technologies, the present invention provides a method and device for monitoring the farrowing status of sows, thereby solving the defects in related technologies.

[0006] According to a first aspect of the present invention, a method for monitoring the farrowing status of a sow is provided, the method comprising:

[0007] Obtain a video of a sow giving birth, the video of which includes the sow's birthing process;

[0008] The sow farrowing video is input into a preset sow farrowing state classification network model to obtain the sow's farrowing state and record farrowing information; wherein, the sow farrowing state classification network model includes a YOLO V5 classification network model, and the SAC module is added to the YOLO V5 classification network model.

[0009] Preferably, the sow farrowing status classification network model is obtained through the following steps:

[0010] Add the SAC module to the YOLO V5 classification network model to build the initial classification network model;

[0011] Acquire sow farrowing video data, extract frames from the sow farrowing video data in time sequence to obtain several sow farrowing images, and generate a sow farrowing dataset based on the sow farrowing images;

[0012] The initial classification network model is trained using the sow farrowing dataset to obtain the sow farrowing state classification network model.

[0013] Preferably, the addition of the SAC module to the YOLO V5 classification network model specifically involves:

[0014] Replace the last C3 module in the backbone of the YOLO V5 classification network model with the SAC module.

[0015] Preferably, generating a sow farrowing dataset based on the sow farrowing images includes:

[0016] Convert the sow farrowing images into a 64-bit floating-point image type;

[0017] The detail index of the converted sow farrowing image is obtained by using a Laplace filter;

[0018] Images of sows giving birth with low detail index are filtered according to preset fuzzy filtering rules;

[0019] The sow farrowing dataset is generated based on the filtered images of sow farrowing.

[0020] Preferably, generating a sow farrowing dataset based on the sow farrowing images further includes:

[0021] The SSIM algorithm was used to calculate the similarity of the sow farrowing images in three components: brightness, contrast, and structure.

[0022] The images of sows giving birth with high similarity are filtered according to preset similarity filtering rules;

[0023] The sow farrowing dataset is generated based on the filtered images of sow farrowing.

[0024] Preferably, constructing the initial classification network model further includes:

[0025] Replace the CrossEntropyLoss function in the YOLO V5 classification network model with the PolyLoss function.

[0026] Preferably, after the step of generating the sow farrowing dataset, the method further includes:

[0027] Data augmentation was performed on the sow farrowing dataset.

[0028] Preferably, the data augmentation process includes any one or more of the following:

[0029] The sow farrowing image is rotated at any angle within a specified range with a specified probability;

[0030] The sow farrowing image is horizontally flipped with a specified probability;

[0031] The brightness, contrast, and saturation of the sow farrowing image are randomly changed with a specified probability;

[0032] Gaussian noise is randomly added to the sow farrowing images with a specified probability;

[0033] The processed sow farrowing image is scaled and cropped at the center to a specified size to generate a new sow farrowing image.

[0034] Preferably, the method further includes:

[0035] The evaluation index for each sow farrowing status category in the sow farrowing status classification network model is calculated according to the following formula, and the evaluation index is used to test the sow farrowing status classification network model:

[0036]

[0037] Wherein, P is the evaluation index of the sow farrowing status category, N is the number of samples whose farrowing status category is accurately predicted by the sow farrowing status classification network model, and T is the total number of samples predicted by the sow farrowing status classification network model.

[0038] Preferably, the step of inputting the sow's farrowing video into a preset sow farrowing state classification network model to obtain the sow's farrowing state and record farrowing information specifically involves:

[0039] After inputting the sow's farrowing video or completing a recording, the first stage of testing is performed;

[0040] In the first stage of detection, when the number of consecutive frames in which the sow is in the first state of farrowing is greater than a first frame number threshold, and the probability of detecting that the sow is in the first state of farrowing is greater than a first probability threshold, the second stage of detection is performed.

[0041] In the second stage of detection, when the number of consecutive frames in which the sow is in the second state of farrowing is greater than the second frame number threshold, and the probability of detecting that the sow is in the second state of farrowing is greater than the second probability threshold, the third stage of detection is performed.

[0042] In the third stage of detection, when the number of consecutive frames in which the sow is in the third state of farrowing is greater than the third frame number threshold, and the probability of detecting the sow in the third state of farrowing is greater than the third probability threshold, the number of piglets born by the sow, the time of farrowing, and the duration of farrowing in the sow farrowing video are recorded, and the current video frame is saved.

[0043] According to a second aspect of the present invention, a sow farrowing status monitoring device is provided, the device comprising:

[0044] The input module is used to acquire videos of sows giving birth, including the sow's birthing process.

[0045] The monitoring module is used to input the sow farrowing video into a preset sow farrowing status classification network model to obtain the sow's farrowing status and record farrowing information; wherein, the sow farrowing status classification network model includes a YOLO V5 classification network model, and an SAC module is added to the YOLO V5 classification network model.

[0046] This invention discloses a method and apparatus for monitoring sow farrowing status. It uses a sow farrowing status classification network model including a SAC module to detect sow farrowing videos, monitoring various states of the sow's farrowing process and recording information such as the number of piglets born, the time of birth, and the duration of farrowing. This invention can record precise information about the birth time of piglets and the duration of the sow's farrowing process, reducing labor costs and improving production management efficiency. Furthermore, the sow farrowing status classification network model using the SAC module can extract more detailed features, resulting in better network model performance and improved accuracy of sow farrowing monitoring results.

[0047] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating a method for monitoring the farrowing status of sows according to an embodiment of the present invention.

[0049] Figure 2 This is a flowchart illustrating a training method for a sow farrowing state classification network model according to an embodiment of the present invention.

[0050] Figure 3 This is a structural diagram of a YOLO V5 classification network model according to an embodiment of the present invention.

[0051] Figure 4 This is a structural diagram of a YOLO V5 classification network model with an added SAC module, according to an embodiment of the present invention.

[0052] Figure 5 This is a schematic diagram illustrating a sow farrowing image extracted from sow farrowing video data according to an embodiment of the present invention.

[0053] Figure 6 This is a schematic diagram illustrating the detection results of a sow's farrowing status according to an embodiment of the present invention.

[0054] Figure 7 This is a schematic diagram illustrating a birth information recording result according to an embodiment of the present invention.

[0055] Figure 8 This is a schematic diagram of a sow farrowing status monitoring and detection system according to an embodiment of the present invention.

[0056] Figure 9 This is a schematic diagram of the hardware structure of a computing device according to an embodiment of the present invention. Detailed Implementation

[0057] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0058] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0059] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0060] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0061] like Figure 1 As shown, Figure 1This is a flowchart illustrating a method for monitoring the farrowing status of sows according to an embodiment of the present invention, comprising the following steps:

[0062] Step S101: Obtain the video of the sow giving birth;

[0063] Step S102: Input the sow farrowing video into the preset sow farrowing status classification network model to obtain the sow farrowing status and record farrowing information.

[0064] In step S101, the acquired sow farrowing video is a monitoring video of a sow that needs to be monitored during the actual production process. The video includes the sow's farrowing process and can be acquired in real time by connecting to monitoring equipment such as a monitor, or it can be acquired through other means.

[0065] In step S102, the preset sow farrowing state classification network model includes a YOLO V5 classification network model, and an SAC (Switchable Atrous Convolution) module is added to the YOLO V5 classification network model. YOLO V5 is a deep learning-based object detection algorithm, a stable version of the YOLO (You Only LookOnce) series of algorithms. The YOLO V5 classification network model uses the YOLO V5 algorithm and is characterized by its small size, high speed, and high flexibility. SAC is a network structure used in the DetectoRS network proposed in 2020. SAC represents the micro-level, convolving features with different dilation rates and using a switch function for control. Essentially, it optimizes the convolution kernel by using an adaptive dilation rate. By using a sow farrowing state classification network model that includes a YOLO V5 classification network model with an added SAC module, we can detect sow farrowing videos and monitor various states of sow farrowing to record information such as the number, time, and duration of piglets born. This allows us to record accurate information about the birth time of piglets and the duration of the sow's farrowing process, reducing labor costs and improving production management efficiency. Furthermore, the sow farrowing state classification network model with the SAC module can extract more detailed features, resulting in better network model performance and improving the accuracy of sow farrowing monitoring results.

[0066] In some embodiments, the aforementioned preset sow farrowing state classification network model may be a sow farrowing state classification network model trained by the training method described below in this invention, or a sow farrowing state classification network model including a SAC module trained by other training methods, or other existing sow farrowing state classification network models including a SAC module.

[0067] like Figure 2 As shown, Figure 2 This is a flowchart illustrating a training method for a sow farrowing state classification network model according to an embodiment of the present invention, comprising the following steps:

[0068] Step S201: Add the SAC module to the YOLO V5 classification network model to build the initial classification network model;

[0069] Step S202: Obtain sow farrowing video data, extract frames from the sow farrowing video data in time sequence to obtain several sow farrowing images, and generate a sow farrowing dataset based on the sow farrowing images.

[0070] Step S203: Train the initial classification network model using the sow farrowing dataset to obtain the sow farrowing state classification network model.

[0071] In step S201, by adding the SAC module to the YOLO V5 classification network model, the initial classification network model obtained by this method can extract more detailed features, resulting in better performance of the network model.

[0072] The improved initial classification network model will be described in detail below. The YOLO V5 classification network model includes YOLO V5n-cls, YOLO V5s-cls, YOLO V5m-cls, and YOLO V5l-cls models.

[0073] like Figure 3 As shown, Figure 3 This is a structural diagram of a YOLO V5 classification network model according to an embodiment of the present invention. The YOLO V5 model architecture is based on a Convolutional Neural Network (CNN) with various convolutional layers in the backbone. The backbone consists of a series of feature extraction layers responsible for extracting features from the input image. These features are then fed into the detection head, which uses a set of convolutional and fully connected layers to perform object detection and classification. The Conv module consists of a 3×3 Conv2d, BatchNorm (BN) activation function, and SiLU (Sigmoid Linear Unit). The C3 module consists of three Conv modules and n DarknetBottlenecks (with residual connections). The DarknetBottleneck consists of two Conv modules.

[0074] In some embodiments, adding the SAC module to the YOLO V5 classification network model in step S201 above to construct the initial classification network model can be achieved by replacing the last C3 module in the backbone network of the YOLO V5 classification network model with the SAC module. Figure 4 As shown, Figure 4 This is a structural diagram of a YOLO V5 classification network model with an added SAC module, according to an embodiment of the present invention. Replacing the last C3 module in the YOLO V5 classification network model Backbone with the SAC module generates an improved YOLO V5 classification network model, i.e., the initial classification network model. This replacement allows the network to extract more detailed features, resulting in the improved sow farrowing state classification network model performing better than the original YOLO V5 classification network model on sow farrowing image datasets.

[0075] In this invention, the SAC module takes ConvMoudule as input and converts it into SAC via the SAConv2d module. The SAConv2d module consists of the SAC and two preceding and following Global Contexts. The SAC part improves model performance by performing convolutions with the same input features at different atrous rates and using a switch function to obtain the results. The convolution operation with weights w and rate r is represented by y = Conv(x, w, r), where x is the input and y is the output. Then, the convolutional layer is converted into SAC using the following formula.

[0076]

[0077] Where r is the hyperparameter of SAC, Δw is the trainable weight, and the switch function S(x) means that the input first goes through an average pooling layer with a 5×5 kernel, and then through a 1×1 convolutional layer.

[0078] This invention uses a 224×224 image as an example to describe in detail the process from image input to model output. The process of inputting a 224×224 image into the improved network structure described above and outputting the detection result is as follows:

[0079] Layer 1: The input image is passed through a 2D convolutional layer with a kernel size of 6×6, a stride of 2, and a padding of 2. This layer has 3 input channels (corresponding to the RGB channels of the image) and 32 output channels. The output feature map has a size of 112×112.

[0080] Layer 2: The output of the first convolutional layer is passed through a 2D convolutional layer with a kernel size of 3×3, a stride of 2, and padding of 0. This layer has 32 input channels and 64 output channels. The output feature map size is 56×56.

[0081] Layer 3: The feature map output from the previous layer is passed through the C3 module, which consists of three convolutional layers with 64 input channels and 64 output channels. The first convolutional layer in the C3 module has 32 output channels, and the third convolutional layer has 64 output channels, resulting in an output feature map of size 56×56.

[0082] Layer 4: The output of the previous layer passes through a 2D convolutional layer with a kernel size of 3×3, a stride of 2, and padding of 0. This layer has 64 input channels and 128 output channels. The output feature map is 28×28 in size.

[0083] Layer 5: The feature map obtained from the 4th convolutional layer is input into the C3 module. The C3 module consists of two convolutional layers with 128 input channels and 128 output channels. The output feature map of this layer is 28×28 in size.

[0084] Layer 6: The feature map output from the previous layer is passed through a 2D convolutional layer with a kernel size of 3×3, a stride of 2, and padding of 0. This layer has 128 input channels and 256 output channels. The output feature map is 14×14 in size.

[0085] Layer 7: The feature map output from Layer 6 is input into the C3 module. The C3 module also consists of three convolutional layers and has 256 input channels and 256 output channels. The output feature map of the C3 module is 14×14 in size.

[0086] Layer 8: The feature map output from the previous layer is input into a 2D convolutional layer with a kernel size of 3×3, a stride of 2, and padding of 0. This layer has 256 input channels and 512 output channels. The output feature map is 7×7 in size.

[0087] Layer 9: The feature map output from Layer 8 is passed through the SAC module. This layer consists of convolutional layers with 512 input channels and 512 output channels, and the output feature map is 7×7 in size. Then, the output feature map is passed to the global average pooling layer, which balances the dimensions of the feature map and produces a 512-dimensional vector.

[0088] Layer 10: The 512-dimensional vector is input into the classification module, which consists of a fully connected layer with one input unit and three output units (there are three classes in the classification task). The output of the classification module is a 3-dimensional vector containing the predicted probability that the input image belongs to each of the three classes.

[0089] The network structure model described above uses a series of convolutional layers and C3 and SAC modules to extract features from the input image at different spatial scales, and then classifies them through global average pooling layers and fully connected layers.

[0090] In step S202, the dataset constructed in this embodiment is based on video data of sow farrowing process. Frames are extracted according to the video time sequence to obtain a series of video frames and generate a sow farrowing dataset.

[0091] In some embodiments, in step S202 above, the images in the sow farrowing dataset can also be processed to remove blurry video frames.

[0092] For example, in some embodiments, when generating a sow farrowing dataset based on sow farrowing images, the following operations may also be performed:

[0093] Convert sow farrowing images to a 64-bit floating-point image type;

[0094] The detail index of the converted sow farrowing image is obtained by using a Laplace filter;

[0095] Images of sows giving birth with low detail index are filtered according to preset fuzzy filtering rules;

[0096] A sow farrowing dataset is generated based on filtered images of sows giving birth.

[0097] By following the steps above, several frames of images with high blurriness in the original sow farrowing dataset can be filtered out, resulting in a higher accuracy for the sow farrowing classification network model trained on this dataset.

[0098] In some embodiments, in step S202 above, the images in the sow farrowing dataset can also be processed to remove video frames with high similarity.

[0099] For example, in some embodiments, when generating a sow farrowing dataset based on sow farrowing images, the following operations may also be performed:

[0100] The SSIM algorithm was used to calculate the similarity of sow farrowing images in three components: brightness, contrast, and structure.

[0101] Images of sows giving birth with high similarity are filtered according to preset similarity filtering rules;

[0102] A sow farrowing dataset is generated based on filtered images of sows giving birth.

[0103] By following the steps above, several frames of images with high similarity in the original sow farrowing dataset can be filtered out, resulting in a higher accuracy for the sow farrowing classification network model trained on this dataset.

[0104] In some embodiments, in step S202 above, the images in the sow farrowing dataset can also be processed simultaneously to remove blurry video frames and highly similar video frames. For example, the blurriness of these video frames can be detected first using the Laplacian operator to filter out relatively blurry video frames, and then the SSIM operator can be used to remove highly similar video frames.

[0105] In some embodiments, the specific steps for filtering blurry and highly similar video frames may be as follows:

[0106] 1) Convert the image to a 64-bit floating-point image type;

[0107] 2) Applying a Laplacian filter to the image highlights the high-frequency components. The formula for calculating the Laplacian filter is as follows:

[0108]

[0109] Where L(x,y) is the pixel value output by the Laplacian filter at image position (x,y), f(i,j) is the input pixel value at position (i,j), G(i,j) is the Gaussian kernel value at position (i,j), w is the image width, and h is the image height. This function is used to measure the amount of detail in an image; a higher L(x,y) value indicates more detail.

[0110] 3) Filter out the two frames with the lowest L(x,y) values ​​out of every 10 frames;

[0111] 4) Calculate the three similarity components of the image (brightness, contrast, and structure) using the SSIM algorithm on the dataset of filtered blurred frames. Set the threshold to 0.9 and filter out video frames with a similarity greater than 0.9.

[0112] In some embodiments, in step S202 above, the images in the sow farrowing dataset can also be classified simultaneously and divided into training, validation, and test sets. For example, in some embodiments, the sow farrowing images can be classified according to the sow's farrowing state. Images showing the head or feet of a piglet being born are classified as the first category; images showing half a piglet's body are classified as the second category; and images showing the entire piglet's body are classified as the third category. The classified sow farrowing images are then divided into training, validation, and test sets in a 9:1 ratio. Furthermore, the sow farrowing images in the training and validation sets are divided into training and validation sets in an 8:2 ratio. The first, second, and third categories can also be referred to by more specific names such as Quarter, Half, and Whole, and this invention does not limit these names.

[0113] For example, in some embodiments, after screening, 13,946 video frames can be obtained, with a frame size of 896×896. These video frames are then divided into three categories according to the sow's farrowing status. Specific classification criteria are as follows... Figure 5 As shown, Figure 5 This is a schematic diagram of a sow farrowing image extracted from sow farrowing video data according to an embodiment of the present invention. In this diagram, Figure (5-1) is an image of the head or feet of the sow giving birth to a piglet, which is of the Quarter class; Figure (5-2) is an image of the half-body of the sow giving birth to a piglet, which is of the Half class; and Figure (5-3) is a video frame of the whole body of the sow giving birth to a piglet, which is of the Whole class.

[0114] After classifying the images, the Quarter class has 3626 images, the Half class has 2650 images, and the Whole class has 7670 images. These images are then divided into training / validation and test sets in a 9:1 ratio, and the training / validation sets are further divided into training and validation sets in an 8:2 ratio. The number of samples in each class after the division is shown in Table 1.

[0115] Table 1. Distribution of samples by category in the dataset

[0116]

[0117] As shown in Table 1, the sample size distribution of the three categories indicates that the Quarter and Half classes have significantly fewer samples than the Whole class. Therefore, to reduce the impact of class imbalance on the accuracy of model training, the sow farrowing status classification network used in this invention can be improved.

[0118] In some embodiments, in step S201, when constructing the initial classification network model, the CrossEntropyLoss function used by the YOLO V5 classification network can be replaced with the PolyLoss function.

[0119] The core idea of ​​PolyLoss is to reweight each sample's contribution to the loss function based on its class frequency. Specifically, PolyLoss assigns weights to each class based on the inverse of its frequency in the training data. This means that classes that are underrepresented in the training data have higher weights in the loss function, and therefore contribute more to the overall loss. By reweighting the contribution of each sample, PolyLoss helps address class imbalance and improves the model's accuracy on classes with fewer samples. Its calculation formula is as follows:

[0120]

[0121] Where, p t It is the predicted probability of the target category, n is the highest degree of the polynomial, and α is the predicted probability of the target category. j These are the weight coefficients of the j-th polynomial basis function. Applying PolyLoss to the improved network structure significantly enhances the recognition performance for Quarter and Half classes.

[0122] In some embodiments, in step S202, after generating the sow farrowing dataset, data augmentation processing can also be performed on the sow farrowing dataset.

[0123] In some embodiments, the data augmentation process described above may include any one or more of the following:

[0124] Rotate the sow farrowing image at any angle within a specified range with a specified probability;

[0125] Flip the sow farrowing image horizontally with a specified probability;

[0126] The brightness, contrast, and saturation of images of sows giving birth are randomly changed with a specified probability.

[0127] Gaussian noise is randomly added to images of sows farrowing with a specified probability;

[0128] The processed sow farrowing image is scaled and the center portion of the image is cropped to specified dimensions to generate a new sow farrowing image.

[0129] Specifically, in some embodiments, the data augmentation process performed on the sow farrowing dataset may include:

[0130] 1) Select angles that are uniformly distributed within the range of [-180°, 180°] and rotate the video frame with a probability of 0.25;

[0131] 2) Flip the video frame horizontally with a probability of 0.5;

[0132] 3) Randomly change the brightness, contrast, and saturation of video frames with a probability of 0.5;

[0133] 4) Randomly add Gaussian noise to video frames with a probability of 0.25;

[0134] After the above four steps, the aspect ratio of the initial images is maintained, all images are scaled, and the shortest side is made equal to 224. Then, the center portion of the images is cropped to a 224×224 size to obtain the expanded dataset. The improved network model is trained using the expanded dataset, and then the improved model is evaluated using the test set.

[0135] In step S203, the initial classification network model based on the SAC-improved YOLO V5 classification network is trained using the training set to obtain the sow farrowing status classification network model used in this invention.

[0136] In some embodiments, the evaluation index for each sow farrowing status category in the sow farrowing status classification network model can be calculated according to the following formula:

[0137]

[0138] Where P is the evaluation index of the sow's farrowing status category, N is the number of samples whose farrowing status category is accurately predicted by the sow farrowing status classification network model, and T is the total number of samples predicted by the sow farrowing status classification network model.

[0139] Specifically, in some embodiments, the model evaluation metric used can be the Top1 Accuracy of the model in accurately identifying each class. This metric is the percentage of predictions where the model's highest predicted class (highest probability) matches the true class label. A higher Top1 Accuracy indicates better model classification performance. The formula for this evaluation metric is as follows:

[0140]

[0141] Among them, Pi It is the Top 1 Accuracy of the i-th category, N i The model accurately predicts the number of samples in the i-th category, T. i It represents the total number of samples predicted by the model.

[0142] Because the SAC module cannot load pre-trained weights from the YOLO V5 classification network after its addition, the models in Table 2 are trained without loading pre-trained weights to demonstrate the advantages of adding SAC and PolyLoss. The YOLO V5 classification network structure is used as the baseline, and each model is trained for 300 epochs. The trained models are then used to predict on the test set, and the Top-1 Accuracy of each model on the test set is shown in Table 2.

[0143] Table 2 shows the Top 1 Accuracy of each model on the test set.

[0144]

[0145]

[0146] like Figure 6 As shown, Figure 6 This is a schematic diagram illustrating the detection results of sow farrowing status according to an embodiment of the present invention, namely, using a trained (+SAC+PolyLoss) model to predict... Figure 5 From the three states, we can obtain Figure 6The detection results are as follows. Figure (6-1) shows the results obtained after the trained sow farrowing state classification network model predicted graph (5-1). Figure (6-1) shows the probabilities of sow farrowing state labels 1 (corresponding to Quarter class), 0 (corresponding to Half class), and 2 (corresponding to Whole class) being 0.80, 0.17, and 0.03, respectively, thus indicating that the sow farrowing state in this frame belongs to the Quarter class. Figure (6-2) shows the results obtained after the trained sow farrowing state classification network model predicted graph (5-2). Figure (6-2) shows the probabilities of sow farrowing state labels 0 (corresponding to Half class), 0, and 2 (corresponding to Whole class), 0, and 2 (corresponding to Whole class), respectively. The probabilities of 1 (corresponding to the Quarter class) and 2 (corresponding to the Whole class) are 0.60, 0.34, and 0.19, respectively, thus it can be determined that the sow's farrowing state in this frame belongs to the Half class. Figure (6-3) shows the results obtained after the training of the sow farrowing state classification network model and the prediction of Figure (5-3). Figure (6-3) shows that the probabilities of sow farrowing state labels 2 (corresponding to the Whole class), 1 (corresponding to the Quarter class), and 0 (corresponding to the Half class) are 0.89, 0.07, and 0.04, respectively, thus it can be determined that the sow's farrowing state in this frame belongs to the Whole class.

[0147] In some embodiments, when inputting a sow farrowing video into a preset sow farrowing state classification network model to obtain the sow's farrowing state and record farrowing information, the sow's farrowing process can be judged based on three stages: the head or foot, half body, and whole body of the piglets produced during the farrowing process, thereby achieving accurate recording.

[0148] In some embodiments, when inputting a sow farrowing video into a preset sow farrowing state classification network model to obtain the sow's farrowing state and record farrowing information, the following steps may be included:

[0149] After inputting the video of the sow giving birth, the first stage of testing is conducted;

[0150] In the first stage of detection, when the number of consecutive frames in which the sow is in the first state of farrowing is greater than the first frame number threshold, and the probability of detecting the sow in the first state of farrowing is greater than the first probability threshold, the second stage of detection is performed.

[0151] In the second stage of detection, when the number of consecutive frames in which the sow is in the second parturition state is greater than the second frame number threshold, and the probability of detecting the sow in the second parturition state is greater than the second probability threshold, the third stage of detection is performed.

[0152] In the third stage of detection, when the number of consecutive frames in which the sow is in the third state of farrowing is greater than the third frame number threshold, and the probability of detecting the sow in the third state of farrowing is greater than the third probability threshold, the number of piglets born by the sow, the time of farrowing, and the duration of farrowing are recorded in the sow farrowing video, and the current video frame is saved.

[0153] Once a record is completed, the first stage of testing will be repeated to test other piglets produced subsequently.

[0154] In the above steps, the first stage refers to the stage when the sow gives birth to the head or feet of the piglet, the second stage refers to the stage when the sow gives birth to half of the piglet, and the third stage refers to the stage when the sow gives birth to the whole piglet; correspondingly, the first state refers to the state when the sow is in the state of having just given birth to the head or feet of the piglet, the second state refers to the state when the sow is in the state of having given birth to half of the piglet, and the third state refers to the state when the sow is in the state of having given birth to the whole piglet.

[0155] In the above steps, the first frame threshold, the second frame threshold, and the third frame threshold can be set according to the normal duration of the sow being in the corresponding state.

[0156] In the above steps, the first probability threshold, the second probability threshold, and the third probability threshold can be set according to the average probability obtained during training to ensure that the sow's farrowing process is stably in the first, second, or third stage.

[0157] Specifically, this embodiment uses a test video of a sow's farrowing process as an example to illustrate the judgment strategy for recording farrowing. The judgment strategy includes the following:

[0158] 1) Define the number of frames in which the Quarter state is detected consecutively as q, the number of frames in which the Half state is detected consecutively as h, and the number of frames in which the Whole state is detected consecutively as w; define the probability of detecting the Quarter state as P. q The probability of detecting the Half state is P. h The probability of detecting a Whole state is P. w Set the frame rate threshold to t. i (i = 1, 2, 3, representing the three states defined above), the probability threshold is... (i = 1, 2, 3, representing the three states defined above);

[0159] 2) When video streams into the model, recording is triggered only if the following conditions are met:

[0160]

[0161] When the video stream is input, the model detects that the sow's farrowing state in the video frame is of class Quarter, and satisfies q≥t1. When the start time is recorded, the detection of the Half state is triggered. Once the Half state detection is initiated, and h ≥ t2, When the model detects that the sow's farrowing state in a video frame is of the Whole class, and satisfies w ≥ t3, the detection of the Whole state is triggered. At each point, the number of piglets born is incremented by 1, and the End Time is recorded. The duration of farrowing is calculated by subtracting the Start Time from the End Time. At this point, the switches for Half and Whole state detection are turned off, completing one recording of a sow's farrowing. Figure 7 As shown, Figure 7 This is a schematic diagram illustrating a farrowing information recording result according to an embodiment of the present invention. In the recorded video frame, the sow's farrowing state belongs to the Whole class. The probabilities of the sow's farrowing state labels 2 (corresponding to Whole class), 1 (corresponding to Quarter class), and 0 (corresponding to Half class) are 0.88, 0.08, and 0.05, respectively. The number of records is 13, and the corresponding time record is 13: 2023-05-20 12:28:39, indicating the date and time of the birth of the 13th piglet. The duration of this piglet's birth is also recorded as 4.59 seconds.

[0162] Specifically, in some embodiments, the pseudocode for the above determination process can be as follows:

[0163]

[0164] Corresponding to the embodiments of the sow farrowing status monitoring method described above, the present invention also provides a sow farrowing status monitoring device.

[0165] like Figure 8 As shown, Figure 8 This invention discloses a sow farrowing status monitoring device according to one embodiment, comprising the following modules:

[0166] Input module 810 is used to acquire a video of a sow giving birth, the video of the sow giving birth including the sow's birthing process;

[0167] The monitoring module 820 is used to input the sow farrowing video into a preset sow farrowing status classification network model to obtain the sow farrowing status and record farrowing information; wherein, the sow farrowing status classification network model includes a SAC module.

[0168] Preferably, the sow farrowing status classification network model is obtained through the following steps:

[0169] Add the SAC module to the YOLO V5 classification network model to build the initial classification network model;

[0170] Acquire sow farrowing video data, extract frames from the sow farrowing video data in time sequence to obtain several sow farrowing images, and generate a sow farrowing dataset based on the sow farrowing images;

[0171] The initial classification network model is trained using the sow farrowing dataset to obtain the sow farrowing state classification network model.

[0172] Preferably, the addition of the SAC module to the YOLO V5 classification network model specifically involves:

[0173] Replace the last C3 module in the backbone of the YOLO V5 classification network model with the SAC module.

[0174] Preferably, generating a sow farrowing dataset based on the sow farrowing images includes:

[0175] Convert the sow farrowing images into a 64-bit floating-point image type;

[0176] The detail index of the converted sow farrowing image is obtained by using a Laplace filter;

[0177] Images of sows giving birth with low detail index are filtered according to preset fuzzy filtering rules;

[0178] The sow farrowing dataset is generated based on the filtered images of sow farrowing.

[0179] Preferably, generating a sow farrowing dataset based on the sow farrowing images further includes:

[0180] The SSIM algorithm was used to calculate the similarity of the sow farrowing images in three components: brightness, contrast, and structure.

[0181] The images of sows giving birth with high similarity are filtered according to preset similarity filtering rules;

[0182] The sow farrowing dataset is generated based on the filtered images of sow farrowing.

[0183] Preferably, constructing the initial classification network model further includes:

[0184] Replace the CrossEntropyLoss function in the YOLO V5 classification network model with the PolyLoss function.

[0185] Preferably, after the step of generating the sow farrowing dataset, the method further includes:

[0186] Data augmentation was performed on the sow farrowing dataset.

[0187] Preferably, the data augmentation process includes any one or more of the following:

[0188] The sow farrowing image is rotated at any angle within a specified range with a specified probability;

[0189] The sow farrowing image is horizontally flipped with a specified probability;

[0190] The brightness, contrast, and saturation of the sow farrowing image are randomly changed with a specified probability;

[0191] Gaussian noise is randomly added to the sow farrowing images with a specified probability;

[0192] The processed sow farrowing image is scaled and cropped at the center to a specified size to generate a new sow farrowing image.

[0193] Preferably, the method further includes:

[0194] The evaluation index for each sow farrowing status category in the sow farrowing status classification network model is calculated according to the following formula, and the evaluation index is used to test the sow farrowing status classification network model:

[0195]

[0196] Wherein, P is the evaluation index of the sow farrowing status category, N is the number of samples whose farrowing status category is accurately predicted by the sow farrowing status classification network model, and T is the total number of samples predicted by the sow farrowing status classification network model.

[0197] Preferably, the step of inputting the sow's farrowing video into a preset sow farrowing state classification network model to obtain the sow's farrowing state and record farrowing information specifically involves:

[0198] After inputting the sow's farrowing video or completing a recording, the first stage of testing is performed;

[0199] In the first stage of detection, when the number of consecutive frames in which the sow is in the first state of farrowing is greater than a first frame number threshold, and the probability of detecting that the sow is in the first state of farrowing is greater than a first probability threshold, the second stage of detection is performed.

[0200] In the second stage of detection, when the number of consecutive frames in which the sow is in the second state of farrowing is greater than the second frame number threshold, and the probability of detecting that the sow is in the second state of farrowing is greater than the second probability threshold, the third stage of detection is performed.

[0201] In the third stage of detection, when the number of consecutive frames in which the sow is in the third state of farrowing is greater than the third frame number threshold, and the probability of detecting the sow in the third state of farrowing is greater than the third probability threshold, the number of piglets born by the sow, the time of farrowing, and the duration of farrowing in the sow farrowing video are recorded, and the current video frame is saved.

[0202] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0203] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0204] The present invention also provides a computer device, which includes at least a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any of the foregoing embodiments.

[0205] Figure 9 The diagram illustrates a more specific hardware structure of a computing device provided by the present invention. This device may include: a processor 901, a memory 902, an input / output interface 903, a communication interface 904, and a bus 905. The processor 901, memory 902, input / output interface 903, and communication interface 904 are interconnected internally via the bus 905.

[0206] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, to execute relevant programs and implement the technical solution provided by this invention. The processor 901 may also include a graphics card, such as an Nvidia Titan X graphics card or a 1080Ti graphics card.

[0207] The memory 902 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 902 can store the operating system and other application programs. When the technical solution provided by this invention is implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901.

[0208] The input / output interface 903 is used to connect input / output modules to enable information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0209] The communication interface 904 is used to connect the communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0210] Bus 905 includes a pathway for transmitting information between various components of the device, such as processor 901, memory 902, input / output interface 903, and communication interface 904.

[0211] It should be noted that although the above-described device only shows the processor 901, memory 902, input / output interface 903, communication interface 904, and bus 905, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the present invention, and not necessarily all the components shown in the figures.

[0212] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the foregoing embodiments.

[0213] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0214] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0215] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0216] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. In implementing the present invention, the functions of each module can be implemented in one or more software and / or hardware. Alternatively, some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0217] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A method for monitoring the farrowing status of sows, characterized in that, The method includes: Obtain a video of a sow giving birth, the video of which includes the sow's birthing process; The sow farrowing video is input into a preset sow farrowing state classification network model to obtain the sow's farrowing state and record farrowing information; wherein, the sow farrowing state classification network model includes a YOLO V5 classification network model, and the SAC module is added to the YOLO V5 classification network model; The sow farrowing status classification network model was trained through the following steps: Add the SAC module to the YOLO V5 classification network model to build the initial classification network model; Acquire sow farrowing video data, extract frames from the sow farrowing video data in time sequence to obtain several sow farrowing images, and generate a sow farrowing dataset based on the sow farrowing images; The initial classification network model is trained using the sow farrowing dataset to obtain the sow farrowing state classification network model. The addition of the SAC module to the YOLO V5 classification network model specifically involves: Replace the last C3 module in the backbone of the YOLO V5 classification network model with the SAC module.

2. The method according to claim 1, characterized in that, The process of generating a sow farrowing dataset based on the sow farrowing images includes: Convert the sow farrowing images into a 64-bit floating-point image type; The detail index of the converted sow farrowing image is obtained by using a Laplace filter; Images of sows giving birth with low detail index are filtered according to preset fuzzy filtering rules; The sow farrowing dataset is generated based on the filtered images of sow farrowing.

3. The method according to claim 1, characterized in that, The process of generating a sow farrowing dataset based on the sow farrowing images further includes: The SSIM algorithm was used to calculate the similarity of the sow farrowing images in three components: brightness, contrast, and structure. The images of sows giving birth with high similarity are filtered according to preset similarity filtering rules; The sow farrowing dataset is generated based on the filtered images of sow farrowing.

4. The method according to claim 1, characterized in that, The construction of the initial classification network model also includes: Replace the CrossEntropyLoss function in the YOLO V5 classification network model with the PolyLoss function.

5. The method according to claim 1, characterized in that, Following the step of generating the sow farrowing dataset, the following is also included: Data augmentation was performed on the sow farrowing dataset.

6. The method according to claim 5, characterized in that, The data augmentation process includes any one or more of the following: The sow farrowing image is rotated at any angle within a specified range with a specified probability; The sow farrowing image is horizontally flipped with a specified probability; The brightness, contrast, and saturation of the sow farrowing image are randomly changed with a specified probability; Gaussian noise is randomly added to the sow farrowing images with a specified probability; The processed sow farrowing image is scaled and cropped at the center to a specified size to generate a new sow farrowing image.

7. The method according to claim 1, characterized in that, The method further includes: The evaluation index for each sow farrowing status category in the sow farrowing status classification network model is calculated according to the following formula, and the evaluation index is used to test the sow farrowing status classification network model: Wherein, P is the evaluation index of the sow farrowing status category, N is the number of samples whose farrowing status category is accurately predicted by the sow farrowing status classification network model, and T is the total number of samples predicted by the sow farrowing status classification network model.

8. The method according to claim 1, characterized in that, The step of inputting the sow's farrowing video into a preset sow farrowing state classification network model to obtain the sow's farrowing state and record farrowing information specifically involves: After inputting the sow's farrowing video or completing a recording, the first stage of testing is performed; In the first stage of detection, when the number of consecutive frames in which the sow is in the first state of farrowing is greater than a first frame number threshold, and the probability of detecting that the sow is in the first state of farrowing is greater than a first probability threshold, the second stage of detection is performed. In the second stage of detection, when the number of consecutive frames in which the sow is in the second state of farrowing is greater than the second frame number threshold, and the probability of detecting that the sow is in the second state of farrowing is greater than the second probability threshold, the third stage of detection is performed. In the third stage of detection, when the number of consecutive frames in which the sow is in the third state of farrowing is greater than the third frame number threshold, and the probability of detecting the sow in the third state of farrowing is greater than the third probability threshold, the number of piglets born by the sow, the time of farrowing, and the duration of farrowing in the sow farrowing video are recorded, and the current video frame is saved.

9. A sow farrowing status monitoring device, characterized in that, The device includes: The input module is used to acquire videos of sows giving birth, including the sow's birthing process. The monitoring module is used to input the sow farrowing video into a preset sow farrowing status classification network model to obtain the sow's farrowing status and record farrowing information; wherein, the sow farrowing status classification network model includes a YOLO V5 classification network model, and an SAC module is added to the YOLO V5 classification network model; The sow farrowing status classification network model was trained through the following steps: Add the SAC module to the YOLO V5 classification network model to build the initial classification network model; Acquire sow farrowing video data, extract frames from the sow farrowing video data in time sequence to obtain several sow farrowing images, and generate a sow farrowing dataset based on the sow farrowing images; The initial classification network model is trained using the sow farrowing dataset to obtain the sow farrowing state classification network model. The addition of the SAC module to the YOLO V5 classification network model specifically involves: Replace the last C3 module in the backbone of the YOLO V5 classification network model with the SAC module.

Images