Non-contact detection method for pig backfat thickness based on residual network of SE and CAM mechanism

Abstract

The present application relates to a pig backfat thickness detection method based on a residual network model of SE and CAM mechanism, comprising: shooting a pig back image in a limiting fence; image cropping processing is carried out to reduce the influence of the background area, and a pig back image sample set is constructed in combination with the pig backfat thickness; a residual network model for pig backfat thickness detection is constructed; training hyperparameters are set, and the residual network model is trained by using the pig back image sample set until the residual network model reaches the accuracy requirement; the pig back image in the limiting fence is shot in real time as the input of the residual network model, and the pig backfat thickness is obtained according to the output of the residual network model. The present application combines a spatial attention mechanism CAM and a channel attention mechanism SE to construct a residual network model, realizes automatic detection of pig backfat thickness without contact, and solves the problems of high stress, high cost and inability to detect in real time in the prior art.

Description

Technical Field

[0001] This invention belongs to the field of industrial pig farming monitoring, specifically involving a method for detecting backfat thickness in pigs based on a residual network model with SE and CAM mechanisms. Background Technology

[0002] Backfat thickness in sows is a crucial factor affecting their reproductive capacity. Dynamic monitoring of backfat thickness is an important means of improving sow reproductive performance and productivity, and a strong guarantee for the economic benefits of pig farms and the realization of large-scale farming. Visual inspection is a primitive method for detecting backfat in sows, supplemented by pressure. This method is subjective, can cause stress in sows, and is difficult to achieve in terms of accuracy and real-time measurement. Besides visual inspection, there is also the carcass method, which uses vernier calipers to measure carcass backfat. This method provides accurate results but is only suitable for detecting dead pigs, and is labor-intensive and cumbersome.

[0003] In 1945, Hazel invented the probe method, which ushered in a new era in pig backfat measurement. This method offered improved accuracy compared to visual inspection, but it required piercing the pig's skin, causing severe stress and negatively impacting pig production. In the 1950s, Wild first applied ultrasound technology to the detection of biological tissues, and since then, ultrasound has gradually become the non-invasive method for measuring backfat thickness in pigs. Currently, the primary method for measuring backfat thickness in pigs is ultrasound. Ultrasound requires professionals to draw the backfat location on the ultrasound image based on specialized theory to calculate the backfat thickness. This reduces the accuracy of the measurement results, is time-consuming and relatively inefficient, and is limited by equipment and the capabilities of the measurement personnel, making real-time, low-cost, non-contact backfat thickness measurement impossible. Summary of the Invention

[0004] The purpose of this invention is to address the aforementioned problems by providing a method for detecting pig backfat thickness based on a residual network model using SE and CAM mechanisms. Based on a first residual module and a second residual module, the invention utilizes the spatial attention mechanism (CAM) to increase the weight of useful feature information in each channel feature map, and the channel attention mechanism (SE) to assign greater weights to more important channels in different channel feature maps. This constructs a residual network model for detecting pig backfat thickness. Images of pig backs captured in the pen are used as input to the detection model, enabling contactless automatic detection of pig backfat thickness. Preprocessing of the images of pig backs captured using the UNet model separates image regions corresponding to irrelevant features, segmenting the pig back image and preventing irrelevant features from affecting the detection accuracy of the model.

[0005] The technical solution of this invention is a method for detecting backfat thickness in pigs based on a residual network model using SE and CAM mechanisms, comprising the following steps: Step 1: Take an image of the pig's back inside the restraint enclosure; Step 2: Crop the image obtained in Step 1 to reduce the influence of the background area, and construct a sample set of pig back images based on the back fat thickness of the pig; Step 3: Construct a residual network model for detecting backfat thickness in pigs; Step 4: Set the training hyperparameters and train the residual network model using a set of pig back images until the residual network model reaches the required accuracy. Step 5: Take real-time images of the pig's back inside the restraint bar, use them as input to the residual network model, and obtain the backfat thickness of the pig based on the output of the residual network model.

[0006] Furthermore, the residual network model includes a first convolutional layer, a first pooling layer, a first residual module, a second convolutional layer, a second pooling layer, a second residual module, a first fully connected layer, and a second fully connected layer connected in sequence.

[0007] Preferably, the input to the residual network model is a grayscale image of a pig's back with a size of 224×224.

[0008] Preferably, both the first and second convolutional layers use 3×3 convolutional kernels and both employ boundary expansion.

[0009] Preferably, both the first pooling layer and the second pooling layer use 2×2 convolution kernels and both use max pooling.

[0010] Furthermore, the first residual module employs a spatial attention mechanism (CAM) to increase the weight of useful feature information in each channel's feature map. Specifically, this includes: 1) Perform global pooling and average pooling on the feature map; 2) Input the feature map obtained in step 1) into the multilayer perceptron (MLP) and use the MLP to learn the features of the channel dimension and the importance of each channel; 3) Perform a "+" operation on the features output by the multilayer perceptron (MLP), then process them through the sigmoid activation function to obtain the attention value, and multiply the attention value by the channel map.

[0011] Preferably, the first residual module adopts a combination of channel attention mechanism (SE) and spatial attention mechanism (CAM). The channel attention mechanism (SE) assigns greater weights to more important channels in different channel feature maps. Specifically, it includes: compressing the H x W x C feature map extracted by the residual module into a 1 x 1 x C feature map using a global average pooling layer, where H, W, and C represent the height, width, and number of channels of the feature map, respectively; inputting the 1 x 1 x C feature map into a fully connected layer to predict the importance of each channel, and then inputting it into a sigmoid activation function to complete the feature map activation process; multiplying the vector obtained from the feature map activation process with the H x W x C feature map extracted by the residual module as input to obtain a new H x W x C feature map.

[0012] Preferably, the UNet model is used to preprocess the pig back image captured in step 1 or step 5 within the restraint bar, separate the image regions corresponding to irrelevant features, segment the pig back image, and then input it into the residual network model. This avoids the image of irrelevant features affecting the detection accuracy of the residual network model and improves the training efficiency or real-time detection efficiency of the residual network model.

[0013] Compared with the prior art, the beneficial effects of the present invention include: 1) This invention constructs a lightweight residual network based on the first residual module and the second residual module as a pig backfat thickness detection model. It realizes the automatic detection of pig backfat thickness based on the back images of pigs in the pen, replacing manual detection and measurement, and solves the problems of high stress, high cost and inability to detect backfat thickness in sows in existing methods.

[0014] 2) This invention utilizes the spatial attention mechanism CAM to increase the weight of useful feature information in each channel feature map, and uses the channel attention mechanism SE to assign greater weight to more important channels in different channel feature maps, thereby improving the detection accuracy of the pig backfat thickness detection model.

[0015] 3) This invention uses the UNet model to preprocess the pig back image within the enclosure obtained by shooting, separates the image region corresponding to irrelevant features, segments the pig back image, and then inputs it into the residual network model, which reduces the impact of irrelevant feature images on the detection accuracy of the residual network model and improves the training efficiency or real-time detection efficiency of the residual network model.

[0016] 4) This invention employs Adam, a first-order gradient optimization algorithm for stochastic objective functions based on low-order moment adaptive estimation, and adopts a learning rate decay strategy. By searching the hyperparameter space using OPTUNA, the optimal combination of network hyperparameters is determined, which improves training efficiency and achieves the optimization of network parameters in the residual network model.

[0017] 5) The method of the present invention is easy to execute on a computer, and its deployment and implementation are simple when combined with hardware devices, which is conducive to its promotion and use. Attached Figure Description

[0018] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0019] Figure 1 This is a schematic diagram of the residual network model in Example 1.

[0020] Figure 2 This is a schematic diagram of the residual network model based on the SE and CAM mechanisms in Example 2.

[0021] Figure 3 This is a schematic diagram of the channel attention mechanism (SE) according to an embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of the spatial attention mechanism CAM according to an embodiment of the present invention.

[0023] Figure 5 This is a schematic diagram showing the actual measurement location of the back fat of a pig.

[0024] Figure 6 This is a schematic diagram illustrating the segmentation of a sow's back image using the UNet model in an embodiment of the present invention.

[0025] Figure 7 a This is the feature map extracted from the first convolutional layer of the residual network model in Example 1.

[0026] Figure 7b This is the feature map extracted from the first residual module layer of the residual network model in Example 1.

[0027] Figure 7c This is the feature map extracted from the second convolutional layer of the residual network model in Example 1.

[0028] Figure 7d This is a feature map extracted from the second residual module layer of the residual network model in Example 1.

[0029] Figure 8 This is a structural diagram of the UNet network in Example 1. Detailed Implementation

[0030] Example 1 Image data for this example was collected from a pig farm in Guangxi Zhuang Autonomous Region, totaling 48 pregnant sows, including those in early and mid-gestation periods. All sows were housed in individual pens, with sows at different gestation stages housed in different pens. Video data for each sow was collected using an Azure Kinect camera. The camera was mounted on a self-made adjustable mobile cart, capturing video data of the sow's back when standing from a top-down angle. The camera recorded video at 30 frames per second, with a video duration of 3 minutes. RGB images were obtained from the video analysis and used as the research subjects in this study. The backfat thickness of the 48 sows was measured using a Renco (LEAN-METER) backfat measuring instrument. The measuring range of the instrument is 4-35 mm, with an error of ±1 mm. The P2 measuring point, commonly used in international pig farming, was selected for the backfat measurement. Figure 4 As shown, Figure 4 Point A represents the actual measurement location of the sow's backfat, and the area near point B represents the vicinity of the sow's highest point. Video data of the sow's back was analyzed to obtain back image data. Considering the similarity of adjacent images, one frame was captured every two seconds. Each sow's video yielded 90 images of 1280×720 resolution, resulting in 4320 RGB images from 48 sows. The captured information includes the sow in the gestation stall, sows in adjacent stalls, and the drainpipe above, among other background elements. To reduce the influence of the background and accelerate the model's forward inference and backpropagation, the sow back images were cropped and grayscaled, retaining more information about the sows in the central gestation stall and minimizing information about sows in adjacent stalls. The image resolution was reduced from 1280×720 to 1080×450. The cropped images, combined with the sow's backfat thickness, were used to construct the sample dataset.

[0031] like Figure 1 As shown, the residual network model contains two convolutional layers, two pooling layers, two residual layers, and two fully connected layers. Classic residual networks such as ResNet18, ResNet50, and ResNet152 have 18, 50, and 152 layers, respectively. However, the dataset used in this study is relatively small. Using a classic residual network to learn from this dataset resulted in severe overfitting, making it unsuitable for this dataset. Extensive experiments testing different network layer counts revealed that a residual network with only 6 layers achieved the best performance. Compared to classic networks, it is a lightweight network, making it more suitable for learning smaller datasets.

[0032] The method for detecting backfat thickness in pigs based on a residual network model includes the following steps: Step 1: Take an image of the pig's back inside the restraint enclosure; Step 2: Crop the image obtained in Step 1 to reduce the influence of the background area, and construct a sample set of pig back images based on the back fat thickness of the pig; Step 3: Construct a residual network model for detecting backfat thickness in pigs; In this embodiment, the residual network model includes a first convolutional layer, a first pooling layer, a first residual module, a second convolutional layer, a second pooling layer, a second residual module, a first fully connected layer, and a second fully connected layer connected in sequence, as follows: Figure 1 As shown.

[0033] The input to the residual network model is a grayscale image of a pig's back, with a size of 224×224.

[0034] Both the first and second convolutional layers use 3×3 convolutional kernels and employ boundary padding. The first convolutional layer has n1 kernels, and the second convolutional layer has n2 kernels. Both kernels have a stride of 1 and zero padding.

[0035] The first and second pooling layers both have a 2×2 kernel size, a stride of 1, and zero padding, and both use max pooling.

[0036] The number of convolutional kernels in the first residual module is the same as the number of convolutional kernels in the first convolutional layer; the number of convolutional kernels in the second residual module is the same as the number of convolutional kernels in the second convolutional layer, and the local connections of the first residual module and the second residual module use the LeakyReLu activation function.

[0037] Step 4: Set the training hyperparameters, where n1 and n2 represent the number of convolutional kernels in convolutional layer 1 and convolutional layer 2, respectively, both ranging from 2. 2 ~6 2 n3 and n3 represent the number of hidden neurons in fully connected layer 1 and fully connected layer 2, respectively, with a range of 2. 2 ~6 2 The weight decays represent Adam's decay coefficient. The larger the value, the stronger the ability to suppress overfitting. The value ranges from 0.1 to 0.00001, as shown in Table 1.

[0038] Table 1 Hyperparameter Search Space The space of five hyperparameter combinations was automatically searched using the OPTUNA automatic hyperparameter search framework. The residual network model was trained using a set of pig back images until the residual network model reached the required accuracy.

[0039] In this embodiment, the Adam algorithm, a first-order gradient optimization algorithm for a stochastic objective function based on low-order moment adaptive estimation, is used to determine the decay coefficient by searching the hyperparameter space.

[0040] The implementation example employs a learning rate decay strategy, which specifically includes: setting the initial learning rate to 0.0001; keeping the learning rate unchanged when the number of rounds is less than 15; decaying the learning rate to 10% of the initial learning rate when the number of rounds is between 15 and 25; and decaying the learning rate again to 10% of the learning rate within the range of 15 to 25 when the number of rounds is greater than 25.

[0041] Step 5: Take real-time images of the pig's back inside the restraint bar, use them as input to the residual network model, and obtain the backfat thickness of the pig based on the output of the residual network model.

[0042] In this embodiment, the feature maps of the four layers extracted by the residual network are analyzed. The feature maps extracted by the first convolutional layer, the first residual module layer, the second convolutional layer, and the second residual module layer of the residual network model are shown in Figure 7. a As shown in Figures 7b, 7c, and 7d, the four layers are, in order, the first convolutional layer, the first residual module layer, the second convolutional layer, and the second residual module layer. It can be observed that besides extracting relevant features of the pig's body, irrelevant features such as the drain pipe above the sow's back and the feeding pen next to it are extracted, ultimately reducing the model's accuracy. Therefore, the VGG16-UNet model is used to preprocess the pig's back image within the restraint pen taken in step 1 or step 5 to separate the image regions corresponding to irrelevant features. The UNet network structure is divided into two parts: feature extraction and enhanced feature extraction, as shown in Figure 7b, 7c, and 7d. Figure 8 As shown, the final output is a mask image of the pig. Post-processing is then performed on the segmented mask image to segment out the pig's back image. The general segmentation process is as follows: Figure 6 As shown in Table 2, the images are then input into the residual network model to avoid irrelevant features from affecting the detection accuracy of the residual network model, thereby improving the training efficiency or real-time detection efficiency of the residual network model.

[0043] Table 2 Comparison of Test Set Detection Results Before and After Segmentation It can be seen that the R-value of the model obtained after image segmentation is... 2 The new back image after segmentation is higher than before segmentation, while the MAE is lower, indicating that the performance of the detection model is improved.

[0044] Example 2 Unlike Example 1, Example 2 adds SE and CAM mechanisms to the residual network model to further improve the accuracy of the model in predicting backfat.

[0045] like Figure 2 As shown, the method for detecting backfat thickness in pigs based on a residual network model using SE and CAM mechanisms includes the following steps: Step 1: Take an image of the pig's back inside the restraint enclosure; Step 2: Cropping is performed on the image obtained in Step 1 to reduce the influence of the background region. The VGG16-UNet model is then used to preprocess the cropped pig back image, separating image regions corresponding to irrelevant features and segmenting the pig back image, as shown below. Figure 5 As shown, a sample set of images of pig back fat was constructed by combining the back fat thickness of pigs; Step 3: Construct a residual network model based on SE and CAM mechanisms for detecting backfat thickness in pigs; In this embodiment, the residual network model includes a first convolutional layer, a first pooling layer, a first residual module, a second convolutional layer, a second pooling layer, a second residual module, a first fully connected layer, and a second fully connected layer connected in sequence, as follows: Figure 1 As shown.

[0046] The input to the residual network model is a grayscale image of a pig's back, with a size of 224×224.

[0047] Both the first and second convolutional layers use 3×3 convolutional kernels and employ boundary padding. The first convolutional layer has n1 kernels, and the second convolutional layer has n2 kernels. Both kernels have a stride of 1 and zero padding.

[0048] The first and second pooling layers both have a 2×2 kernel size, a stride of 1, and zero padding, and both use max pooling.

[0049] The number of convolutional kernels in the first residual module is the same as the number of convolutional kernels in the first convolutional layer; the number of convolutional kernels in the second residual module is the same as the number of convolutional kernels in the second convolutional layer, and the local connections of the first residual module and the second residual module use the LeakyReLu activation function.

[0050] Step 4: Set the training hyperparameters and train the residual network model using a set of pig back images until the residual network model reaches the required accuracy. In this embodiment, the Adam algorithm, a first-order gradient optimization algorithm for a stochastic objective function based on low-order moment adaptive estimation, is used.

[0051] The implementation example employs a learning rate decay strategy, which specifically includes: setting the initial learning rate to 0.0001; keeping the learning rate unchanged when the number of rounds is less than 15; decaying the learning rate to 10% of the initial learning rate when the number of rounds is between 15 and 25; and decaying the learning rate again to 10% of the learning rate within the range of 15 to 25 when the number of rounds is greater than 25.

[0052] Step 5: Take real-time images of the pig's back inside the restraint bar, use them as input to the residual network model, and obtain the backfat thickness of the pig based on the output of the residual network model.

[0053] The first residual module employs a combination of channel attention (SE) and spatial attention (CAM). It utilizes SE to assign greater weights to more important channels in different channel feature maps, such as... Figure 3 As shown, the specific steps include: compressing the H x W x C feature map extracted by the residual module into a 1 x 1 x C feature map using a global average pooling layer, where H, W, and C represent the height, width, and number of channels of the feature map, respectively; inputting the 1 x 1 x C feature map into a fully connected layer to predict the importance of each channel, and then inputting it into a sigmoid activation function to complete the feature map activation process; multiplying the vector obtained from the feature map activation process with the H x W x C feature map extracted by the residual module as input to obtain a new H x W x C feature map.

[0054] like Figure 4 As shown, the spatial attention mechanism (CAM) is used to increase the weight of useful feature information in the feature map of each channel, specifically including: 1) Perform global pooling and average pooling on the feature map; 2) Input the feature map obtained in step 1) into the multilayer perceptron (MLP) and use the MLP to learn the features of the channel dimension and the importance of each channel; 3) Perform a "+" operation on the features output by the multilayer perceptron (MLP), then process them through the sigmoid activation function to obtain the attention value, and multiply the attention value by the channel map.

[0055] In this embodiment, similar to the first residual module, the second residual module adopts a combination of channel attention mechanism (SE) and spatial attention mechanism (CAM).

[0056] The results show that, compared with Example 1, the accuracy of the residual network model in predicting backfat is further improved after adding the channel attention mechanism (SE) and the spatial attention mechanism (CAM).

Claims

1. A method for detecting backfat thickness in pigs based on a residual network model using SE and CAM mechanisms, characterized in that, Includes the following steps: Step 1: Take an image of the pig's back inside the restraint enclosure; Step 2: The image obtained in Step 1 is cropped to reduce the influence of the background area. The VGG16-UNet model is used to preprocess the cropped pig back image to separate the image regions corresponding to irrelevant features, segment the pig back image, and construct a pig back image sample set by combining the backfat thickness of the pig. Step 3: Construct a residual network model based on SE and CAM mechanisms for detecting backfat thickness in pigs; the residual network model includes a first convolutional layer, a first pooling layer, a first residual module, a second convolutional layer, a second pooling layer, a second residual module, a first fully connected layer, and a second fully connected layer connected in sequence; the input of the residual network model is a grayscale image of the pig's back, with a size of 224×224; both the first and second convolutional layers use 3×3 convolutional kernels and both employ boundary padding; both the first and second pooling layers use 2×2 convolutional kernels and both employ max pooling; The first residual module adopts a combination of channel attention mechanism (SE) and spatial attention mechanism (CAM). The spatial attention mechanism is used to increase the weight of useful feature information in each channel feature map, and the channel attention mechanism (SE) is used to assign greater weight to the more important channels in different channel feature maps. The method of increasing the weight of useful feature information in each channel feature map by utilizing spatial attention mechanism specifically includes: 1) performing global pooling and average pooling on the feature map; 2) inputting the feature map obtained in step 1) into a multilayer perceptron (MLP) and using the MLP to learn the features of the channel dimension and the importance of each channel; 3) performing a "+" operation on the features output by the MLP, then mapping them through a sigmoid activation function to obtain the attention value, and multiplying the attention value with the channel map; The channel attention mechanism (SE) assigns greater weights to more important channels in different channel feature maps. Specifically, it includes: compressing the H×W×C feature map extracted by the residual module into a 1×1×C feature map using a global average pooling layer, where H, W, and C represent the height, width, and number of channels of the feature map, respectively; inputting the 1×1×C feature map into a fully connected layer to predict the importance of each channel, and then inputting it into a sigmoid activation function to complete the feature map activation process; multiplying the vector obtained from the feature map activation process with the H×W×C feature map extracted by the residual module as input to obtain a new H×W×C feature map. The structure of the second residual module is the same as that of the first residual module; Step 4: Set the training hyperparameters and train the residual network model using a sample set of pig back images until the residual network model reaches the required accuracy. Step 5: Take real-time images of the pig's back inside the restraint bar, use them as input to the residual network model, and obtain the backfat thickness of the pig based on the output of the residual network model.

2. The method for detecting backfat thickness in pigs according to claim 1, characterized in that, In step 4, the setting of training hyperparameters adopts Adam, a first-order gradient optimization algorithm for stochastic objective functions based on low-order moment adaptive estimation, and uses OPTUNA to search the hyperparameter space to determine the optimal combination of network hyperparameters.

Images