Animal gender recognition method, system, device and medium based on deep learning
By constructing a multi-scale aggregation and feature fusion network and using spatial and channel attention modules to accelerate the network's focus on foreground information, the problem of low classification accuracy in animal gender recognition is solved, achieving more efficient gender recognition.
Patent Information
- Application Number
- CN202310493881.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-26
- Publication Date
- 2025-10-17
- Estimated Expiration
- 2043-04-26
AI Technical Summary
The problem of low classification accuracy in the process of animal sex identification in the existing technology.
By constructing a multi-scale aggregation and multi-scale feature fusion network, the spatial and channel attention modules are used to accelerate the network to focus on foreground information and suppress background information, and the expanded convolution operation is combined to obtain richer feature representation.
The accuracy and detection speed of animal gender recognition are improved, and the performance and robustness of the model are enhanced.
Smart Images
Figure CN116503908B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image classification, and particularly relates to an animal gender recognition method, system, device and medium based on deep learning. BACKGROUND
[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.
[0003] The related research on image classification has been widely concerned. Image classification contains a variety of categories, such as classification of different animals, plants, different objects, human gender recognition and animal gender recognition. The animal gender recognition technology can be applied to the wild animal protection, so as to better predict and grasp the number of important endangered animals. Gender recognition can also be applied to large-scale farms, so as to faster calculate the gender ratio and predict the number trend of sheep in the sheep farm.
[0004] The main task of gender classification is to identify the gender of sheep in the image. According to the development context of image classification, the image classification can be divided into two periods: traditional image classification algorithm and deep learning-based classification algorithm.
[0005] The traditional classification algorithm is mainly based on manual feature extraction. However, with the continuous increase of data volume, the continuous improvement of recognition effect and running speed, and the continuous improvement of the accuracy required by actual application, in recent years, the traditional target detection algorithm has been gradually replaced by the emerging deep learning-based classification algorithm.
[0006] At present, a large number of classification algorithms based on deep learning have appeared, for example, ResNet, VGG and transform network all achieve good classification effect, but there are disadvantages of long training time and slow convergence speed in the training process. Therefore, the attention module helps the network to pay more attention to the foreground information, amplify the position containing more useful information, and suppress the position containing irrelevant information or less information. The SE module uses two FC layers and an activation function to generate a relationship matrix between channels. The advantage of Senet is that the module is small and easy to integrate, and can be used multiple times in a network, and the disadvantage is that the improvement ability of the network is limited. CBAM and SGE combine spatial attention and channel attention. CBAM focuses on the channel information and position information of the feature map respectively, the channel attention module performs average pooling and maximum pooling on the feature map and then passes through two consecutive convolution layers, the advantage is that the calculation speed is relatively fast, and the disadvantage is that it cannot extract the information of a channel well; the spatial attention module calculates the mean and maximum value based on the channel dimension, and then splices through consecutive fully connected layers, the advantage is that the calculation speed is relatively fast, and the disadvantage is that the mean and maximum values are not fully utilized. The network will have problems such as loss of bottom features, low utilization rate, and bottom features cannot affect high-level features during training. And when using convolution kernels of different sizes for convolution operation, the size of the receptive field and the extracted features are different, so using a convolution kernel of only one size to calculate the feature map will lose some information, but using multiple convolution kernels of different sizes for convolution will increase the calculation amount.
[0007] In the implementation of the present application, the inventor has found that the prior art has the following technical problems: the animal gender recognition process has the defect of low classification accuracy. SUMMARY
[0008] In order to solve the problems of the prior art, the present application provides an animal gender recognition method, system, device and medium based on deep learning; it is proved by a visual algorithm that the gender can be distinguished from the image of a sheep, and the image contains gender information.
[0009] In a first aspect, the present application provides an animal gender recognition method based on deep learning;
[0010] The animal gender recognition method based on deep learning comprises:
[0011] Obtaining an animal face image to be recognized;
[0012] Inputting the animal face image to be recognized into a trained animal gender recognition model, and outputting an animal gender recognition result;
[0013] The trained animal gender recognition model is configured to: perform basic feature extraction on the animal face image to be recognized to obtain a basic feature map; perform deep feature extraction on the basic feature map to obtain deep feature maps at different stages; perform feature enhancement on the deep feature maps at different stages to obtain enhanced feature maps at different stages; add the enhanced feature maps at different stages element by element to obtain an optimized feature map; perform two rounds of multi-scale aggregation and pooling operations on the optimized feature map to obtain a final feature map; and perform classification processing on the final feature map to obtain an animal gender classification result.
[0014] In a second aspect, the present application provides an animal gender recognition system based on deep learning.
[0015] The animal gender recognition system based on deep learning comprises:
[0016] An acquisition module configured to acquire an animal face image to be recognized.
[0017] An identification module configured to input the animal face image to be recognized into the trained animal gender recognition model and output an animal gender recognition result.
[0018] The trained animal gender recognition model is configured to: perform basic feature extraction on the animal face image to be recognized to obtain a basic feature map; perform deep feature extraction on the basic feature map to obtain deep feature maps at different stages; perform feature enhancement on the deep feature maps at different stages to obtain enhanced feature maps at different stages; add the enhanced feature maps at different stages element by element to obtain an optimized feature map; perform two rounds of multi-scale aggregation and pooling operations on the optimized feature map to obtain a final feature map; and perform classification processing on the final feature map to obtain an animal gender classification result.
[0019] In a third aspect, the present application further provides an electronic device comprising:
[0020] a memory for non-transiently storing computer readable instructions; and
[0021] a processor for running the computer readable instructions,
[0022] When the computer readable instructions are run by the processor, the method of the first aspect is executed.
[0023] In a fourth aspect, the present application further provides a storage medium for non-transiently storing computer readable instructions, wherein when the non-transient computer readable instructions are executed by a computer, the instructions of the method of the first aspect are executed.
[0024] In a fifth aspect, the present application also provides a computer program product comprising a computer program for implementing the method of the first aspect above when run on one or more processors.
[0025] Compared with the prior art, the present application has the beneficial effects that:
[0026] The present application considers the long network training time and slow convergence speed by designing a new attention module, and uses spatial and channel attention modules to accelerate the network to focus on foreground information and suppress background information, so as to find more important features for gender classification.
[0027] The present application can help the network to process the input data at different scales to obtain more comprehensive and informative feature representations by constructing a multi-scale aggregation and multi-scale feature fusion network. Low-level feature representations may contain edge and texture information, while high-level feature representations may contain more abstract semantic information. By fusing these feature representations, richer feature representations can be obtained, thereby improving the performance of the model.
[0028] The present application finally proves that the gender can be distinguished through the image of the sheep by the visualization algorithm, and the image contains gender information. BRIEF DESCRIPTION OF DRAWINGS
[0029] The drawings accompanying the specification of the present application form a part thereof and serve to provide further understanding of the present application, together with the exemplary embodiments of the present application and their description, and do not constitute an improper limitation of the present application.
[0030] Figure 1 The flowchart of the network as a whole;
[0031] Figure 2 The network structure diagram as a whole;
[0032] Figure 3 The structure diagram of the attention module;
[0033] Fig. 4 (a) - Fig. 4 (f) are the original image of sheep and the image after data enhancement;
[0034] Fig. 5 (a) - Fig. 5 (g) are the network training process picture, the result picture and the visualization picture;
[0035] Figure 6 The internal structure diagram of the first multi-scale aggregation module;
[0036] Figure 7 The internal structure diagram of the SE module. DETAILED DESCRIPTION
[0037] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
[0038] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.
[0039] The embodiments in the present application and the features in the embodiments can be combined with each other without conflict.
[0040] Embodiment One
[0041] The embodiment provides an animal gender recognition method based on deep learning;
[0042] As shown in Figure 1 The animal gender recognition method based on deep learning comprises the following steps.
[0043] S101: An animal face image to be recognized is acquired;
[0044] S102: The animal face image to be recognized is input into a trained animal gender recognition model, and an animal gender recognition result is output;
[0045] The trained animal gender recognition model is used for:
[0046] Basic feature extraction is performed on the animal face image to be recognized, and a basic feature map is obtained;
[0047] Deep feature extraction is performed on the basic feature map, and deep feature maps at different stages are obtained;
[0048] Feature enhancement is performed on the deep feature maps at different stages, and enhanced feature maps at different stages are obtained;
[0049] The enhanced feature maps at different stages are added element by element to obtain an optimized feature map;
[0050] The optimized feature map is subjected to two rounds of multi-scale aggregation and pooling operations, and a final feature map is obtained;
[0051] The final feature map is classified to obtain an animal gender classification result.
[0052] Further, as shown in the figure, the trained animal gender recognition model comprises: Figure 2
[0053] a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module and a fifth feature extraction module connected in sequence;
[0054] The output end of the third feature extraction module is connected with the input end of the first attention module;
[0055] The output end of the fourth feature extraction module is connected with the input end of the first transposed convolution layer, the output end of the first transposed convolution layer is connected with the input end of the second transposed convolution layer, and the output end of the second transposed convolution layer is connected with the input end of the second attention module;
[0056] The output end of the fifth feature extraction module is connected with the input end of the third transposed convolution layer, the output end of the third transposed convolution layer is connected with the input end of the fourth transposed convolution layer, and the output end of the fourth transposed convolution layer is connected with the input end of the third attention module;
[0057] The output end of the first attention module, the output end of the second attention module and the output end of the third attention module are all connected with the input end of the splicing unit; the splicing unit performs element-by-element addition on the feature maps output by the three attention modules;
[0058] The output end of the splicing unit is connected with the input end of the convolution layer C1; the output end of the convolution layer is connected with the input end of the first multi-scale aggregation module, the output end of the first multi-scale aggregation module is connected with the input end of the first pooling layer, the output end of the first pooling layer is connected with the input end of the convolution layer C2, the output end of the convolution layer C2 is connected with the input end of the second multi-scale aggregation module, the output end of the second multi-scale aggregation module is connected with the input end of the second pooling layer, the output end of the second pooling layer is connected with the input end of the convolution layer C3, the output end of the convolution layer C3 is connected with the input end of the first full connection layer, the output end of the first full connection layer is connected with the input end of the second full connection layer, and the output end of the second full connection layer is connected with the input end of the activation function layer J1.
[0059] Further, the first feature extraction module is implemented by using a Conv1 layer of the ResNet34 network; the second feature extraction module is implemented by using the first two residual modules of a Conv2_x layer of the ResNet34 network in series; the third feature extraction module is implemented by using the first two residual modules of a Conv3_x layer of the ResNet34 network in series; the fourth feature extraction module is implemented by using the first two residual modules of a Conv4_x layer of the ResNet34 network in series; and the fifth feature extraction module is implemented by using the first two residual modules of a Conv5_x layer of the ResNet34 network in series.
[0060] Further, the Conv1 layer of the ResNet34 network is a 7*7 convolutional layer; the first residual module and the second residual module of the Conv2_x layer of the ResNet34 network have the same internal structure, and the first residual module of the Conv2_x layer comprises, in series, an input end of the first residual module, a 3*3 convolutional layer, a 3*3 convolutional layer, and an output end of the first residual module, wherein the output end of the first residual module is also connected to the input end of the first residual module.
[0061] The first residual module and the second residual module of the Conv3_x layer of the ResNet34 network have the same internal structure, and the first residual module of the Conv3_x layer comprises, in series, an input end of the first residual module, a 3*3 convolutional layer, a 3*3 convolutional layer, and an output end of the first residual module, wherein the output end of the first residual module is also connected to the input end of the first residual module.
[0062] The first residual module and the second residual module of the Conv4_x layer of the ResNet34 network have the same internal structure, and the first residual module of the Conv4_x layer comprises, in series, an input end of the first residual module, a 3*3 convolutional layer, a 3*3 convolutional layer, and an output end of the first residual module, wherein the output end of the first residual module is also connected to the input end of the first residual module.
[0063] The first residual module and the second residual module of the Conv5_x layer of the ResNet34 network have the same internal structure, and the first residual module of the Conv5_x layer comprises, in series, an input end of the first residual module, a 3*3 convolutional layer, a 3*3 convolutional layer, and an output end of the first residual module, wherein the output end of the first residual module is also connected to the input end of the first residual module.
[0064] Further, an SE module is arranged between the Conv3_x layer and the Conv4_x layer.
[0065] Further, an SE module is arranged between the Conv4_x layer and the Conv5_x layer.
[0066] Further, an SE module is arranged between the Conv5_x layer and the third transposed convolution layer.
[0067] The SE (Squeeze Excitation) module is a channel attention module that strengthens the channels of the input feature map. The SE module output does not change the size of the input feature map.
[0068] The structure of the SE module is shown in Figure 7 The SE module includes:
[0069] The SE module input end, the global pooling layer, the third full connection layer, the activation function layer RELU, the fourth full connection layer, the activation function layer sigmoid, the multiplier Q1 and the SE module output end are sequentially connected. The SE module input end is connected with the input end of the multiplier Q1.
[0070] The SE module performs a global adaptive pooling operation on the input feature. The H and W of the feature map become 1. Then the feature map is passed through two consecutive full connection layers. The first full connection layer has a ReLU activation function. The second full connection layer has a Sigmoid activation function. Finally, the obtained spatial attention matrix is multiplied with the original input feature map.
[0071] It should be understood that the feature map output by each convolutional layer is one quarter of the input size, and the number of channels is doubled. An SE module is added after the 1*1 convolution of the second residual module in each layer to accelerate the attention to important channels and accelerate convergence.
[0072] It should be understood that the outputs of the Conv3_x, Conv4_x and Conv5_x are the feature maps extracted by the network, denoted as C3, C4 and C5. The obtained feature maps C4 and C5 are enlarged in size by transposed convolution to make them the same size as C3, preparing for feature fusion.
[0073] Further, as shown in Figure 3 The internal structures of the first attention module, the second attention module and the third attention module are the same.
[0074] Further, the first attention module includes a channel attention unit and a spatial attention unit connected in sequence.
[0075] The channel attention unit includes a channel attention unit input end, a first branch, a second branch and a channel attention unit output end.
[0076] The input end of the channel attention unit is connected with the input end of the first multiplier through a first branch;
[0077] The input end of the channel attention unit is connected with the input end of the first multiplier through a second branch;
[0078] The second branch comprises, in sequence, an inflation convolution layer p1, an inflation convolution layer p2, an inflation convolution layer p3, an inflation convolution layer p4 and a convolution layer; the inflation convolution layer p1 is connected with the input end of the channel attention unit, and the convolution layer is connected with the input end of the first multiplier;
[0079] The output end of the first multiplier is connected with the output end of the channel attention unit;
[0080] The spatial attention unit comprises a spatial attention unit input end, a third branch, a fourth branch and a spatial attention unit output end;
[0081] The third branch comprises, in sequence, a max-pooling layer, a second multiplier, a full connection layer, an activation function layer J2 and a third multiplier;
[0082] The fourth branch comprises, in sequence, an average-pooling layer and an activation function layer J3;
[0083] The input end of the max-pooling layer and the input end of the average-pooling layer are both connected with the input end of the spatial attention unit; the output end of the third multiplier is connected with the output end of the spatial attention unit; the output end of the activation function layer J3 is connected with the input end of the second multiplier; and the input end of the spatial attention unit is connected with the input end of the third multiplier.
[0084] It should be understood that the attention module comprises a channel attention module and a spatial attention module; in the channel attention module, the feature map size is changed to 1*1 by using a continuous inflation convolution operation; and in the spatial attention module, the mean value and the maximum value are calculated in the channel dimension of the feature map, then the maximum value is subjected to a Softmax processing, and then the maximum value is multiplied with the mean value element by element as a weight coefficient.
[0085] The advantage of using the inflation convolution is that a larger receptive field can be obtained, the calculation amount is smaller, the calculation speed is faster, and the importance of the channel can be better obtained. The maximum value is multiplied with the mean value as a weight coefficient, so that the mean value can be properly adjusted, and the overall characteristics can be better reflected.
[0086] Then, the C3, C4 and C5 feature maps are subjected to feature enhancement by using the attention module, and the feature maps obtained after the enhancement are added element by element.
[0087] Since the feature map sizes outputted by Conv3_x, Conv4_x and Conv5_x are the same, the size changes of the feature maps in the attention module are the same, taking one of them as an example.
[0088] Taking the output of Conv3_x as an example, the size of the input feature map is R 32*128*28*28 .
[0089] Firstly, the feature map enters the channel attention module to realize feature enhancement in the channel dimension. The channel attention module includes four consecutive dilated convolutions, the convolution kernel of each dilated convolution is 3*3, the dilation coefficients are 1, 2, 4 and 8 respectively, the input and output channels remain unchanged, and finally a convolution with a convolution kernel of 2*2 is calculated. The size of the output feature map is R 32*128*1*1 Then, the input feature map is multiplied by the output enhancement coefficient to realize feature enhancement in the channel dimension, and the product is taken as the output of the channel attention module.
[0090] By using the dilated convolution, the H and W of the feature map are quickly reduced to 1, and the last pixel has a large receptive field. Compared with the ordinary convolution, the calculation amount is smaller and the calculation speed is faster.
[0091] The output of the channel attention module is taken as the input of the spatial attention module, and the size of the input feature map is R 32 *128*28*28 Firstly, the input feature map is averaged and maximally pooled in the channel dimension, and then the size of the feature map becomes R 32*128*1*1 The averaged feature map is flattened, then processed by Softmax and multiplied by the H and W values of the feature map to ensure that the numerical value is not too small. Then, it is folded back to the previous size, and then multiplied element by element with the maximally pooled feature map to adjust the importance of the maximally pooled feature map. Finally, it is subjected to convolution operation and sigmoid activation processing. The obtained spatial enhancement coefficient is multiplied element by element with the feature map input into the spatial module, and the result is output.
[0092] Because the maximum value and the average value contain different important information in the feature map, using the average value to further adjust the maximum value can obtain more detailed information.
[0093] Further, the internal structures of the first multi-scale aggregation module and the second multi-scale aggregation module are consistent.
[0094] Further, as shown in Figure 6 , the first multi-scale aggregation module comprises:
[0095] The obtained feature map is subjected to three parallel multi-scale convolution operations:
[0096] (1) the feature map is sequentially subjected to dilated convolution operation with a convolution layer q1 and a dilated convolution layer p5, 0 elements are filled in the edge of the feature map, and a first output value is obtained; a convolution kernel of the convolution layer q1 is 3*3; a convolution kernel of the dilated convolution layer is 3*3, and an expansion coefficient is (1, 2);
[0097] (2) the feature map is sequentially subjected to dilated convolution operation with a convolution layer q2 and a dilated convolution layer p6, 0 elements are filled in the edge of the feature map, and a second output value is obtained; a convolution kernel of the convolution layer q2 is 3*3; a convolution kernel of the dilated convolution layer is 5*5, and an expansion coefficient is (1, 2);
[0098] (3) the feature map is sequentially subjected to dilated convolution operation with a convolution layer q3 and a dilated convolution layer p7, 0 elements are filled in the edge of the feature map, and a third output value is obtained; a convolution kernel of the convolution layer q3 is 3*3; a convolution kernel of the dilated convolution layer p7 is 7*7, and an expansion coefficient is (1, 2);
[0099] The first output value, the second output value and the third output value obtained are added together element by element.
[0100] After the feature maps of the three groups of convolution outputs are added element by element, maximum pooling is performed, and then the feature map obtained is doubled in the number of channels by using a convolution kernel of 1*1, and the feature map size output is R 32*256*14*14 .
[0101] The multi-scale convolution and pooling operation can change the feature map into one quarter.
[0102] Further, the basic feature extraction on the animal face image to be recognized is implemented by the first feature extraction module and the second feature extraction module.
[0103] It should be understood that the convolution layer with a convolution kernel size of 7 at the beginning of the ResNet34 network and the maximum pooling layer and the first two residual modules of Conv_2 are reserved together as the basic feature extraction stage.
[0104] Illustratively, first, the size of the input image is cropped to 224*224, then convolution calculation is performed with a convolution kernel with a step of 2 and a size of 7, and the output feature map size is R 32*64*112*112 . The feature map is subjected to maximum pooling operation, and then the feature map size output by the first two residual modules of the Conv_2 layer in the Resnet34 network is R 32*64*56*56 .
[0105] Further, the deep layer feature extraction on the basic feature map is implemented by the third feature extraction module, the fourth feature extraction module and the fifth feature extraction module.
[0106] It should be understood that the first two residual modules of the Conv3_x, Conv4_x, and Conv5_x layers in the ResNet34 network are reserved as the deep feature extraction stage. The output of the high layers of the Conv4_x and Conv5_x is increased to the same size as the feature map of the output of the Conv3_x by using transposed convolution.
[0107] The Conv3_x, Conv4_x, and Conv5_x layers are the first two modules in the corresponding layers in the Resnet34 network, and the size of the feature map output by each layer is R 32*128*28*28 , R 32*256*14*14 , and R 32*256*7*7 , respectively.
[0108] The sizes of the feature maps output by the three layers of the Conv3_x, Conv4_x, and Conv5_x are different, so the feature map with a smaller size is up-sampled to increase the size of the feature map in the application:
[0109] For the output of the Conv4_x, two consecutive transposed convolutions are first performed, and the convolution kernels are 3*3 and 4*4, respectively; the step size is (2, 1), and the padding coefficient is 1; finally, the channel number is adjusted from 256 to 128 by using convolution.
[0110] For the output of the Conv5_x, two consecutive transposed convolutions are first performed, and the convolution kernels are both 4*4, the step size is both 2, and the padding coefficient is 1.
[0111] Finally, the channel number is adjusted from 512 to 256, and then from 256 to 128 by using convolution calculation. The size and channel number of the feature maps output by the three layers are ensured to be the same.
[0112] Further, the deep feature maps of different stages are enhanced to obtain enhanced feature maps of different stages, which is realized by the first attention module, the second attention module, and the third attention module. The channel attention and spatial attention module are used to increase the local features.
[0113] Further, the enhanced feature maps of different stages are added element by element to obtain the optimized feature map, which is realized by the splicing unit.
[0114] Further, the optimized feature map is subjected to two rounds of multi-scale aggregation and pooling operation to obtain the final feature map, which is realized by the first multi-scale aggregation module, the first pooling layer, the convolution layer C2, the second multi-scale aggregation module, the second pooling layer, and the convolution layer C3. The convolution kernel of the convolution layer C2 is 3×3, the input channel is 128, and the output channel is 256. The convolution kernel of the convolution layer C3 is 3×3, the input channel is 256, and the output channel is 512.
[0115] Further, the final feature map is classified to obtain the animal gender classification result, which is achieved by a first fully connected layer, a second fully connected layer and an activation function layer J1. Finally, two consecutive fully connected layers and Softmax processing are performed to obtain the probability of each class corresponding to the final image.
[0116] When the last extracted feature is classified, the input feature size is R 32*256*7*7 First, average pooling is performed, and the feature map size becomes R 32*256*1*1 Then, the feature map is flattened into a line. Then, two fully connected layers are followed, the first fully connected layer has 512 neurons as input and 1000 neurons as output, the second fully connected layer has 1000 neurons as input and two neurons as output, and finally, Softmax processing is performed to obtain the probability of different genders.
[0117] Further, the trained animal gender recognition model, the training process includes:
[0118] The training set is constructed, and the training set includes animal face images with known animal gender;
[0119] The training set is input into the animal gender recognition model to train the model, and the optimizer used for training is the Adam optimizer;
[0120] When the loss function value of the model no longer decreases, the training is stopped, and the trained animal gender recognition model is obtained;
[0121] Nonlinear dimension reduction algorithm (t-SNE, t-distribute stochastic neighbor embedding) is used for dimension reduction, and different gender sheep images are reduced to two-dimensional images to find the distinction between rams and ewes in images, and the images are used as the basis for gender classification;
[0122] The gradient weighted class activation mapping (Grad-CAM, Gradient-Weighted Class Activation Mapping) visualization algorithm is used to visualize the key areas that the network focuses on when distinguishing the gender of sheep, so as to find that the identification areas of rams and ewes are different.
[0123] The t-SNE method is used to reduce different gender images to two dimensions at the same time, and the distribution of different gender images is observed; the trained parameters in the network are loaded, the image is input to identify the gender of the sheep, and the Grad-CAM is used to visualize the key area.
[0124] As shown in FIGS. 5(a)-5(g), two ram photos and two ewe photos are selected, and the pictures are input into the network at one time to learn to extract features, and finally the feature map size is R4*512*7*7 Then the feature map is unfolded into a straight line The feature map size becomes R 4*128*49 Finally, the feature map is divided into two categories in two-dimensional space.
[0125] As shown in FIGS. 5(a)-5(g), the gender separability of the sheep image is proved by using the t-SNE algorithm, then the key features for gender separation are found by using Grad-CAM. The trained weights are imported into the network, and Grad-CAM is used on the last convolutional layer of the Resnet34 network to locate and highlight the distinguishing areas, while the network outputs the gender of the sheep image, the key areas of the classification are visualized.
[0126] Further, the construction of the training set comprises: obtaining animal face images of known animal gender; and performing image cropping and data enhancement processing to obtain the training set.
[0127] Further, the loss function of the model comprises:
[0128] The calculation formula of the cross entropy loss function CrossEntropy Loss is:
[0129]
[0130] The probability predicted for each class is p and 1-p, respectively, wherein:
[0131] Y i represents the label of sample i, and the positive class is 1 and the negative class is 0;
[0132] Y i represents the probability that sample i is predicted as a positive class.
[0133] By using CrossEntropy Loss, the difference between the true classification value and the predicted classification value can be measured, and the smaller the value of CrossEntropy Loss, the closer the true value and the predicted value, and the better the prediction effect of the model. The initial learning rate is 0.0001, and after every 10 rounds, the learning rate becomes one tenth of the previous one.
[0134] The present application proposes a method for realizing sheep gender classification based on an attention module, multi-scale feature fusion and scale aggregation of dilated convolution. The method uses transpose convolution to increase the size of the small feature map, and uses the attention module to promote the fusion of different size feature maps to be more sufficient. At the same time, multiple dilated convolution blocks of different scales are used for convolution operation so that the feature map contains important information of different scales. This method not only improves the detection accuracy and detection speed, but also has stronger robustness.
[0135] As shown in FIGS. 4(a)-4(f), part of the pictures in the data set used by the present application are shown, including the original pictures without processing and the pictures after data enhancement processing.
[0136] The present application provides a sheep gender classification method based on a network of scale aggregation using attention modules, multi-scale feature fusion and dilated convolution. The outputs of multiple stages in the network are used for feature fusion, preserving important semantic information at different stages. A new attention module is added, which reduces the image size through a dilated convolution block and has a larger receptive field and smaller computational complexity. The mean value is then used to further adjust the results of max pooling, better utilizing the mean and maximum values. When reducing the size of the feature map for the second time, using different scale convolution kernels for calculation will have different receptive fields, which can simultaneously take into account the details, edges and shape textures in the image.
[0137] The present application provides a sheep gender classification method based on a network of scale aggregation using attention modules, multi-scale feature fusion and dilated convolution, including: using the initial convolution layer of the Resnet34 network and the first two residual blocks of Conv2_x as the basic feature extraction stage; obtaining the outputs of Conv3_x, Conv4_x and Conv5_x, then increasing the size of the feature map through convolution, then performing feature enhancement through the attention module, and finally performing multi-scale feature fusion; inputting the feature map and different size convolution kernels for calculation. By filling 0 elements around the feature, the size of the feature map after convolution can be kept unchanged, and then the feature maps are added element by element; obtaining the probability corresponding to different genders, summarizing the loss function and training strategy of the present application; using the t-SNE algorithm and Grad-CAM visualization algorithm to prove that the sheep images have gender separability and contain certain gender information. The present application not only diversifies the gender classification method but also provides an efficient method and effectively improves the accuracy of image classification.
[0138] Embodiment Two
[0139] The present embodiment provides an animal gender recognition system based on deep learning;
[0140] The animal gender recognition system based on deep learning comprises:
[0141] The acquisition module is configured to acquire an animal face image to be recognized;
[0142] The recognition module is configured to input the animal face image to be recognized into the trained animal gender recognition model and output an animal gender recognition result;
[0143] The trained animal gender recognition model is configured to: perform basic feature extraction on a face image of an animal to be recognized to obtain a basic feature map; perform deep feature extraction on the basic feature map to obtain deep feature maps at different stages; perform feature enhancement on the deep feature maps at different stages to obtain enhanced feature maps at different stages; add the enhanced feature maps at different stages element by element to obtain an optimized feature map; perform two rounds of multi-scale aggregation and pooling operations on the optimized feature map to obtain a final feature map; and perform classification processing on the final feature map to obtain an animal gender classification result.
[0144] It should be noted that the above acquisition module and recognition module correspond to steps S101-S102 in Embodiment One, and the above modules have the same examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment One. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer executable instructions.
[0145] The description of each embodiment in the above embodiments has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.
[0146] The proposed system can be implemented in other ways. For example, the system embodiments described above are only illustrative, for example, the division of the above modules is only a logical functional division, and in actual implementation, there can be another division method, for example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.
[0147] Embodiment Three
[0148] The embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected with the memory, and the one or more computer programs are stored in the memory; when the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes the method in Embodiment One.
[0149] It should be understood that in the embodiment, the processor can be a central processing unit CPU, and the processor can also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, ready-to-program gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.
[0150] The memory can include read-only memory and random access memory, and provide instructions and data to the processor, a portion of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.
[0151] In the implementation process, each step of the above method can be completed by integrated logic circuit of hardware in the processor or instruction in the form of software.
[0152] The method in embodiment one can be directly embodied as hardware processor execution completion, or executed by hardware and software modules in the processor. The software module can be located in random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory, and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.
[0153] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the present embodiment can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the function is executed in hardware or software mode depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.
[0154] Embodiment four
[0155] The present embodiment also provides a computer readable storage medium for storing computer instructions, which are executed by a processor to complete the method described in embodiment one.
[0156] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. Animal gender recognition method based on deep learning, characterized by: include: Obtaining an animal face image to be identified; Input the animal face image to be identified into the trained animal gender recognition model and output the animal gender recognition result; The trained animal gender recognition model is used to: extract basic features from the facial image of the animal to be recognized to obtain a basic feature map; perform deep feature extraction on the basic feature map to obtain deep feature maps at different stages; perform feature enhancement on the deep feature maps at different stages to obtain enhanced feature maps at different stages; perform element-by-element addition on the enhanced feature maps at different stages to obtain an optimized feature map; perform two rounds of multi-scale aggregation and pooling operations on the optimized feature map to obtain a final feature map; perform classification processing on the final feature map to obtain an animal gender classification result; The trained animal gender recognition model comprises: A first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module, and a fifth feature extraction module connected in sequence; The output of the third feature extraction module is connected to the input of the first attention module; The output of the fourth feature extraction module is connected to the input of the first transposed convolutional layer, the output of the first transposed convolutional layer is connected to the input of the second transposed convolutional layer, and the output of the second transposed convolutional layer is connected to the input of the second attention module; The output of the fifth feature extraction module is connected to the input of the third transposed convolutional layer, the output of the third transposed convolutional layer is connected to the input of the fourth transposed convolutional layer, and the output of the fourth transposed convolutional layer is connected to the input of the third attention module; The output of the first attention module, the output of the second attention module, and the output of the third attention module are all connected to the input of the splicing unit; the splicing unit adds the feature maps output by the three attention modules element by element; The output end of the splicing unit is connected to the input end of the convolutional layer C1; the output end of the convolutional layer is connected to the input end of the first multi-scale aggregation module, the output end of the first multi-scale aggregation module is connected to the input end of the first pooling layer, the output end of the first pooling layer is connected to the input end of the convolutional layer C2, the output end of the convolutional layer C2 is connected to the input end of the second multi-scale aggregation module, the output end of the second multi-scale aggregation module is connected to the input end of the second pooling layer, the output end of the second pooling layer is connected to the input end of the convolutional layer C3, the output end of the convolutional layer C3 is connected to the input end of the first fully connected layer, the output end of the first fully connected layer is connected to the input end of the second fully connected layer, and the output end of the second fully connected layer is connected to the input end of the activation function layer J1.
2. The animal gender identification method based on deep learning according to claim 1, characterized in that: The first feature extraction module is implemented by the Conv1 layer of the ResNet34 network; the second feature extraction module is implemented by the first two residual modules of the Conv2_x layer of the ResNet34 network connected in series; the third feature extraction module is implemented by the first two residual modules of the Conv3_x layer of the ResNet34 network connected in series; the fourth feature extraction module is implemented by the first two residual modules of the Conv4_x layer of the ResNet34 network connected in series; the fifth feature extraction module is implemented by the first two residual modules of the Conv5_x layer of the ResNet34 network connected in series; An SE module is set between the Conv3_x layer and the Conv4_x layer; an SE module is set between the Conv4_x layer and the Conv5_x layer; the SE module includes: the input end of the SE module, the global pooling layer, the third fully connected layer, the activation function layer RELU, the fourth fully connected layer, the activation function layer sigmoid, the multiplier Q1 and the output end of the SE module connected in sequence; wherein, the input end of the SE module is connected to the input end of the multiplier Q1.
3. The animal gender identification method based on deep learning according to claim 1, characterized in that: The first attention module includes: a channel attention unit and a spatial attention unit connected in sequence; The channel attention unit comprises: a channel attention unit input end, a first branch, a second branch and a channel attention unit output end; An input end of the channel attention unit is connected to an input end of the first multiplier through a first branch; The input end of the channel attention unit is connected to the input end of the first multiplier through the second branch; The second branch includes: a dilated convolution layer p1, a dilated convolution layer p2, a dilated convolution layer p3, a dilated convolution layer p4, and a convolution layer connected in sequence; the dilated convolution layer p1 is connected to the input end of the channel attention unit, and the convolution layer is connected to the input end of the first multiplier; The output of the first multiplier is connected to the output of the channel attention unit; The spatial attention unit comprises: a spatial attention unit input end, a third branch, a fourth branch and a spatial attention unit output end; The third branch includes: a maximum pooling layer, a second multiplier, a fully connected layer, an activation function layer J2 and a third multiplier connected in series in sequence; The fourth branch includes: an average pooling layer and an activation function layer J3 connected in series; The input end of the maximum pooling layer and the input end of the average pooling layer are both connected to the input end of the spatial attention unit; the output end of the third multiplier is connected to the output end of the spatial attention unit; the output end of the activation function layer J3 is connected to the input end of the second multiplier; the input end of the spatial attention unit is connected to the input end of the third multiplier.
4. The animal gender identification method based on deep learning as claimed in claim 1, characterized in that: The first multi-scale aggregation module includes: The obtained feature map is subjected to three parallel multi-scale convolution operations: (1) Perform dilated convolution operations on the feature map, the convolution layer q1, and the dilated convolution layer p5 in sequence, filling the edges of the feature map with 0 elements to obtain the first output value; the convolution kernel of the convolution layer q1 is 3*3; the convolution kernel of the dilated convolution layer is 3*3, and the dilation coefficient is (1, 2); (2) Perform dilated convolution operations on the feature map, the convolution layer q2, and the dilated convolution layer p6 in sequence, filling the edges of the feature map with 0 elements to obtain the second output value; the convolution kernel of the convolution layer q2 is 3*3; the convolution kernel of the dilated convolution layer is 5*5, and the dilation coefficient is (1, 2); (3) Perform dilated convolution operations on the feature map, the convolution layer q3, and the dilated convolution layer p7 in sequence, filling the edges of the feature map with 0 elements to obtain the third output value; the convolution kernel of the convolution layer q3 is 3*3; the convolution kernel of the dilated convolution layer p7 is 7*7, and the dilation coefficient is (1, 2); The obtained first output value, second output value, and third output value are added together element by element.
5. The animal gender identification method based on deep learning as claimed in claim 1, characterized in that: The basic feature extraction of the animal face image to be identified is performed to obtain a basic feature map, which is achieved by a first feature extraction module and a second feature extraction module; The deep feature extraction of the basic feature map to obtain deep feature maps at different stages is achieved by a third feature extraction module, a fourth feature extraction module and a fifth feature extraction module; The feature enhancement of the deep feature maps at different stages is performed to obtain enhanced feature maps at different stages, which is achieved by the first attention module, the second attention module and the third attention module; The optimized feature map is subjected to two rounds of multi-scale aggregation and pooling operations to obtain the final feature map, which is achieved by a first multi-scale aggregation module, a first pooling layer, a convolutional layer C2, a second multi-scale aggregation module, a second pooling layer and a convolutional layer C3; The final feature map is classified to obtain the animal gender classification result, which is achieved through the first fully connected layer, the second fully connected layer and the activation function layer J1.
6. The animal gender identification method based on deep learning as claimed in claim 1, characterized in that: The training process of the trained animal gender recognition model includes: Constructing a training set, the training set comprising: animal facial images of known animal genders; Input the training set into the animal gender recognition model and train the model using the Adam optimizer. When the loss function value of the model no longer decreases, the training is stopped to obtain the trained animal gender recognition model; Using a nonlinear dimensionality reduction algorithm, the images of sheep of different genders are reduced to two-dimensional images to find the distinction between rams and ewes in the images, and the images are used as the basis for gender classification; Using a gradient-weighted class activation mapping visualization algorithm, we visualized the key areas that the network focused on when distinguishing the sex of sheep, and found that the recognition areas for rams and ewes were different. The loss function of the model includes: The calculation formula of the cross entropy loss function CrossEntropy Loss is: The predicted probabilities for each class are p and 1-p respectively, where Indicates the label of sample i, the positive class is 1 and the negative class is 0; It represents the probability that sample i is predicted to be positive.
7. Animal gender recognition system based on deep learning, characterized by: include: An acquisition module is configured to: acquire a facial image of an animal to be identified; The recognition module is configured to: input the animal face image to be recognized into the trained animal gender recognition model and output the animal gender recognition result; The trained animal gender recognition model is used to: extract basic features from the facial image of the animal to be recognized to obtain a basic feature map; perform deep feature extraction on the basic feature map to obtain deep feature maps at different stages; perform feature enhancement on the deep feature maps at different stages to obtain enhanced feature maps at different stages; perform element-by-element addition on the enhanced feature maps at different stages to obtain an optimized feature map; perform two rounds of multi-scale aggregation and pooling operations on the optimized feature map to obtain a final feature map; perform classification processing on the final feature map to obtain an animal gender classification result; The trained animal gender recognition model comprises: A first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module, and a fifth feature extraction module connected in sequence; The output of the third feature extraction module is connected to the input of the first attention module; The output of the fourth feature extraction module is connected to the input of the first transposed convolutional layer, the output of the first transposed convolutional layer is connected to the input of the second transposed convolutional layer, and the output of the second transposed convolutional layer is connected to the input of the second attention module; The output of the fifth feature extraction module is connected to the input of the third transposed convolutional layer, the output of the third transposed convolutional layer is connected to the input of the fourth transposed convolutional layer, and the output of the fourth transposed convolutional layer is connected to the input of the third attention module; The output of the first attention module, the output of the second attention module, and the output of the third attention module are all connected to the input of the splicing unit; the splicing unit adds the feature maps output by the three attention modules element by element; The output end of the splicing unit is connected to the input end of the convolutional layer C1; the output end of the convolutional layer is connected to the input end of the first multi-scale aggregation module, the output end of the first multi-scale aggregation module is connected to the input end of the first pooling layer, the output end of the first pooling layer is connected to the input end of the convolutional layer C2, the output end of the convolutional layer C2 is connected to the input end of the second multi-scale aggregation module, the output end of the second multi-scale aggregation module is connected to the input end of the second pooling layer, the output end of the second pooling layer is connected to the input end of the convolutional layer C3, the output end of the convolutional layer C3 is connected to the input end of the first fully connected layer, the output end of the first fully connected layer is connected to the input end of the second fully connected layer, and the output end of the second fully connected layer is connected to the input end of the activation function layer J1.
8. An electronic device, comprising: a memory for non-transitory storage of computer-readable instructions; as well as a processor for executing said computer-readable instructions, When the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 6 is executed.
9. A storage medium, characterized in that: Computer-readable instructions are non-transitory stored, wherein when the non-transitory computer-readable instructions are executed by a computer, the instructions of the method according to any one of claims 1 to 6 are executed.