Animal identification method and device

Abstract

The invention discloses an animal identification method and device, and the method comprises the steps: employing an improved YOLO network model for detection, obtaining the multi-scale features of a to-be-analyzed image based on a backbone network, the multi-scale features at least comprise a first scale feature map and a second scale feature map, and the scale of the first scale feature map is larger than that of the second scale feature map; inputting the first scale feature map and the second scale feature map into a cross attention network, outputting a first cross attention feature map and a second cross attention feature map, and performing feature fusion on the first cross attention feature map and the second cross attention feature map to obtain a fused feature map; and inputting the fused feature map into a detection head, when the face and the nose of the animal in the to-be-analyzed image are detected at the same time, outputting a face image and a nose image of the animal, and performing animal identification based on the face image and the nose image of the animal. The method can detect the face and nose of the animal in the image at the same time, and greatly improves the accuracy of animal recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an animal recognition method and device. BACKGROUND

[0002] Facial recognition of pets using machine learning can achieve functions such as tracking of lost pets and confirmation of the identity of the owner. Taking dog detection as an example, dog face and dog nose detection are an important part of dog recognition. The inventors have found that in actual applications, the following situations often occur: the size of the dog face and the dog nose differs greatly, resulting in a situation in which the dog nose is moderate and the dog face is out of the frame in the captured image, or the dog face is moderate and the dog nose is too small in the image; and when the image is captured, the dog face or the dog nose is often blocked because the dog is generally in a moving state. These situations result in abnormal situations such as dog face detection but no dog nose detection, no dog face detection but dog nose detection, dog face and dog nose detection but the relative positions are obviously incorrect, and single-class isolated false detection in image detection, and therefore the current animal recognition result accuracy is often low. SUMMARY

[0003] The purpose of the present application is to provide an animal recognition method that can simultaneously detect the face and nose of an animal in an image, thereby greatly improving the accuracy of animal recognition.

[0004] To achieve the above purpose, the present application provides an animal recognition method, which comprises the following steps:

[0005] The improved YOLO network model is used to detect animals, and the improved YOLO network model comprises a backbone network, a feature fusion network, and a detection head. The feature fusion network adopts a cross-attention network, and multi-scale features of a to-be-analyzed image are obtained based on the backbone network. The multi-scale features at least comprise a first scale feature map and a second scale feature map, and the scale of the first scale feature map is greater than that of the second scale feature map.

[0006] The first scale feature map and the second scale feature map are input into the cross-attention network to output a first cross-attention feature map and a second cross-attention feature map. The first cross-attention feature map and the second cross-attention feature map are subjected to feature fusion to obtain a fused feature map.

[0007] The fused feature map is input into the detection head, and when the face and nose of an animal in the to-be-analyzed image are simultaneously detected, the face image and the nose image of the animal are output.

[0008] Based on the face image and the nose image of the animal, animal recognition is performed.

[0009] Further, the step of obtaining the multi-scale features of the to-be-analyzed image based on the backbone network comprises the following steps:

[0010] performing convolution and down-sampling operations on the image to be analyzed to output a first scale feature map, the scale of the first scale feature map being N times smaller than the image to be analyzed;

[0011] performing convolution and down-sampling operations on the first scale feature map to output a second scale feature map, the scale of the second scale feature map being 2N times smaller than the image to be analyzed.

[0012] Further, the cross-attention network comprises a first cross-attention layer, a first feedforward neural network layer, a second cross-attention layer and a second feedforward neural network layer, and the step of inputting the first scale feature map and the second scale feature map into the cross-attention network to output a first cross-attention feature map and a second cross-attention feature map comprises:

[0013] The first cross-attention layer transforms the first scale feature map to obtain a corresponding first query matrix, a first key matrix and a first value matrix;

[0014] The second cross-attention layer transforms the second scale feature map to obtain a corresponding second query matrix, a second key matrix and a second value matrix;

[0015] The first cross-attention layer determines a first attention matrix based on the second query matrix and the first key matrix, normalizes the first attention matrix, and performs matrix multiplication on the normalized first attention matrix and the first value matrix to obtain a first cross-attention weighted feature, the first cross-attention weighted feature being input into the first feedforward neural network layer for processing to obtain the first cross-attention feature map;

[0016] The second cross-attention layer determines a second attention matrix based on the first query matrix and the second key matrix, normalizes the second attention matrix, and performs matrix multiplication on the normalized second attention matrix and the second value matrix to obtain a second cross-attention weighted feature, the second cross-attention weighted feature being input into the second feedforward neural network layer for processing to obtain the second cross-attention feature map.

[0017] Further, the step of fusing the first cross-attention feature map and the second cross-attention feature map to obtain a fused feature map comprises:

[0018] performing up-sampling operations on the second cross-attention feature map to output an up-sampled second cross-attention feature map, so that the scales of the first cross-attention feature map and the up-sampled second cross-attention feature map are consistent;

[0019] fusing the first cross-attention feature map and the up-sampled second cross-attention feature map to obtain a fused feature map.

[0020] Further, based on the face image and the nose image of the animal, animal recognition is performed, including:

[0021] The face image and the nose image of the animal are input into a neural network to extract the face feature of the animal and the nose feature of the animal;

[0022] The breed of the animal is confirmed based on the face feature of the animal;

[0023] Noseprint recognition is performed based on the nose feature of the animal to determine the identity information of the animal.

[0024] Further, the method further includes:

[0025] According to the constraint relationship of the number of face detection boxes and the number of nose detection boxes, the size constraint relationship of the face detection box and the nose detection box, and the constraint relationship of the relative position between the face detection box and the nose detection box, a result consistency loss function is determined;

[0026] According to the result consistency loss function, the category and confidence loss function, and the detection box regression loss function, a target loss function of the improved YOLO network model is determined, and model training is performed based on the target loss function to ensure that the improved YOLO network model can simultaneously detect the face and nose of the animal in the image to be analyzed.

[0027] Further, the method further includes:

[0028] According to the constraint relationship of the number of face detection boxes and the number of nose detection boxes, the size constraint relationship of the face detection box and the nose detection box, and the constraint relationship of the relative position between the face detection box and the nose detection box, a result consistency loss function L consit is determined, which is expressed as:

[0029]

[0030] wherein, represents the probability corresponding to the i-th face detection box, represents the probability of the j-th nose detection box, fbox_w i represents the width of the i-th face detection box, fbox_h i represents the height of the i-th face detection box, nbox_w j represents the width of the j-th nose detection box, nbox_h j represents the height of the j-th nose detection box, fbox_w gt represents the true value of the face width, fbox_h gt represents the true value of the face height, nbox_w gt represents the true value of the nose width, nbox_h gtfceteri represents the real value of the height of the nose, i fcenter represents the center point of the i-th face detection frame, j fcenter represents the center point of the j-th nose detection frame, gt fceteri represents the real value of the center point of the face, gt fceteri represents the real value of the center point of the nose,

[0031] According to the result consistency loss function L consit , the category and confidence loss function L class , and the detection frame regression loss function L regress Determine the target loss function L of the improved YOLO network model detect , based on the target loss function L detect Model training is carried out to ensure that the improved YOLO network model can simultaneously detect the face and nose of the animal in the image to be analyzed, and the target loss function L detect is expressed as:

[0032] L detect = L class + L regress + L consist .

[0033] Based on the above purpose, the application provides an animal recognition device, which comprises:

[0034] The feature extraction module is used to detect animals by using the improved YOLO network model, and the improved YOLO network model comprises a backbone network, a feature fusion network and a detection head. The feature fusion network adopts a cross-attention network, and the multi-scale features of the image to be analyzed are obtained based on the backbone network. The multi-scale features at least include a first scale feature map and a second scale feature map, and the scale of the first scale feature map is larger than that of the second scale feature map.

[0035] The feature fusion module is used to input the first scale feature map and the second scale feature map into the cross-attention network, output the first cross-attention feature map and the second cross-attention feature map, and fuse the first cross-attention feature map and the second cross-attention feature map to obtain the fused feature map.

[0036] The detection head module is used to input the fused feature map into the detection head, and when the face and nose of the animal in the image to be analyzed are simultaneously detected, the face image and the nose image of the animal are output.

[0037] The recognition module is used to recognize the animal based on the face image and the nose image of the animal.

[0038] Further, the detection head module is specifically used for:

[0039] According to the constraint relationship of the obtained number of face detection boxes and the number of nose detection boxes, the size constraint relationship of the face detection box and the nose detection box, and the constraint relationship of the relative position between the face detection box and the nose detection box, a result consistency loss function is determined;

[0040] According to the result consistency loss function, the category and confidence loss function, and the detection box regression loss function, a target loss function of the improved YOLO network model is determined, and model training is performed based on the target loss function, so as to ensure that the improved YOLO network model can simultaneously detect the face and nose of the animal in the image to be analyzed.

[0041] Based on the above purpose, the present application provides a computer device, comprising:

[0042] A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;

[0043] A memory for storing a computer program;

[0044] A processor for executing the computer program stored on the memory to realize the steps of the method described above.

[0045] The present application improves the YOLO model, uses a cross attention network as a feature fusion network layer, realizes attention learning between targets of different scales and different categories, strengthens the correlation between the face and nose of the animal in the image, can simultaneously detect the face and nose in the image, and improves the accuracy of animal recognition; under the premise that the model calculation amount does not increase significantly, the detection accuracy of multiple highly correlated targets is greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 is a first flowchart of an animal recognition method according to an embodiment of the present application;

[0047] Figure 2 is a structure diagram of an improved YOLO network model according to an embodiment of the present application;

[0048] Figure 3 is a second flowchart of an animal recognition method according to an embodiment of the present application;

[0049] Figure 4 is a structure diagram of a cross attention network according to an embodiment of the present application;

[0050] Figure 5 is a third flowchart of an animal recognition method according to an embodiment of the present application;

[0051] Figure 6 This is a schematic diagram of a feature fusion method provided according to an embodiment of this application;

[0052] Figure 7 This is the fourth flowchart of the animal identification method provided according to the embodiments of this application;

[0053] Figure 8 This is a system block diagram of an animal identification device provided according to an embodiment of this application;

[0054] Figure 9 This is a schematic diagram of the structure of a computer device provided according to an embodiment of this application. Detailed Implementation

[0055] The present application will be described in detail below with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present application. Any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the protection scope of the present application.

[0056] Please refer to Figure 1 This application provides an animal identification method. The identification method includes steps S101-S104.

[0057] Step S101: An improved YOLO network model is used to detect animals. The improved YOLO network model includes a backbone network, a feature fusion network, and a detection head. The feature fusion network adopts a cross-attention network. Based on the backbone network, multi-scale features of the image to be analyzed are obtained. The multi-scale features include at least a first-scale feature map and a second-scale feature map. The scale of the first-scale feature map is larger than that of the second-scale feature map.

[0058] This application employs an improved YOLO network model to detect animals in images, such as... Figure 2 The diagram shows the structure of the improved YOLO network model. The improved YOLO network model includes a backbone network, a feature fusion network, and a detection head. This application improves the feature fusion network of the YOLO network model, employing a cross-attention network. The YOLO model is an object recognition and localization algorithm based on deep neural networks, its most notable feature being its high speed. YOLO directly uses a single convolutional neural network to predict multiple bounding boxes (rectangular boxes containing a certain object) and class probabilities.

[0059] The image to be analyzed is acquired by a camera, and the face and nose of the animal in the image to be analyzed are target objects to be detected. The backbone network includes multiple convolutional networks, such as conv3*3 convolution, res-block, and the like, which extract rich semantic features in the input image based on the convolutional network. In order to improve the accuracy of target object detection in the image, multi-scale features are obtained in the image to be analyzed, wherein the multi-scale features at least include a first scale feature map and a second scale feature map, the scale of the first scale feature map is larger than that of the second scale feature map, and the multi-scale feature maps are fused in the subsequent target detection process to detect the target based on the fused feature map. Multi-scale feature processing refers to processing the image to be analyzed at different scales to obtain a feature map representation of the corresponding scale. In the multi-scale feature, each scale refers to a different degree of scaling or sampling of the image to be analyzed to obtain a representation of different sizes. For example, the multi-scale feature can include, but is not limited to, feature maps of multiple scales of 4 times, 8 times, 16 times, 32 times, 64 times, etc.

[0060] In step S102, the first scale feature map and the second scale feature map are input into the cross-attention network to output a first cross-attention feature map and a second cross-attention feature map, and the first cross-attention feature map and the second cross-attention feature map are fused to obtain a fused feature map.

[0061] The cross-attention network calculates the correlation between two different input feature maps to capture the relationship between them. The first scale feature map and the second scale feature map are input into the cross-attention network to obtain a first cross-attention feature and a second cross-attention feature that fuse the characteristics of the first scale feature map and the second scale feature map. By adjusting the weight of the feature through the cross-attention mechanism, the attention can be focused on the important feature part, so that the obtained attention feature not only has global self-attention feature, but also has attention feature between different scales, which strengthens the correlation between targets of different scales and improves the effect of feature fusion.

[0062] Fusing feature maps of different scales can further strengthen the relevance between targets of different scales and improve the detection ability of different scales.

[0063] For example, the feature fusion processing mode directly adds the first cross-attention feature map and the second cross-attention feature map to obtain the fused feature map.

[0064] For example, the feature fusion processing mode directly adds the first cross-attention feature map and the second cross-attention feature map and then performs convolution processing to obtain the fused feature map.

[0065] Step S103, input the fused feature map into the detection head, and output the face image and the nose image of the animal when the face and the nose of the animal in the image to be analyzed are detected simultaneously.

[0066] The detection head includes some convolutional layers, pooling layers and fully connected layers to perform target detection on the input fused feature map. The detection head outputs the face detection frame, the face class confidence and the face detection frame regression value in the image to be analyzed, and outputs the nose detection frame, the nose class confidence and the nose detection frame regression value. Different target objects in the image to be analyzed have different feature information, and the detection head in the YOLO model performs target detection to output the detection frame containing the target object and the class confidence of the target object, which is used to determine whether each target object is the face or the nose of the animal, and further determine the type of the animal. For a certain target object, each target frame containing the target object can be taken as a candidate target frame, and the detection head calculates the class confidence of the target object in each candidate target frame according to a preset confidence algorithm. The information of each candidate target frame can include (x, y, w, h) and class confidence, a total of 5 pieces of information, wherein x and y represent the coordinates of the center of the target object contained in the target frame, and w and h are the width and height of the target frame. From the multiple candidate target frames, the one with the maximum class confidence, that is, the one with the most image features of the target object, is determined as the detection frame containing the target object. Therefore, by inputting the fused feature map into the detection head, the face detection frame, the face class confidence and the face detection frame regression value in the image to be analyzed are obtained, and the nose detection frame, the nose class confidence and the nose detection frame regression value are obtained. The face image is obtained by image cropping based on the obtained face detection frame. The nose image is obtained by image cropping based on the obtained nose detection frame.

[0067] Illustratively, the detection head usually predicts the position information of the target object through a grid algorithm or an anchor box algorithm. Illustratively, the grid algorithm divides the input image into N*N grids, wherein if the center of a target falls in the grid, the grid is responsible for predicting the target, and each grid predicts multiple target frames. In addition to regressing the position of the target frame itself, each target frame also needs to attach a confidence value representing the probability of containing the target object in the predicted target frame. Illustratively, the anchor box algorithm predicts the matching degree between each anchor box and the target object according to the image features and the information of the anchor box, generates the corresponding confidence and class probability by pre-setting some anchor boxes, which are rectangular frames according to different sizes, width-height ratios and length-width ratios.

[0068] For example, in order to select a candidate target frame that can best reflect the image features of the target object from multiple candidate target frames, a preset NMS (Non-Maximum Suppression) processing algorithm can be used to search for multiple candidate target frames of a target object, and an optimal solution is searched to select a candidate target frame that contains the most image features of the target object as the output detection frame.

[0069] In this embodiment, when the face and the nose in the image to be analyzed are detected at the same time, the face image and the nose image of the animal are output, so that the accuracy of the target object recognition in the image can be further guaranteed.

[0070] In step S104, animal recognition is performed based on the face image and the nose image of the animal.

[0071] The image recognition is performed based on the obtained face image and nose image, so that the animal information in the image to be analyzed is obtained.

[0072] In this embodiment, the YOLO model is improved, and a cross-attention network is used as a feature fusion network to realize attention learning between targets of different scales and different categories, and the correlation between the face and the nose of the animal in the image is strengthened, so that the abnormal situation that the face is detected but the nose is not detected, the face is not detected but the nose is detected, and the face and the nose are both detected but the relative position is obviously not equal can be reduced, the face and the nose of the animal in the image can be detected at the same time, the accuracy of animal recognition is improved, and the detection accuracy of multiple categories of highly related targets is greatly improved without significant increase in model calculation amount.

[0073] As an optional implementation manner, as shown in Figure 3 The step of obtaining the multi-scale features of the image to be analyzed based on the backbone network includes:

[0074] In S301, convolution and down-sampling operations are performed on the image to be analyzed to output a first scale feature map, and the scale of the first scale feature map is N times smaller than the image to be analyzed.

[0075] The first scale feature map is output by a continuous convolution layer and a down-sampling operation on the image to be analyzed. For example, the backbone network includes a plurality of convolution layers and pooling layers, and realizes down-sampling processing of the image to be analyzed. Taking N=16 as an example, the backbone network includes five convolution layers and pooling layers. The image to be analyzed is processed by the first convolution layer and the first pooling layer to obtain a 2-fold down-sampled feature map, processed by the second convolution layer and the second pooling layer to obtain a 4-fold down-sampled feature map, processed by the third convolution layer and the third pooling layer to obtain an 8-fold down-sampled feature map, processed by the fourth convolution layer and the fourth pooling layer to obtain a 16-fold down-sampled feature map.

[0076] In S302, the first scale feature map is processed by a convolution layer and a down-sampling operation to output a second scale feature map, and the scale of the second scale feature map is 2N times smaller than that of the image to be analyzed.

[0077] As in the embodiment of S301, the first scale feature map is processed by a convolution layer and a down-sampling operation to output a second scale feature map. Taking N=16 as an example, on the basis of S302, the 16-fold down-sampled feature map is processed by the fifth convolution layer and the fifth pooling layer to obtain a 32-fold down-sampled feature map.

[0078] According to the above-mentioned embodiments, the first scale feature map and the second scale feature map can be obtained by processing the image to be analyzed, and the scale of the first scale feature map is larger than that of the second scale feature map. It can be understood that down-sampling will cause the resolution of the input image to decrease, thereby causing the loss of detailed information. Therefore, in the first scale feature map and the second scale feature map, the first scale feature map retains more features of small targets in the image, and the second scale feature map retains more features of large targets in the image, and the details of the target objects in the image are obtained from the feature maps of different scales. Taking the face and nose in the image of a dog as an example, the first scale feature map retains more features of the nose, thereby enhancing the extraction ability of the nose features, and the second scale feature map retains more features of the face, thereby enhancing the extraction ability of the face features.

[0079] As an optional implementation, the cross-attention network can be a cross-attention (Cross-attention) network in a Transformer model. As shown in FIG. 3, the cross-attention network includes a query (Query) layer, a key (Key) layer, and a value (Value) layer. Figure 4The structure diagram of the cross attention network is shown, and the cross attention network includes a first cross attention layer, a first feedforward neural network layer, a second cross attention layer, and a second feedforward neural network layer. The first cross attention layer corresponds to an input layer corresponding to an input first scale feature map. The input layer and the first cross attention layer further include position encoding. The first cross attention layer and the first feedforward neural network layer further include a residual and normalization layer (Add&Norm). Add represents a residual connection for preventing network degradation. Norm represents Layer Normalization for normalizing the activation value of each layer. The first feedforward neural network layer is used for nonlinear transformation of the feature to increase the expression ability of the model, thereby improving the performance of the model in sequence modeling and semantic understanding tasks. The first feedforward neural network layer further includes a residual and normalization layer. The second cross attention layer corresponds to an input layer corresponding to an input second scale feature map. The input layer and the second cross attention layer further include position encoding. The second cross attention layer and the second feedforward neural network layer further include a residual and normalization layer (Add&Norm). The second feedforward neural network layer is used for nonlinear transformation of the feature to increase the expression ability of the model, thereby improving the performance of the model in sequence modeling and semantic understanding tasks. The second feedforward neural network layer further includes a residual and normalization layer.

[0080] As an optional implementation manner, as shown in Figure 5 and Figure 6 , the step of inputting the first scale feature map and the second scale feature map into the cross attention network and outputting the first cross attention feature map and the second cross attention feature map includes:

[0081] S501, the first cross attention layer linearly transforms the first scale feature map to obtain a corresponding first query matrix Q1, a first key matrix K1, and a first value matrix V1.

[0082] The query vector in the attention mechanism usually represents the information that needs to be focused or queried at present. The key vector is information in the data set for comparison with the query vector. The value vector contains the actual information. In cross attention, the linear layer weights of the query vector and the key vector are shared, and the weights of the value vector are different. In the first cross attention layer, three weight matrices are defined, corresponding to the first query vector weight matrix W1 Q , the first key vector weight matrix W1 K , and the first value vector weight matrix W1 V . These weight matrices are obtained through model training. The embedding vector of each pixel in the first scale feature map is calculated with the first query vector weight matrix W1 Q , the first key vector weight matrix W1K , the first value vector weight matrix W1 V , the first key matrix K1 and the first value matrix V1.

[0083] S502, the second cross-attention layer linearly transforms the second scale feature map to obtain the corresponding second query matrix Q2, the second key matrix K2 and the second value matrix V2.

[0084] Similarly, in the second cross-attention layer, the second query vector weight matrix W2 Q , the second key vector weight matrix W2 K , the second value vector weight matrix W2 V is defined. By calculating the embedding vector of each pixel in the second scale feature map respectively, the product of the second query matrix weight matrix W2 Q , the second key matrix weight matrix W2 K , the second value matrix weight matrix W2 V is obtained. The corresponding second query matrix Q2, the second key matrix K2 and the second value matrix V2.

[0085] S503, the first cross-attention layer determines the first attention matrix M1 based on the second query matrix Q2 and the first key matrix K1, normalizes the first attention matrix M1, and performs matrix multiplication between the normalized first attention matrix and the first value matrix V1 to obtain the first cross-attention weighted feature. The first cross-attention weighted feature is input into the first feedforward neural network layer for processing to obtain the first cross-attention feature map T1.

[0086] The first attention matrix M1 can be obtained by matrix multiplication between the second query matrix Q2 and the first key matrix K1. The attention matrix reflects the similarity or correlation between the query matrix and the key matrix. The first attention matrix M1 is normalized by the softmax function to obtain the normalized first attention matrix. The normalized first attention matrix is multiplied with the first value matrix V1 to obtain the first cross-attention weighted feature, which is realized by the following formula:

[0087]

[0088] Wherein, d k represents the dimension of the key vector, wherein denotes a scaling factor, and Attention(Q, K, V) denotes an attention feature. The formula calculates the mutual dependency between different tokens using the K, Q, and V matrices. Q and K can be used to calculate the similarity between the current token and other tokens, and the similarity is used as a weight to perform weighted summation on V, and the result can be used as the token of the next layer. K and V come from the first scale feature map, and Q comes from the second scale feature map. The first feedforward neural network layer is used as a linear transformation layer to process the first cross-attention weighted feature to obtain the first cross-attention feature map T1. The feedforward neural network layer can be composed of two linear transformation layers, with a nonlinear activation function (usually ReLU) in between. The feature vector of each position of the input sequence is taken as input, and is mapped to a higher-dimensional space through two linear transformations, and then is nonlinearly transformed through the activation function, and is mapped back to the original feature dimension through another linear transformation.

[0089] In S504, the second cross-attention layer determines a second attention matrix M2 based on the first query matrix Q1 and the second key matrix K2, normalizes the second attention matrix M2, and performs matrix multiplication on the normalized second attention matrix and the second value matrix V2 to obtain a second cross-attention weighted feature. The second cross-attention weighted feature is input into a second feedforward neural network layer for processing to obtain a second cross-attention feature map T2.

[0090] Similarly, the second attention matrix M2 can be obtained by performing matrix multiplication on the first query matrix Q1 and the second key matrix K2. The second attention matrix M2 is normalized by a softmax function to obtain a normalized second attention matrix. The normalized second attention matrix is multiplied with the second value matrix V2 to obtain a second cross-attention weighted feature. The second cross-attention weighted feature is processed by a second feedforward neural network layer as a linear transformation layer to obtain a second cross-attention feature map T2.

[0091] In this embodiment, the first scale feature map and the second scale feature map are input into the cross-attention network to obtain a first cross-attention feature and a second cross-attention feature. The first cross-attention feature and the second cross-attention feature both fuse the characteristics of the first scale feature map and the characteristics of the second scale feature map, increase the feature attention between different scale features, and can improve the correlation between different scale targets, thereby accurately detecting the face and nose in the image to be analyzed. The cross-attention mechanism in the Transformer model can reduce the number of model parameters in the network and reduce the complexity of model operation.

[0092] As an optional implementation manner, the step of performing feature fusion on the first cross-attention feature map and the second cross-attention feature map to obtain a fused feature map comprises:

[0093] The second cross-attention feature map is subjected to an upsampling operation to output an upsampling-processed second cross-attention feature map, so that the first cross-attention feature map and the upsampling-processed second cross-attention feature map have the same scale;

[0094] The first cross-attention feature map and the upsampling-processed second cross-attention feature map are subjected to feature fusion to obtain a fused feature map.

[0095] Because the first scale feature map and the second scale feature map have different scales, the first cross-attention feature map and the second cross-attention feature map also have different scales, and the second cross-attention feature map needs to be subjected to an upsampling operation so that the scales of the two are consistent. For example, the second cross-attention feature map is restored to the same size as the first cross-attention feature map through convolution and upsampling operation (upsampling by 2 times), and the first cross-attention feature map and the second cross-attention feature map subjected to upsampling by 2 times are subjected to feature fusion to obtain a fused feature map.

[0096] As an optional implementation manner, as shown in Figure 7 the step of performing animal recognition based on the face image and the nose image of the animal comprises:

[0097] In step S701, the face image and the nose image of the animal are input into a neural network to extract the face feature of the animal and the nose feature of the animal.

[0098] The obtained face image and nose image of the animal are subjected to image size processing, for example, image scaling processing, to scale the images to a suitable size, and the face image and the nose image subjected to the image scaling processing are input into a pre-trained neural network to obtain the face feature and the nose feature of the animal through face feature extraction and nose feature extraction of the neural network. The neural network includes but is not limited to a deep neural network such as ResNet and MobileNet.

[0099] In step S702, the breed of the animal is determined based on the face feature of the animal.

[0100] The breed of the animal is determined based on the face feature of the animal. For example, the face feature of the animal is compared with the animal breed feature stored in a database to determine which breed the animal belongs to, for example, whether the animal is a dog or a cat. For example, the feature comparison method can use a cosine similarity calculation method. For example, animal 1 is determined to be a dog and a breed of a country dog, and animal 2 is determined to be a cat and a breed of a Liuhua cat.

[0101] Step S703, based on the nose features of the animal, noseprint recognition is performed to determine the identity information of the animal.

[0102] The identity information of the animal is determined by performing noseprint recognition on the animal. For example, the noseprint features of the animal are compared with the noseprint features stored in the database to determine the identity information of the animal. The database stores a plurality of noseprint features and an ID number corresponding to each noseprint feature, which is used to uniquely identify the identity information of the animal. For example, the ID number is a dog certificate, and the dog certificate number is obtained through noseprint recognition. The corresponding owner identity information can be obtained through the dog certificate number, which realizes the management and tracking of pet dogs, and facilitates the search for lost pet dogs. For example, the feature comparison method can use the cosine similarity calculation method.

[0103] In the model training process, the relevant loss function needs to be calculated to better train the model. The loss function of the YOLO network model is improved in this application. In addition to the basic loss function of the YOLO network model, consistency constraints are also made for the face and nose detected in the image at the same time. In an embodiment of the present application, according to the constraint relationship between the number of face detection boxes and the number of nose detection boxes, the size constraint relationship between the face detection box and the nose detection box, and the constraint relationship between the relative position of the face detection box and the nose detection box, the result consistency loss function is determined; according to the result consistency loss function, the class and confidence loss function, and the detection box regression loss function, the target loss function of the improved YOLO network model is determined, and the model is trained based on the target loss function to ensure that the improved YOLO network model can simultaneously detect the face and nose of the animal in the image to be analyzed. Adjust the model parameters of the YOLO network model based on the target loss function, thereby optimizing the overall performance of the model. In this embodiment, the loss function of the YOLO network model is improved, and whether the face and nose are detected at the same time, whether the relative size ratio of the face and nose detection boxes meets the expectation, and whether the relative position of the face and nose detection boxes meets the expectation are constrained, which further ensures that the face and nose in the image are simultaneously detected, improves the detection accuracy of the model, and reduces the model false detection rate.

[0104] For convenience of calculation, the number, average position and average size of the detection box are used as statistics to calculate the result consistency loss function. For example, according to the constraint relationship between the number of face detection boxes and the number of nose detection boxes, the size constraint relationship between the face detection box and the nose detection box, and the constraint relationship between the relative position of the face detection box and the nose detection box, the result consistency loss function L consit is represented as:

[0105]

[0106] wherein, represents the probability corresponding to the i-th face detection frame, represents the probability of the j-th nose detection frame, the first term of the formula indicates that the number of faces and noses needs to be consistent in the weighted statistical sense, fbox_w i represents the width of the i-th face detection frame, fbox_h i represents the height of the i-th face detection frame, the two additions represent the size of the face detection frame, nbox_w j represents the width of the j-th nose detection frame, nbox_h j represents the height of the j-th nose detection frame, the two additions represent the size of the nose detection frame, fbox_w gt represents the true value of the face width, fbox_h gt represents the true value of the face height, nbox_w gt represents the true value of the nose width, nbox_h gt represents the true value of the nose height, the second term of the formula indicates that the difference between the average size of the face detection frame and the average size of the nose detection frame should be equal to the difference in size between the true values of the face and the nose, fcenter i represents the center point of the i-th face detection frame, ncenter j represents the center point of the j-th nose detection frame, fcenter gt represents the true value of the face center point, ncenter gt represents the true value of the nose center point, the second term of the formula indicates that the relative position of the face and nose detection frames should be the same as the true relative position of the face and nose. The model usually has redundant predictions before NMS post-processing, and the detection frames shown in the formula are all detection frames before NMS processing.

[0107] According to the result consistency loss function L consit , the category and confidence loss function L class , and the detection frame regression loss function L regress , the target loss function L detect of the improved YOLO network model is determined, and the model is trained based on the target loss function L detect to ensure that the improved YOLO network model can simultaneously detect the face and nose of the animal in the image to be analyzed, and the target loss function L detect is represented as:

[0108] L detect = L class + L regress + L consist .

[0109] The class and confidence loss function and the bounding box regression loss function are basic loss functions of the YOLO network model. For example, the cross-entropy is used to calculate the class and confidence loss, the Sigmoiod is used to calculate the regression of the coordinates of the bounding box, and the IoU loss is used. The above formula gives one implementation of the result consistency loss function. There can be many implementations, and in other possible examples, the size constraint relationship between the face bounding box and the nose bounding box can be implemented by calculating the ratio of the sizes of the face bounding box and the nose bounding box.

[0110] As shown in one of the animal recognition devices, the device comprises: Figure 8

[0111] The feature extraction module 801 is configured to detect animals by using the improved YOLO network model, the improved YOLO network model comprises a backbone network, a feature fusion network, and a detection head, the feature fusion network adopts a cross-attention network, and the multi-scale features of the image to be analyzed are obtained based on the backbone network, the multi-scale features at least comprise a first scale feature map and a second scale feature map, the scale of the first scale feature map is larger than that of the second scale feature map;

[0112] The feature fusion module 802 is configured to input the first scale feature map and the second scale feature map into the cross-attention network, output a first cross-attention feature map and a second cross-attention feature map, and fuse the first cross-attention feature map and the second cross-attention feature map to obtain a fused feature map;

[0113] The detection head module 803 is configured to input the fused feature map into the detection head, and output the face image and the nose image of the animal when the face and the nose of the animal in the image to be analyzed are detected at the same time.

[0114] The recognition module 804 is configured to recognize the animal based on the face image and the nose image of the animal.

[0115] ​The image to be analyzed is acquired by a camera, and the feature extraction module 801 detects using an improved YOLO network model. The improved YOLO network model includes a backbone network, a feature fusion network, and a detection head. The feature fusion network uses a cross-attention network, and the multi-scale features of the image to be analyzed are obtained based on the backbone network. The backbone network includes multiple convolution networks, and the features in the input image are extracted based on the convolution networks. The feature fusion module 802 uses a cross-attention network of a Transformer model, inputs the first scale feature map and the second scale feature map into the cross-attention network, obtains the first cross-attention feature and the second cross-attention feature which are fused with the characteristics of the first scale feature map and the second scale feature map, and the correlation between targets of different scales is strengthened by the obtained cross-attention features. The detection head module 803 uses a detection head of a YOLO network model, and the detection head includes some convolution layers, pooling layers, and fully connected layers. The detection head detects the target in the image, outputs a detection frame containing the target object and a class confidence of the target object, and respectively obtains the face image and the nose image of the animal by image cropping through the obtained face detection frame and nose detection frame. The recognition module 804 inputs the face image and the nose image of the animal into a neural network that has been trained, extracts features, obtains the face feature and the nose feature of the animal, confirms the breed of the animal through the face feature of the animal, and determines the identity information of the animal through the nose feature of the animal.

[0116] The embodiment improves the YOLO model, uses a cross-attention network as a feature fusion network, realizes attention learning between targets of different scales and different categories, strengthens the correlation between the face and the nose of the animal in the image, can detect the face and the nose in the image at the same time, improves the accuracy of animal recognition, and significantly improves the detection accuracy of multiple targets with high correlation without significantly increasing the model calculation amount.

[0117] As an optional implementation manner, the feature extraction module 801 is specifically configured to:

[0118] perform convolution and down-sampling operations on the image to be analyzed, and output a first scale feature map, wherein the scale of the first scale feature map is N times smaller than the image to be analyzed;

[0119] perform convolution and down-sampling operations on the first scale feature map, and output a second scale feature map, wherein the scale of the second scale feature map is 2N times smaller than the image to be analyzed.

[0120] As an optional implementation manner, the cross-attention network includes a first cross-attention layer, a first feedforward neural network layer, a second cross-attention layer, and a second feedforward neural network layer, and the feature fusion module 702 is specifically configured to:

[0121] The first cross-attention layer linearly transforms the first scale feature map to obtain a corresponding first query matrix, a first key matrix and a first value matrix;

[0122] The second cross-attention layer linearly transforms the second scale feature map to obtain a corresponding second query matrix, a second key matrix and a second value matrix;

[0123] The first cross-attention layer determines a first attention matrix based on the second query matrix and the first key matrix, performs normalization processing on the first attention matrix, performs matrix multiplication on the normalized first attention matrix and the first value matrix to obtain a first cross-attention weighted feature, and inputs the first cross-attention weighted feature into the first feedforward neural network layer for processing to obtain a first cross-attention feature map.

[0124] The second cross-attention layer determines a second attention matrix based on the first query matrix and the second key matrix, performs normalization processing on the second attention matrix, performs matrix multiplication on the normalized second attention matrix and the second value matrix to obtain a second cross-attention weighted feature, and inputs the second cross-attention weighted feature into the second feedforward neural network layer for processing to obtain a second cross-attention feature map.

[0125] As an optional implementation, the detection head module 803 is specifically configured to:

[0126] According to the constraint relationship of the obtained number of face detection boxes and the number of nose detection boxes, the size constraint relationship of the face detection box and the nose detection box, and the constraint relationship of the relative position between the face detection box and the nose detection box, a result consistency loss function is determined;

[0127] According to the result consistency loss function, the category and confidence loss function, and the detection box regression loss function, a target loss function of the improved YOLO network model is determined, and model training is performed based on the target loss function to ensure that the improved YOLO network model can simultaneously detect the face and nose of the animal in the image to be analyzed.

[0128] The embodiment improves the loss function of the YOLO network model, and constraints whether the face and nose are simultaneously detected, whether the relative size ratio of the face detection box and the nose detection box meets the expectation, and whether the relative position of the face detection box and the nose detection box meets the expectation, thereby improving the detection accuracy of the model from multiple aspects.

[0129] The embodiment of the present application also provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, any one of the animal recognition methods described above is implemented.

[0130] Figure 9Fig. 1 is a schematic diagram of a hardware structure of a computer device provided by an embodiment of the present application. Figure 9 The computer device shown includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, and the processor 901, the communication interface 902, and the memory 903 complete communication with each other through the communication bus 904. Among them, Figure 9 The connection mode between the processor 901, the communication interface 902, and the memory 903 shown is only exemplary, and in the implementation process, the processor 901, the communication interface 902, and the memory 903 can also be connected in communication with each other in other connection modes other than the communication bus 904.

[0131] The memory 903 can be used to store a computer program, and the computer program can include instructions and data, and implement the steps of any one of the animal recognition methods described above. In the embodiments of the present application, the memory 903 can be various types of storage media, such as random access memory (RAM), read only memory (ROM), non-volatile RAM (NVRAM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, optical storage, and registers, etc. The memory 903 can include a hard disk and / or an internal memory.

[0132] The processor 901 can be a general-purpose processor, which can be a processor that reads and executes a computer program (such as a computer program) stored in a memory (such as the memory 903) to perform specific steps and / or operations, and the general-purpose processor can use data stored in the memory (such as the memory 903) in the process of executing the steps and / or operations. The general-purpose processor can be, for example but not limited to, a central processing unit (CPU). In addition, the processor 901 can also be a special-purpose processor, which can be a processor specially designed to perform specific steps and / or operations. The special-purpose processor can be, for example but not limited to, ASIC and FPGA, etc. In addition, the processor 901 can also be a combination of multiple processors, such as a multi-core processor.

[0133] The communication interface 902 can include an input / output (I / O) interface, a physical interface, and a logical interface, etc. for realizing the interconnection of devices inside the network device, and an interface for realizing the interconnection of the network device and other devices (e.g. network devices). The communication network can be an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 902 can be a module, a circuit, a transceiver, or any device capable of realizing communication.

[0134] In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 901 or the instructions in the form of software. The method disclosed in combination with the embodiments of the present application can be directly embodied as hardware processor execution completion, or executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field, such as random access memory flash, read-only memory, programmable read-only memory, electrically erasable programmable memory, register, etc. The storage medium is located in the memory 903, and the processor 901 reads the information in the memory 903, and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0135] Although the preferred embodiments of the present application have been disclosed for illustrative purposes, those skilled in the art will realize that various modifications, additions and substitutions are possible without departing from the scope and spirit of the present application disclosed by the appended claims.

Claims

1. An animal identification method, characterized in that, The method includes: An improved YOLO network model is used to detect animals. The improved YOLO network model includes a backbone network, a feature fusion network, and a detection head. The feature fusion network adopts a cross-attention network. Based on the backbone network, multi-scale features of the image to be analyzed are obtained. The multi-scale features include at least a first-scale feature map and a second-scale feature map. The scale of the first-scale feature map is larger than that of the second-scale feature map. The first scale feature map and the second scale feature map are input into the cross-attention network, and the first cross-attention feature map and the second cross-attention feature map are output. The first cross-attention feature map and the second cross-attention feature map are fused to obtain the fused feature map. The fused feature map is input into the detection head. When the face and nose of the animal in the image to be analyzed are detected simultaneously, the face image and nose image of the animal are output. Animal identification is performed based on the animal's face and nose images.

2. The animal identification method as described in claim 1, characterized in that, The step of acquiring multi-scale features of the image to be analyzed based on the backbone network includes: The image to be analyzed is subjected to convolution and downsampling operations to output a first-scale feature map, the scale of which is one-Nth of the image to be analyzed. The first-scale feature map is convolved and downsampled to output a second-scale feature map, the scale of which is 1 / 2N of the image to be analyzed.

3. The animal identification method as described in claim 1, characterized in that, The cross-attention network includes a first cross-attention layer, a first feedforward neural network layer, a second cross-attention layer, and a second feedforward neural network layer. The step of inputting the first-scale feature map and the second-scale feature map into the cross-attention network and outputting the first cross-attention feature map and the second cross-attention feature map includes: The first cross-attention layer transforms the first scale feature map to obtain the corresponding first query matrix, first key matrix, and first value matrix; The second cross-attention layer transforms the second scale feature map to obtain the corresponding second query matrix, second key matrix, and second value matrix; The first cross-attention layer determines the first attention matrix based on the second query matrix and the first key matrix, performs regression on the first attention matrix, performs matrix multiplication on the regression first attention matrix and the first value matrix to obtain the first cross-attention weighted feature, and inputs the first cross-attention weighted feature into the first feedforward neural network layer for processing to obtain the first cross-attention feature map. The second cross-attention layer determines the second attention matrix based on the first query matrix and the second key matrix, performs regression on the second attention matrix, and performs matrix multiplication between the regression second attention matrix and the second value matrix to obtain the second cross-attention weighted feature. The second cross-attention weighted feature is input into the second feedforward neural network layer for processing to obtain the second cross-attention feature map.

4. The animal identification method as described in claim 1, characterized in that, The step of fusing the first cross-attention feature map and the second cross-attention feature map to obtain a fused feature map includes: The second cross-attention feature map is upsampled to output the upsampled second cross-attention feature map so that the scales of the first cross-attention feature map and the upsampled second cross-attention feature map are consistent. The first cross-attention feature map and the upsampled second cross-attention feature map are fused to obtain a fused feature map.

5. The animal identification method as described in claim 1, characterized in that, Animal identification is performed based on the animal's face and nose images, including: The face and nose images of the animal are input into a neural network to extract the facial features and nose features of the animal; The species of the animal is identified based on its facial features; Based on the animal's nasal features, nasal print recognition is performed to determine the animal's identity information.

6. The animal identification method as described in claim 1, characterized in that, The method further includes: Based on the constraints on the number of face and nose bounding boxes, the size constraints on face and nose bounding boxes, and the constraints on the relative positions of face and nose bounding boxes, determine the result consistency loss function. Based on the result consistency loss function, category and confidence loss function, and detection box regression loss function, the target loss function of the improved YOLO network model is determined. The model is trained based on the target loss function to ensure that the improved YOLO network model can simultaneously detect the face and nose of the animal in the image to be analyzed.

7. The animal identification method as described in claim 1, characterized in that, The method further includes: Based on the constraints on the number of face and nose bounding boxes, the size constraints on face and nose bounding boxes, and the constraints on the relative positions between face and nose bounding boxes, the result consistency loss function L is determined. consit , is represented as: in, This represents the probability corresponding to the i-th face detection box. fbox_w represents the probability of the j-th nose detection box. i fbox_h represents the width of the i-th face detection bounding box. i nbox_w represents the height of the i-th face detection bounding box. j nbox_h represents the width of the j-th nose detection bounding box. j fbox_w represents the height of the j-th nose detection bounding box. gt fbox_h represents the actual width of the face. gt nbox_w represents the true height of the face. gt nbox_h represents the actual width of the nose. gt fcenter represents the true value of nose height. i ncenter represents the center point of the i-th face detection bounding box. j fcenter represents the center point of the j-th nose detection box. gt ncenter represents the true value of the center point of the face. gt This represents the actual value of the center point of the nose; Based on the consistency loss function L consit Category and confidence loss function L class and the detection box regression loss function L regress Determine the target loss function L for the improved YOLO network model. detect Based on the target loss function L detect The model is trained to ensure that the improved YOLO network model can simultaneously detect the animal's face and nose in the image to be analyzed, wherein the target loss function L... detect Represented as: L detect L class +L regress +L consist 。 8. An animal identification device, characterized in that, The device includes: The feature extraction module is used to detect animals using an improved YOLO network model. The improved YOLO network model includes a backbone network, a feature fusion network, and a detection head. The feature fusion network uses a cross-attention network to obtain multi-scale features of the image to be analyzed based on the backbone network. The multi-scale features include at least a first-scale feature map and a second-scale feature map, wherein the scale of the first-scale feature map is larger than that of the second-scale feature map. The feature fusion module is used to input the first scale feature map and the second scale feature map into the cross-attention network, output the first cross-attention feature map and the second cross-attention feature map, and fuse the first cross-attention feature map and the second cross-attention feature map to obtain the fused feature map. The detection head module is used to input the fused feature map into the detection head, and when the face and nose of the animal in the image to be analyzed are detected at the same time, the animal's face image and nose image are output. The recognition module is used to identify the animal based on its face and nose images.

9. The animal identification device as described in claim 8, characterized in that, The detection head module is specifically used for: Based on the constraints on the number of face and nose bounding boxes, the size constraints on face and nose bounding boxes, and the constraints on the relative positions of face and nose bounding boxes, determine the result consistency loss function. Based on the result consistency loss function, category and confidence loss function, and detection box regression loss function, the target loss function of the improved YOLO network model is determined. The model is trained based on the target loss function to ensure that the improved YOLO network model can simultaneously detect the face and nose of the animal in the image to be analyzed.

10. A computer device, characterized in that, include: The system includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the steps of the method as described in any one of claims 1-7.

Images