Wild animal identification method and device

Abstract

The present invention provides a method and device for identifying wild animals. The method extracts features from an image to be processed to obtain feature maps of multiple scales. After selecting several feature maps to be activated, key point prediction is performed on the feature maps of multiple scales based on a key point prediction layer to obtain key point heat maps under corresponding multiple channels. The feature map to be activated and the key point heat maps under multiple channels are then fused based on a key point representation layer to obtain key point representations of each key point. The key point representations of multiple key points in the image to be identified are then matched pairwise with the key point representations of multiple key points in a first sample image corresponding to a specific wild animal species to obtain a key point matching result of the image to be identified. Based on the key point matching result, it can be determined whether the species of the animal in the image to be identified is the specific wild animal species, thereby overcoming the interference caused by different animal motion states and improving the accuracy of wild animal identification.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method and device for identifying wild animals. Background Art

[0002] Wildlife population surveys can effectively understand population dynamics and habitat status, and are crucial for ecosystem protection. Early wildlife researchers mostly used point and line sampling to survey the habitat distribution ranges of different species. Due to the difficulty of accessing some areas, field surveys are extremely difficult, and manual surveys are time-consuming and labor-intensive, which cannot meet the needs of wildlife surveys. Later, with the rapid development of photography technology, camera traps were often used to deploy cameras within the habitat distribution range. When wild animals approached the camera, dynamic perception algorithms were automatically triggered to take photos of them, and wildlife biologists identified the species of the wildlife in the photos. With the development of computer artificial intelligence, some work has also adopted a combination of manual and machine methods, using supervised wildlife recognition models to identify wild animals to improve the efficiency of wildlife identification.

[0003] However, wildlife images captured by camera traps often show animals in motion, and different states of motion can significantly vary their appearance. This makes it difficult for wildlife recognition models to accurately identify wildlife species in different states of motion. Furthermore, for some rare wildlife species, the number of sample images available is limited, and the training effect of wildlife recognition models with such a small number of samples is also very limited, further reducing wildlife recognition accuracy. Summary of the Invention

[0004] The present invention provides a method and device for identifying wild animals, which are used to solve the defect of poor accuracy in identifying wild animals in the prior art.

[0005] The present invention provides a method for identifying wild animals, comprising:

[0006] Performing feature extraction on the image to be processed based on multiple feature extraction layers of different scales of the key point prediction model in sequence to obtain multiple feature maps of different scales, and selecting a plurality of feature maps to be activated from the multiple feature maps of different scales; wherein the image to be processed includes the image to be identified and a first sample image corresponding to any wild animal species;

[0007] Based on the key point prediction layer of the key point prediction model, key point prediction is performed on the multiple feature maps of different scales to obtain key point heat maps under multiple channels corresponding to the image to be processed;

[0008] Based on the key point representation layer of the key point prediction model, the feature map to be activated and the key point heat maps under the multiple channels are fused to obtain key point representations of multiple key points in the image to be processed;

[0009] The key point representations of multiple key points in the image to be identified are matched pairwise with the key point representations of multiple key points in the first sample image corresponding to any one of the wild animal species to obtain a key point matching result of the image to be identified, and the species of the animal in the image to be identified is determined based on the key point matching result.

[0010] According to a wildlife identification method provided by the present invention, the key point representation layer based on the key point prediction model fuses the feature map to be activated and the key point heat maps under the multiple channels to obtain key point representations of multiple key points in the image to be processed, specifically including:

[0011] Upsampling the plurality of feature maps to be activated to a scale consistent with the scale of the key point heat map based on an upsampling layer, and splicing the upsampled feature maps to be activated to obtain a spliced ​​feature map to be activated;

[0012] Performing feature extraction on the stitched front activation feature map based on a convolutional layer to obtain a fused feature map, and fusing the fused feature map with the key point heat maps under the multiple channels to obtain activation feature maps under the multiple channels;

[0013] Based on the maximum pooling layer, pooling processing is performed on the activation feature maps under the multiple channels to obtain key point representations of multiple key points in the image to be processed.

[0014] According to a method for identifying wild animals provided by the present invention, the key point prediction model is trained based on the following steps:

[0015] acquiring key point heat maps under multiple channels corresponding to the second sample image and key point representations of multiple sample key points in the second sample image based on the key point prediction model, and classifying the key point representations of the multiple sample key points in the second sample image based on the key point classification layer to obtain classification results of the multiple sample key points in the second sample image;

[0016] Calculating a key point classification loss based on the classification results and sample type labels of the plurality of sample key points in the second sample image;

[0017] Calculating a key point heat map loss based on key point heat maps and sample heat maps under multiple channels corresponding to the second sample image;

[0018] Parameters of the key point prediction model are adjusted based on the key point classification loss and the key point heat map loss.

[0019] According to a wildlife identification method provided by the present invention, the parameters of the key point prediction model are adjusted based on the key point classification loss and the key point heat map loss, specifically including:

[0020] Adjusting parameters of the keypoint prediction model based on the transformation adaptability loss, the keypoint classification loss, and the keypoint heatmap loss;

[0021] The transformation adaptability loss is determined based on the following steps:

[0022] After performing image transformation on the second sample image based on the first image transformation function to obtain a first transformed sample image, the key point heat map under multiple channels corresponding to the first transformed sample image is obtained using the key point prediction model, and the key point heat map under multiple channels corresponding to the second sample image is subjected to image transformation based on the first image transformation function to obtain a transformed key point heat map under multiple channels corresponding to the second sample image; based on the difference between the key point heat map under multiple channels corresponding to the first transformed sample image and the transformed key point heat map under multiple channels corresponding to the second sample image, the transformation adaptability loss is determined.

[0023] According to a wildlife identification method provided by the present invention, the parameters of the key point prediction model are adjusted based on the transformation adaptability loss, the key point classification loss, and the key point heat map loss, specifically including:

[0024] Adjusting parameters of the key point prediction model based on the transformation consistency loss, the transformation adaptability loss, the key point classification loss, and the key point heat map loss;

[0025] The transformation consistency loss is determined based on the following steps:

[0026] The key point representations of multiple sample key points in the second sample image are spliced ​​to obtain a complete key point vector of the second sample image; after performing image transformation on the second sample image based on the second image transformation function to obtain a second transformed sample image, the key point heat map under multiple channels corresponding to the second transformed sample image is obtained using the key point prediction model, and the key point heat map under multiple channels corresponding to the second transformed sample image is spliced ​​to obtain a complete key point vector of the second transformed sample image; based on the difference between the complete key point vector of the second sample image and the complete key point vector of the second transformed sample image, the transformation adaptability loss is determined.

[0027] According to a method for identifying wild animals provided by the present invention, determining the type of the animal in the image to be identified based on the key point matching result specifically includes:

[0028] Determining, based on the key point matching results, a degree of matching between each key point in the image to be identified and its matching key point in the first sample image;

[0029] Based on the matching degree between each key point in the image to be identified and its matching key point in the first sample image and a first preset matching threshold, it is determined whether the type of animal in the image to be identified is any of the wild animal types.

[0030] According to a method for identifying wild animals provided by the present invention, determining whether the species of the animal in the image to be identified is any of the wild animal species based on the degree of matching between each key point in the image to be identified and its matching key point in the first sample image and a first preset matching threshold, specifically includes:

[0031] determining positions of key points in the image to be identified and the first sample image based on key point heat maps under multiple channels corresponding to the image to be identified and the first sample image respectively;

[0032] Based on the position of each key point in the image to be identified, intercepting the key point image block of each key point in the image to be identified, and obtaining the image features of the key point image block of each key point in the image to be identified;

[0033] Based on the position of each key point in the first sample image, intercepting a key point image block of each key point in the first sample image, and obtaining image features of the key point image block of each key point in the first sample image;

[0034] determining an image matching degree between the image to be identified and the first sample image based on image features of key point image blocks of each key point in the image to be identified and image features of key point image blocks of each key point in the first sample image;

[0035] Based on the matching degree between each key point in the image to be identified and its matching key point in the first sample image and a first preset matching threshold, as well as the image matching degree between the image to be identified and the first sample image and a second preset matching threshold, determine whether the type of animal in the image to be identified is any of the wild animal species.

[0036] According to a method for identifying wild animals provided by the present invention, capturing key point image blocks of each key point in any image specifically includes:

[0037] Determining a proportion of animals in any one of the images based on edge key points of each key point in any one of the images;

[0038] Determining the size of the key point image block to be intercepted based on the proportion of animals in the any one image;

[0039] Based on the size of the key point image block to be intercepted, the key point image block of each key point in any one image is intercepted with the position of each key point in any one image as the center.

[0040] According to a wildlife identification method provided by the present invention, determining the image matching degree between the image to be identified and the first sample image based on the image features of the key point image blocks of each key point in the image to be identified and the image features of the key point image blocks of each key point in the first sample image specifically includes:

[0041] Based on the prior knowledge of the any one wild animal species, determining a distinguishing key point in the any one wild animal species, and determining a degree of match between image features of a key point image block of the distinguishing key point in the image to be identified and image features of a key point image block of the distinguishing key point in the first sample image;

[0042] An image matching degree between the image to be identified and the first sample image is determined based on a matching degree between image features of a key point image block of a distinctive key point in the image to be identified and image features of a key point image block of a distinctive key point in the first sample image.

[0043] The present invention also provides a wild animal identification device, comprising:

[0044] a feature extraction unit configured to sequentially extract features from the image to be processed based on multiple feature extraction layers of different scales of the key point prediction model to obtain multiple feature maps of different scales, and select a plurality of feature maps to be activated from the multiple feature maps of different scales; wherein the image to be processed includes an image to be identified and a first sample image corresponding to any wild animal species;

[0045] A key point prediction unit, configured to perform key point prediction on the plurality of feature maps of different scales based on the key point prediction layer of the key point prediction model, to obtain key point heat maps under a plurality of channels corresponding to the image to be processed;

[0046] A key point representation unit, configured to fuse the feature map to be activated and the key point heat maps under the multiple channels based on the key point representation layer of the key point prediction model to obtain key point representations of multiple key points in the image to be processed;

[0047] An animal identification unit is used to match the key point representations of multiple key points in the image to be identified with the key point representations of multiple key points in the first sample image corresponding to any one of the wild animal species, thereby obtaining a key point matching result of the image to be identified, and determining the species of the animal in the image to be identified based on the key point matching result.

[0048] The present invention also provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for identifying wild animals as described above is implemented.

[0049] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for identifying wild animals as described in any one of the above.

[0050] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-described methods for identifying wild animals.

[0051] The present invention provides a method and device for identifying wild animals. The method extracts features from a to-be-processed image through multiple feature extraction layers of a key point prediction model at different scales to obtain feature maps at different scales. After selecting a number of feature maps to be activated from the feature maps at different scales, the method predicts key points on the feature maps at different scales based on the key point prediction layer to obtain key point heat maps under corresponding multiple channels. The method then fuses the to-be-activated feature map with the key point heat maps under multiple channels based on the key point representation layer to obtain key point representations of each key point, thereby extracting key point representations with rich semantics and motion invariance. The method then matches the key point representations of multiple key points in the to-be-identified image with the key point representations of multiple key points in a first sample image corresponding to a specific wild animal species in pairs to obtain a key point matching result of the to-be-identified image. Based on the key point matching result, the method determines whether the species of the animal in the to-be-identified image is the specific wild animal species, thereby overcoming the interference caused by different animal motion states and improving the accuracy of wild animal identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1This is one of the flow charts of the wild animal identification method provided by the present invention;

[0054] Figure 2 Schematic diagram of the process of the key point prediction model training method provided by the present invention;

[0055] Figure 3 This is the second flow chart of the wild animal identification method provided by the present invention;

[0056] Figure 4 1 is a flow chart of the image block interception method provided by the present invention;

[0057] Figure 5 It is a schematic structural diagram of the wild animal identification device provided by the present invention;

[0058] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0059] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0060] Figure 1 FIG. 1 is a flow chart of the method for identifying wild animals provided by the present invention, such as Figure 1 As shown, the method includes:

[0061] Step 110, sequentially performing feature extraction on the image to be processed based on multiple feature extraction layers of different scales of the key point prediction model to obtain multiple feature maps of different scales, and selecting a plurality of feature maps to be activated from the multiple feature maps of different scales; wherein the image to be processed includes the image to be identified and a first sample image corresponding to any wild animal species;

[0062] Step 120: Based on the key point prediction layer of the key point prediction model, perform key point prediction on the multiple feature maps of different scales to obtain key point heat maps under multiple channels corresponding to the image to be processed;

[0063] Step 130: Based on the key point representation layer of the key point prediction model, the feature map to be activated and the key point heat maps under the multiple channels are fused to obtain key point representations of multiple key points in the image to be processed;

[0064] Step 140, matching the key point representations of multiple key points in the image to be identified with the key point representations of multiple key points in the first sample image corresponding to any one of the wild animal species in pairs to obtain a key point matching result of the image to be identified, and determining the species of the animal in the image to be identified based on the key point matching result.

[0065] Specifically, a trained key point prediction model is obtained, which can realize the recognition of animal key points (such as eyes, mouths, horns, etc.) in images containing wild animals and extract key point representations of each key point. Among them, the key point representation extracted by the key point prediction model can characterize the image semantic information of the key point of the animal, such as semantic information such as position, shape, texture, color, etc. More importantly, the key point representation has motion invariance, that is, when the same type of wild animal is in different motion states, the key point representation of any key point of the wild animal extracted by the key point prediction model is consistent (that is, the similarity is high). Specifically, the key point detection model includes three main structures: a feature extraction layer, a key point prediction layer and a key point representation layer, which are respectively used for image feature extraction, key point prediction and key point vector representation.

[0066] Among them, the feature extraction layer has multiple different scales and is responsible for performing multi-scale feature extraction on the image to be processed, thereby obtaining multiple feature maps of different scales. Each feature extraction layer can be constructed based on a convolutional network. Here, the degree of abstraction of the image semantics contained in the feature maps of different scales is different. The feature maps with larger scales contain lower-level image semantics and can reflect more image detail information. The feature maps with smaller scales contain higher-level image semantics and can reflect more regional type information. It should be noted that the image to be processed can include the image to be identified and the first sample image corresponding to any wild animal species, that is, the processing method for the image to be identified and the first sample image corresponding to the wild animal species is the same. Based on the multiple feature maps of different scales output by the feature extraction layer, the feature maps output by some intermediate-scale feature extraction layers can be selected as feature maps to be activated for subsequent key point representation.

[0067] The key point prediction layer is used to predict key points based on the feature maps of multiple different scales output by the feature extraction layer, and obtain key point heat maps under multiple channels corresponding to the image to be processed. Among them, the key point prediction layer includes multiple branches, which are used to predict the key point heat maps under each channel, and each branch can be constructed based on the convolution layer, activation layer, etc. Here, the channels corresponding to the image to be processed correspond to the key point types of the above-mentioned wild animal species, that is, the number of key point types of the above-mentioned wild animal species is consistent with the number of channels corresponding to the image to be processed. The probability value of each pixel contained in the key point heat map under any channel indicates the possibility that the pixel is the key point type corresponding to the channel. Therefore, based on the key point heat map under the channel, the key point corresponding to the channel (such as the eye) can be located in the image to be processed.

[0068] The key point representation layer is used to fuse the key point heat maps under multiple channels selected from the feature map to be activated and the output of the key point prediction layer, activate and strengthen the positions of the corresponding key points in the feature map to be activated, and thus obtain the key point representation of each key point. Among them, fusion can be performed based on several selected feature maps to be activated to fuse the semantic information of different levels contained in different feature maps to be activated, which helps to improve the semantic expression ability of the key point representation. For the key point heat map under any channel, the fusion result of the feature map to be activated can be fused again with the key point heat map under the channel to select the feature value at the position of the key point corresponding to the channel in the fusion result of the feature map to be activated, thereby constructing the key point representation of the key point corresponding to the channel. Here, since the key point heat map under the channel contains the position information of the corresponding key point, the feature value at the position of the key point corresponding to the channel can be extracted from the fusion result of the feature map to be activated based on the position information of the key point.

[0069] After extracting the key point representations of each key point in the image to be identified and the first sample image corresponding to the above-mentioned wild animal species through the above steps, the key point representations of multiple key points in the image to be identified and the key point representations of multiple key points in the first sample image corresponding to the wild animal species can be matched one by one to obtain the key point matching results of the image to be identified. During the matching process, the distance (e.g., Euclidean distance) between the key point representation of any key point in the image to be identified and the key point representation of each key point in the first sample image can be calculated, and the key point in the first sample image that is closest to it is selected as the matching key point of the key point in the image to be identified. The key point matching result of the image to be identified includes the matching key points of each key point in the image to be identified in the first sample image. Since the key point representations output by the key point prediction model are motion invariant, that is, when the same species of wild animals are in different motion states, the key point representations of any key points of the wild animals extracted by the key point prediction model are consistent. Therefore, if the animal contained in the image to be identified is of this species of wild animal, the key point representation of any key point in the image to be identified should be similar to the key point representation of the key point in the first sample image. Therefore, through the above-mentioned matching method, a corresponding matching key point can be found in the first sample image for the key point in the image to be identified, and the similarity between the key point representations of the two key points should be high. Therefore, based on the key point matching results of the image to be identified, it can be determined whether the species of the animal in the image to be identified is of this species of wild animal (that is, the species of wild animal corresponding to the first sample image). Among them, if there are matching key points for more than a preset number of key points in the key point matching results of the image to be identified and the similarity between the key point representations is higher than a preset value, it can be determined that the species of the animal in the image to be identified is of this species of wild animal.

[0070] The method provided by the embodiment of the present invention performs feature extraction on the image to be processed through multiple feature extraction layers of different scales of the key point prediction model to obtain multiple feature maps of different scales, and selects a number of feature maps to be activated from the feature maps of multiple scales. Then, based on the key point prediction layer, key point prediction is performed on the feature maps of multiple scales to obtain key point heat maps under corresponding multiple channels. Then, based on the key point representation layer, the feature map to be activated and the key point heat maps under multiple channels are fused to obtain key point representations of each key point, and key point representations with rich semantics and motion invariance can be extracted. Subsequently, the key point representations of multiple key points in the image to be identified are matched with the key point representations of multiple key points in the first sample image corresponding to a specific wild animal species in pairs to obtain a key point matching result of the image to be identified, and based on the key point matching result, it can be determined whether the species of the animal in the image to be identified is the specific wild animal species, thereby overcoming the interference caused by different animal motion states and improving the accuracy of wild animal identification.

[0071] Based on the above embodiment, the key point representation layer based on the key point prediction model fuses the feature map to be activated and the key point heat maps under the multiple channels to obtain the key point representation of multiple key points in the image to be processed, specifically including:

[0072] Upsampling the plurality of feature maps to be activated to a scale consistent with the scale of the key point heat map based on an upsampling layer, and splicing the upsampled feature maps to be activated to obtain a spliced ​​feature map to be activated;

[0073] Performing feature extraction on the stitched front activation feature map based on a convolutional layer to obtain a fused feature map, and fusing the fused feature map with the key point heat maps under the multiple channels to obtain activation feature maps under the multiple channels;

[0074] Based on the maximum pooling layer, pooling processing is performed on the activation feature maps under the multiple channels to obtain key point representations of multiple key points in the image to be processed.

[0075] Specifically, based on the upsampling layer included in the key point representation layer, several feature maps to be activated are upsampled separately so that their scales are consistent with the scale of the key point heat map, and the upsampled feature maps to be activated are spliced ​​to obtain a spliced ​​feature map to be activated. It can be seen that the spliced ​​feature map to be activated contains the image semantic information of each feature map to be activated, but the image semantic information of each feature map to be activated is relatively independent of each other. Therefore, feature extraction can be performed on the spliced ​​feature map to be activated based on the convolution layer, and the different image semantic information contained in the spliced ​​feature map to be activated is associated and fused to obtain a fused feature map. Among them, the scale of the fused feature map is also consistent with the scale of the key point heat map.

[0076] Subsequently, the fused feature map is fused with the key point heat maps under multiple channels to obtain activation feature maps under multiple channels. Among them, for the key point heat map under any channel, since the scale of the fused feature map is also consistent with the scale of the key point heat map, in order to use the key point position information in the key point heat map to select the feature value of the corresponding position in the fused feature map, the fused feature map and the key point heat map under any channel can be fused using the dot multiplication method to obtain the activation feature map under the channel. Since the probability value of the pixel in the key point heat map under the channel that may belong to the key point corresponding to the channel is higher, after the dot multiplication of the fused feature map and the key point heat map under the channel, the feature value at the corresponding key point position in the activation feature map under the channel will be enhanced. Therefore, the activation feature maps under multiple channels can be pooled separately based on the maximum pooling layer to obtain the key point representation of the key point corresponding to each channel. For example, for the activation feature map under any channel, the largest feature value in each row can be selected row by row to form the key point representation of the key point corresponding to the channel, or the largest feature value in each column can be selected column by column to form the key point representation of the key point corresponding to the channel.

[0077] Based on any of the above embodiments, Figure 2 As shown, the key point prediction model is trained based on the following steps:

[0078] Step 210: obtaining key point heat maps under multiple channels corresponding to the second sample image and key point representations of multiple sample key points in the second sample image based on the key point prediction model, and classifying the key point representations of the multiple sample key points in the second sample image based on the key point classification layer to obtain classification results for the multiple sample key points in the second sample image;

[0079] Step 220: Calculate the key point classification loss based on the classification results and sample type labels of the plurality of sample key points in the second sample image;

[0080] Step 230: Calculate the keypoint heatmap loss based on the keypoint heatmaps and sample heatmaps under multiple channels corresponding to the second sample image;

[0081] Step 240: Adjust the parameters of the key point prediction model based on the key point classification loss and the key point heat map loss.

[0082] Specifically, the second sample image used by the key point prediction model during the training process can be the same as or different from the first sample image. When there are fewer sample images of a specific wild animal species, sample images of other wild animal species of the same family and similar body shape can be selected as training samples for the key point prediction model. In addition, the second sample image can be an image of an animal of the same wild animal species in different motion states. In order to improve the semantic expression ability of the key point representation extracted by the key point prediction model while ensuring the motion invariance of the key point representation, the key point heat map under multiple channels corresponding to the second sample image and the key point representation of multiple sample key points in the second sample image can be obtained based on the key point prediction model (the method is the same as the method of extracting the key point heat map under each channel in the image to be processed and the key point representation of each key point in the above embodiment), and the key point representation of multiple sample key points in the second sample image is classified based on the key point classification layer (which can be a fully connected network) to obtain the classification results of multiple sample key points in the second sample image. Based on the classification results of multiple sample key points in the second sample image and the sample type labels, the key point classification loss can be calculated. The higher the degree of consistency between the classification results of multiple sample keypoints in the second sample image and the sample type labels, the smaller the keypoint classification loss. By adjusting the model parameters of the keypoint prediction model using the above keypoint classification loss, the keypoint detection model can be guided to detect similar features of the same keypoint (the same keypoint type of the same wildlife species) in different motion states.

[0083] Since the performance of the key point representation layer in the key point prediction model is closely related to the performance of the key point prediction layer, in order to further improve the performance of the key point representation layer, the key point heat map loss can also be calculated based on the key point heat maps under multiple channels corresponding to the second sample image and the sample heat maps of the second sample image under multiple channels, and the parameters of the key point prediction model are adjusted in combination with the above-mentioned key point classification loss and the key point heat map loss. Among them, the higher the degree of consistency between the key point heat maps under multiple channels corresponding to the second sample image and the sample heat maps of the second sample image under multiple channels, the smaller the key point heat map loss. By adjusting the parameters of the key point prediction model in combination with the key point classification loss and the key point heat map loss, the multi-task mechanism is used to enable the key point prediction layer and the key point representation layer to be collaboratively optimized, thereby further improving the model training efficiency and training effect.

[0084] Based on any of the foregoing embodiments, adjusting the parameters of the key point prediction model based on the key point classification loss and the key point heat map loss specifically includes:

[0085] Adjusting parameters of the keypoint prediction model based on the transformation adaptability loss, the keypoint classification loss, and the keypoint heatmap loss;

[0086] The transformation adaptability loss is determined based on the following steps:

[0087] After performing image transformation on the second sample image based on the first image transformation function to obtain a first transformed sample image, the key point heat map under multiple channels corresponding to the first transformed sample image is obtained using the key point prediction model, and the key point heat map under multiple channels corresponding to the second sample image is subjected to image transformation based on the first image transformation function to obtain a transformed key point heat map under multiple channels corresponding to the second sample image; based on the difference between the key point heat map under multiple channels corresponding to the first transformed sample image and the transformed key point heat map under multiple channels corresponding to the second sample image, the transformation adaptability loss is determined.

[0088] Specifically, for the key point prediction layer, the key point positions indicated in the key point heat map it outputs need to change with the image transformation. However, when the number of training samples of the key point prediction model is small (due to the high cost of manual annotation, resulting in fewer available training samples), the key point heat map loss, as a supervised loss, has a weak ability to guide the model. Therefore, in order to improve the performance of the key point prediction layer, an unsupervised loss is introduced for the key point prediction layer, namely, the transformation adaptability loss, which combines the transformation adaptability loss, the key point classification loss and the key point heat map loss to adjust the parameters of the key point prediction model.

[0089] Among them, in order to calculate the transformation adaptability loss, the second sample image can be transformed based on the first image transformation function (such as image scaling, rotation, translation and other image transformation methods, and any image transformation method can be randomly selected as the first image transformation function in each round of training) to obtain a first transformed sample image, and then the key point prediction model is used to obtain the key point heat map under multiple channels corresponding to the first transformed sample image. At the same time, the key point heat map under multiple channels corresponding to the second sample image is also transformed based on the first image transformation function to obtain the transformed key point heat map under multiple channels corresponding to the second sample image. According to the difference between the key point heat map under multiple channels corresponding to the first transformed sample image and the transformed key point heat map under multiple channels corresponding to the second sample image, the transformation adaptability loss can be calculated.

[0090] Based on any of the foregoing embodiments, adjusting the parameters of the key point prediction model based on the transformation adaptability loss, the key point classification loss, and the key point heat map loss specifically includes:

[0091] Adjusting parameters of the key point prediction model based on the transformation consistency loss, the transformation adaptability loss, the key point classification loss, and the key point heat map loss;

[0092] The transformation consistency loss is determined based on the following steps:

[0093] The key point representations of multiple sample key points in the second sample image are spliced ​​to obtain a complete key point vector of the second sample image; after performing image transformation on the second sample image based on the second image transformation function to obtain a second transformed sample image, the key point heat map under multiple channels corresponding to the second transformed sample image is obtained using the key point prediction model, and the key point heat map under multiple channels corresponding to the second transformed sample image is spliced ​​to obtain a complete key point vector of the second transformed sample image; based on the difference between the complete key point vector of the second sample image and the complete key point vector of the second transformed sample image, the transformation adaptability loss is determined.

[0094] Specifically, the keypoint representation layer must output motion-invariant keypoint representations. This means that even when the image is transformed, the keypoint representations for the same keypoints should remain consistent. Considering that motion invariance of keypoint representations is a core foundation for wildlife recognition, to further ensure the motion invariance of the keypoint representations output by the keypoint prediction model, an additional unsupervised loss, namely, the transformation consistency loss, is introduced to the keypoint representation layer. This loss combines the transformation consistency loss, the transformation adaptability loss, the keypoint classification loss, and the keypoint heatmap loss to adjust the parameters of the keypoint prediction model.

[0095] Among them, in order to calculate the transformation consistency loss, on the one hand, the key point representations of each sample key point in the second sample image can be spliced ​​to obtain the complete key point vector of the second sample image; on the other hand, based on the second image transformation function (such as image scaling, rotation, translation and other image transformation methods, each round of training can randomly select any image transformation method as the second image transformation function), the second sample image can be transformed to obtain a second transformed sample image, and then the key point prediction model is used to obtain the key point heat map under multiple channels corresponding to the second transformed sample image, and the key point heat map under multiple channels corresponding to the second transformed sample image is spliced ​​to obtain the complete key point vector of the second transformed sample image. Based on the difference between the complete key point vector of the second sample image and the complete key point vector of the second transformed sample image, the transformation adaptability loss can be calculated.

[0096] Based on any of the foregoing embodiments, determining the type of animal in the image to be identified based on the key point matching result specifically includes:

[0097] Determining, based on the key point matching results, a degree of matching between each key point in the image to be identified and its matching key point in the first sample image;

[0098] Based on the matching degree between each key point in the image to be identified and its matching key point in the first sample image and a first preset matching threshold, it is determined whether the type of animal in the image to be identified is any of the wild animal types.

[0099] Specifically, the matching key points in the first sample image and the matching degree between each key point of the image to be identified in the key point matching results are obtained. Based on the matching degree between each key point in the image to be identified and its matching key point in the first sample image and a first preset matching threshold, it is possible to determine whether the species of the animal in the image to be identified is the wild animal species corresponding to the first sample image. If the matching degree between more than a preset number of key points in the image to be identified and their matching key points in the first sample image is greater than the first preset matching threshold, the species of the animal in the image to be identified is determined to be the wild animal species corresponding to the first sample image.

[0100] Based on any of the above embodiments, Figure 3 As shown, the determining whether the species of the animal in the image to be identified is any of the wild animal species based on the matching degree between each key point in the image to be identified and its matching key point in the first sample image and a first preset matching threshold specifically includes:

[0101] Step 310: determining the positions of key points in the image to be identified and the first sample image based on key point heat maps under multiple channels corresponding to the image to be identified and the first sample image respectively;

[0102] Step 320: based on the position of each key point in the image to be identified, intercept the key point image block of each key point in the image to be identified, and obtain the image features of the key point image block of each key point in the image to be identified;

[0103] Step 330: based on the position of each key point in the first sample image, intercept the key point image block of each key point in the first sample image, and obtain image features of the key point image block of each key point in the first sample image;

[0104] Step 340: determining an image matching degree between the image to be identified and the first sample image based on image features of the key point image blocks of each key point in the image to be identified and image features of the key point image blocks of each key point in the first sample image;

[0105] Step 350: Determine whether the species of the animal in the image to be identified is any of the wild animal species based on the matching degree between each key point in the image to be identified and its matching key point in the first sample image and a first preset matching threshold, as well as the image matching degree between the image to be identified and the first sample image and a second preset matching threshold.

[0106] Specifically, based on the key point heat maps under multiple channels corresponding to the image to be identified, the position of each key point in the image to be identified can be determined. Using the position of each key point in the image to be identified, the key point image block of each key point in the image to be identified can be intercepted, and the image features of the key point image block of each key point in the image to be identified can be obtained. Among them, the image block interception can be performed with each key point position as the center. In addition, based on the key point heat maps under multiple channels corresponding to the image to be identified and the first sample image, the position of each key point in the image to be identified and the first sample image is determined. Based on the key point heat maps under multiple channels corresponding to the first sample image, the position of each key point in the first sample image can be determined. Using the position of each key point in the first sample image, the key point image block of each key point in the first sample image can be intercepted, and the image features of the key point image block of each key point in the first sample image can be obtained.

[0107] The degree of image matching between the image to be identified and the first sample image can be determined based on the image features of the keypoint image blocks of each keypoint in the image to be identified and the image features of the keypoint image blocks of each keypoint in the first sample image. The image features of the keypoint image block of any keypoint in the image to be identified can be matched with the image features of the keypoint image block of its matching keypoint in the first sample image to obtain the degree of matching for that keypoint. The degree of image matching between the image to be identified and the first sample image can be determined by combining the degree of matching for each keypoint in the image to be identified. Whether the species of the animal in the image to be identified is the species of wild animal corresponding to the first sample image can be determined based on the degree of matching between each keypoint in the image to be identified and its matching keypoint in the first sample image and a first preset matching threshold, as well as the degree of image matching between the image to be identified and the first sample image and a second preset matching threshold. By setting two criteria to determine whether the species of the animal in the image to be identified is the species of wild animal corresponding to the first sample image, namely, the degree of matching between more than a preset number of keypoints in the image to be identified and their matching keypoints in the first sample image is greater than the first preset matching threshold, and the degree of image matching between the image to be identified and the first sample image is greater than the second preset matching threshold, the accuracy of wild animal identification can be further improved.

[0108] Based on any of the above embodiments, Figure 4As shown, the key point image blocks of each key point in any image are intercepted, specifically including:

[0109] Step 410, determining the proportion of animals in any one image based on the edge key points of each key point in any one image;

[0110] Step 420 , determining the size of the key point image block to be intercepted based on the proportion of animals in the any one image;

[0111] Step 430 : Based on the size of the key point image block to be intercepted and with the position of each key point in any one image as the center, intercept the key point image block of each key point in any one image.

[0112] Specifically, when capturing any image (the image to be identified or the first sample image), the proportion of animals in the image can be determined based on the edge keypoints among the keypoints in the image. The edge keypoints of the image can be determined based on the positions of the keypoints in the image. For example, the keypoints with the smallest x-coordinate, the largest x-coordinate, the smallest y-coordinate, and the largest y-coordinate can be selected as edge keypoints. Based on the positions of the edge keypoints, a rectangular frame can be determined to encompass all keypoints. Based on the size of the rectangular frame and the size of the image, the proportion of animals in the image can be calculated. Based on the proportion of animals in the image, the size of the keypoint image block to be captured is determined. The larger the proportion of animals in the image, the larger the size of the keypoint image block to be captured. Conversely, the smaller the proportion of animals in the image, the smaller the size of the keypoint image block to be captured, thereby avoiding capturing areas outside the keypoints. Subsequently, based on the size of the key point image block to be intercepted, the key point image block of each key point in the image is intercepted with the position of each key point in the image as the center.

[0113] Based on any of the foregoing embodiments, determining the image matching degree between the image to be identified and the first sample image based on the image features of the key point image blocks of each key point in the image to be identified and the image features of the key point image blocks of each key point in the first sample image specifically includes:

[0114] Based on the prior knowledge of the any one wild animal species, determining a distinguishing key point in the any one wild animal species, and determining a degree of match between image features of a key point image block of the distinguishing key point in the image to be identified and image features of a key point image block of the distinguishing key point in the first sample image;

[0115] An image matching degree between the image to be identified and the first sample image is determined based on a matching degree between image features of a key point image block of a distinctive key point in the image to be identified and image features of a key point image block of a distinctive key point in the first sample image.

[0116] Specifically, when matching the image features of the keypoint image blocks of each keypoint in the image to be identified with the image features of the keypoint image blocks of each keypoint in the first sample image, one or more discriminative keypoints of the wildlife species corresponding to the first sample image can be determined based on prior knowledge of the wildlife species corresponding to the first sample image. These discriminative keypoints can be used as targets for image feature matching, and the degree of match between the image features of the keypoint image blocks of the discriminative keypoints in the image to be identified and the image features of the keypoint image blocks of the same discriminative keypoints in the first sample image can be determined. The discriminative keypoints in the image to be identified and the same discriminative keypoints in the first sample image can be determined from the image to be identified and the first sample image based on the type of the discriminative keypoints and combined with keypoint heat maps in multiple channels corresponding to the image to be identified and the first sample image. Subsequently, the degree of image matching between the image to be identified and the first sample image can be determined based on the degree of match between the image features of the keypoint image blocks of the discriminative keypoints in the image to be identified and the image features of the keypoint image blocks of the same discriminative keypoints in the first sample image.

[0117] A wild animal identification device provided by the present invention is described below. The wild animal identification device described below and the wild animal identification method described above can be referenced to each other.

[0118] Based on any of the above embodiments, Figure 5 FIG. 1 is a schematic diagram of the structure of the wild animal identification device provided by the present invention. Figure 5 As shown, the device includes: a feature extraction unit 510, a key point prediction unit 520, a key point representation unit 530 and an animal recognition unit 540.

[0119] The feature extraction unit 510 is configured to sequentially extract features from the image to be processed based on multiple feature extraction layers of different scales of the key point prediction model to obtain multiple feature maps of different scales, and select a plurality of feature maps to be activated from the multiple feature maps of different scales; wherein the image to be processed includes the image to be identified and a first sample image corresponding to any wild animal species;

[0120] The key point prediction unit 520 is configured to perform key point prediction on the multiple feature maps of different scales based on the key point prediction layer of the key point prediction model to obtain key point heat maps under multiple channels corresponding to the image to be processed;

[0121] The key point representation unit 530 is configured to fuse the feature map to be activated and the key point heat maps under the multiple channels based on the key point representation layer of the key point prediction model to obtain key point representations of multiple key points in the image to be processed;

[0122] The animal identification unit 540 is used to match the key point representations of multiple key points in the image to be identified with the key point representations of multiple key points in the first sample image corresponding to any one of the wild animal species, obtain the key point matching results of the image to be identified, and determine the type of animal in the image to be identified based on the key point matching results.

[0123] The device provided by the embodiment of the present invention performs feature extraction on the image to be processed through multiple feature extraction layers of different scales of the key point prediction model to obtain multiple feature maps of different scales, and selects a number of feature maps to be activated from the feature maps of multiple scales. Then, based on the key point prediction layer, key point prediction is performed on the feature maps of multiple scales to obtain key point heat maps under corresponding multiple channels. Then, based on the key point representation layer, the feature map to be activated and the key point heat maps under multiple channels are fused to obtain key point representations of each key point, and key point representations with rich semantics and motion invariance can be extracted. Subsequently, the key point representations of multiple key points in the image to be identified are matched with the key point representations of multiple key points in the first sample image corresponding to a specific wild animal species in pairs to obtain a key point matching result of the image to be identified, and based on the key point matching result, it can be determined whether the species of the animal in the image to be identified is the specific wild animal species, thereby overcoming the interference caused by different animal motion states and improving the accuracy of wild animal identification.

[0124] Based on any of the above embodiments, the key point representation layer based on the key point prediction model fuses the feature map to be activated and the key point heat maps under the multiple channels to obtain key point representations of multiple key points in the image to be processed, specifically including:

[0125] Upsampling the plurality of feature maps to be activated to a scale consistent with the scale of the key point heat map based on an upsampling layer, and splicing the upsampled feature maps to be activated to obtain a spliced ​​feature map to be activated;

[0126] Performing feature extraction on the stitched front activation feature map based on a convolutional layer to obtain a fused feature map, and fusing the fused feature map with the key point heat maps under the multiple channels to obtain activation feature maps under the multiple channels;

[0127] Based on the maximum pooling layer, pooling processing is performed on the activation feature maps under the multiple channels to obtain key point representations of multiple key points in the image to be processed.

[0128] Based on any of the above embodiments, the key point prediction model is trained based on the following steps:

[0129] acquiring key point heat maps under multiple channels corresponding to the second sample image and key point representations of multiple sample key points in the second sample image based on the key point prediction model, and classifying the key point representations of the multiple sample key points in the second sample image based on the key point classification layer to obtain classification results of the multiple sample key points in the second sample image;

[0130] Calculating a key point classification loss based on the classification results and sample type labels of the plurality of sample key points in the second sample image;

[0131] Calculating a key point heat map loss based on key point heat maps and sample heat maps under multiple channels corresponding to the second sample image;

[0132] Parameters of the key point prediction model are adjusted based on the key point classification loss and the key point heat map loss.

[0133] Based on any of the foregoing embodiments, adjusting the parameters of the key point prediction model based on the key point classification loss and the key point heat map loss specifically includes:

[0134] Adjusting parameters of the keypoint prediction model based on the transformation adaptability loss, the keypoint classification loss, and the keypoint heatmap loss;

[0135] The transformation adaptability loss is determined based on the following steps:

[0136] After performing image transformation on the second sample image based on the first image transformation function to obtain a first transformed sample image, the key point heat map under multiple channels corresponding to the first transformed sample image is obtained using the key point prediction model, and the key point heat map under multiple channels corresponding to the second sample image is subjected to image transformation based on the first image transformation function to obtain a transformed key point heat map under multiple channels corresponding to the second sample image; based on the difference between the key point heat map under multiple channels corresponding to the first transformed sample image and the transformed key point heat map under multiple channels corresponding to the second sample image, the transformation adaptability loss is determined.

[0137] Based on any of the foregoing embodiments, adjusting the parameters of the key point prediction model based on the transformation adaptability loss, the key point classification loss, and the key point heat map loss specifically includes:

[0138] Adjusting parameters of the key point prediction model based on the transformation consistency loss, the transformation adaptability loss, the key point classification loss, and the key point heat map loss;

[0139] The transformation consistency loss is determined based on the following steps:

[0140] The key point representations of multiple sample key points in the second sample image are spliced ​​to obtain a complete key point vector of the second sample image; after performing image transformation on the second sample image based on the second image transformation function to obtain a second transformed sample image, the key point heat map under multiple channels corresponding to the second transformed sample image is obtained using the key point prediction model, and the key point heat map under multiple channels corresponding to the second transformed sample image is spliced ​​to obtain a complete key point vector of the second transformed sample image; based on the difference between the complete key point vector of the second sample image and the complete key point vector of the second transformed sample image, the transformation adaptability loss is determined.

[0141] Based on any of the foregoing embodiments, determining the type of animal in the image to be identified based on the key point matching result specifically includes:

[0142] Determining, based on the key point matching results, a degree of matching between each key point in the image to be identified and its matching key point in the first sample image;

[0143] Based on the matching degree between each key point in the image to be identified and its matching key point in the first sample image and a first preset matching threshold, it is determined whether the type of animal in the image to be identified is any of the wild animal types.

[0144] Based on any of the foregoing embodiments, determining whether the species of the animal in the image to be identified is any of the wild animal species based on the degree of matching between each key point in the image to be identified and its matching key point in the first sample image and a first preset matching threshold specifically includes:

[0145] determining positions of key points in the image to be identified and the first sample image based on key point heat maps under multiple channels corresponding to the image to be identified and the first sample image respectively;

[0146] Based on the position of each key point in the image to be identified, intercepting the key point image block of each key point in the image to be identified, and obtaining the image features of the key point image block of each key point in the image to be identified;

[0147] Based on the position of each key point in the first sample image, intercepting a key point image block of each key point in the first sample image, and obtaining image features of the key point image block of each key point in the first sample image;

[0148] determining an image matching degree between the image to be identified and the first sample image based on image features of key point image blocks of each key point in the image to be identified and image features of key point image blocks of each key point in the first sample image;

[0149] Based on the matching degree between each key point in the image to be identified and its matching key point in the first sample image and a first preset matching threshold, as well as the image matching degree between the image to be identified and the first sample image and a second preset matching threshold, determine whether the type of animal in the image to be identified is any of the wild animal species.

[0150] Based on any of the above embodiments, intercepting a key point image block of each key point in any image specifically includes:

[0151] Determining a proportion of animals in any one of the images based on edge key points of each key point in any one of the images;

[0152] Determining the size of the key point image block to be intercepted based on the proportion of animals in the any one image;

[0153] Based on the size of the key point image block to be intercepted, the key point image block of each key point in any one image is intercepted with the position of each key point in any one image as the center.

[0154] Based on any of the foregoing embodiments, determining the image matching degree between the image to be identified and the first sample image based on the image features of the key point image blocks of each key point in the image to be identified and the image features of the key point image blocks of each key point in the first sample image specifically includes:

[0155] Based on the prior knowledge of the any one wild animal species, determining a distinguishing key point in the any one wild animal species, and determining a degree of match between image features of a key point image block of the distinguishing key point in the image to be identified and image features of a key point image block of the distinguishing key point in the first sample image;

[0156] An image matching degree between the image to be identified and the first sample image is determined based on a matching degree between image features of a key point image block of a distinctive key point in the image to be identified and image features of a key point image block of a distinctive key point in the first sample image.

[0157] Figure 6 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 6As shown, the electronic device may include: a processor (processor) 610, a memory (memory) 620, a communication interface (Communications Interface) 630 and a communication bus 640, wherein the processor 610, the memory 620, and the communication interface 630 communicate with each other through the communication bus 640. The processor 610 can call the logic instructions in the memory 620 to execute a method for identifying wild animals, which includes: sequentially performing feature extraction on the image to be processed based on multiple feature extraction layers of different scales of a key point prediction model to obtain multiple feature maps of different scales, and selecting a number of feature maps to be activated from the feature maps of different scales; wherein the image to be processed includes the image to be identified and a first sample image corresponding to any wild animal species; based on the key point prediction layer of the key point prediction model, performing key point prediction on the feature maps of multiple scales to obtain key point heat maps under multiple channels corresponding to the image to be processed; based on the key point representation layer of the key point prediction model, fusing the feature map to be activated and the key point heat maps under the multiple channels to obtain key point representations of multiple key points in the image to be processed; matching the key point representations of multiple key points in the image to be identified with the key point representations of multiple key points in the first sample image corresponding to any wild animal species in pairs to obtain a key point matching result of the image to be identified, and determining the type of animal in the image to be identified based on the key point matching result.

[0158] In addition, the logic instructions in the above-mentioned memory 620 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0159] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute a wildlife identification method provided by the above methods, the method comprising: sequentially performing feature extraction on a to-be-processed image based on multiple feature extraction layers of different scales of a key point prediction model to obtain multiple feature maps of different scales, and selecting a number of feature maps to be activated from the multiple feature maps of different scales; wherein, the to-be-processed image includes an image to be identified and a first sample image corresponding to any wildlife species; based on The key point prediction layer of the key point prediction model performs key point prediction on the multiple feature maps of different scales to obtain key point heat maps under multiple channels corresponding to the image to be processed; based on the key point representation layer of the key point prediction model, the feature map to be activated and the key point heat maps under the multiple channels are fused to obtain key point representations of multiple key points in the image to be processed; the key point representations of multiple key points in the image to be identified and the key point representations of multiple key points in the first sample image corresponding to any wild animal species are matched pairwise to obtain key point matching results of the image to be identified, and the species of the animal in the image to be identified is determined based on the key point matching results.

[0160] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute a method for identifying wild animals provided above, the method comprising: performing feature extraction on the image to be processed based on multiple feature extraction layers of different scales of a key point prediction model in sequence to obtain multiple feature maps of different scales, and selecting a number of feature maps to be activated from the feature maps of different scales; wherein the image to be processed includes the image to be identified and a first sample image corresponding to any wild animal species; based on the key point prediction layer of the key point prediction model, performing key point prediction on the feature maps of multiple different scales to obtain key point heat maps under multiple channels corresponding to the image to be processed; based on the key point representation layer of the key point prediction model, fusing the feature map to be activated and the key point heat maps under the multiple channels to obtain key point representations of multiple key points in the image to be processed; matching the key point representations of multiple key points in the image to be identified with the key point representations of multiple key points in the first sample image corresponding to any wild animal species in pairs to obtain a key point matching result of the image to be identified, and determining the species of animal in the image to be identified based on the key point matching result.

[0161] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0162] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for identifying wild animals, characterized in that: include: Performing feature extraction on the image to be processed based on multiple feature extraction layers of different scales of the key point prediction model in sequence to obtain multiple feature maps of different scales, and selecting a plurality of feature maps to be activated from the multiple feature maps of different scales; wherein the image to be processed includes the image to be identified and a first sample image corresponding to any wild animal species; Based on the key point prediction layer of the key point prediction model, key point prediction is performed on the multiple feature maps of different scales to obtain key point heat maps under multiple channels corresponding to the image to be processed; Based on the key point representation layer of the key point prediction model, the feature map to be activated and the key point heat maps under the multiple channels are fused to obtain key point representations of multiple key points in the image to be processed; Matching key point representations of multiple key points in the image to be identified with key point representations of multiple key points in the first sample image corresponding to any one of the wild animal species in pairs to obtain a key point matching result of the image to be identified, and determining the species of the animal in the image to be identified based on the key point matching result; The key point representation layer based on the key point prediction model fuses the feature map to be activated and the key point heat maps under the multiple channels to obtain key point representations of multiple key points in the image to be processed, specifically including: Upsampling the plurality of feature maps to be activated to a scale consistent with the scale of the key point heat map based on an upsampling layer, and splicing the upsampled feature maps to be activated to obtain a spliced ​​feature map to be activated; Performing feature extraction on the stitched front activation feature map based on a convolutional layer to obtain a fused feature map, and fusing the fused feature map with the key point heat maps under the multiple channels to obtain activation feature maps under the multiple channels; Based on the maximum pooling layer, pooling processing is performed on the activation feature maps under the multiple channels to obtain key point representations of multiple key points in the image to be processed.

2. A method for identifying wild animals according to claim 1, characterized in that: The key point prediction model is trained based on the following steps: acquiring key point heat maps under multiple channels corresponding to the second sample image and key point representations of multiple sample key points in the second sample image based on the key point prediction model, and classifying the key point representations of the multiple sample key points in the second sample image based on the key point classification layer to obtain classification results of the multiple sample key points in the second sample image; Calculating a key point classification loss based on the classification results and sample type labels of the plurality of sample key points in the second sample image; Calculating a key point heat map loss based on key point heat maps and sample heat maps under multiple channels corresponding to the second sample image; Parameters of the key point prediction model are adjusted based on the key point classification loss and the key point heat map loss.

3. A method for identifying wild animals according to claim 2, characterized in that: The adjusting the parameters of the key point prediction model based on the key point classification loss and the key point heat map loss specifically includes: Adjusting parameters of the keypoint prediction model based on the transformation adaptability loss, the keypoint classification loss, and the keypoint heatmap loss; The transformation adaptability loss is determined based on the following steps: After performing image transformation on the second sample image based on the first image transformation function to obtain a first transformed sample image, the key point heat map under multiple channels corresponding to the first transformed sample image is obtained using the key point prediction model, and the key point heat map under multiple channels corresponding to the second sample image is subjected to image transformation based on the first image transformation function to obtain a transformed key point heat map under multiple channels corresponding to the second sample image; based on the difference between the key point heat map under multiple channels corresponding to the first transformed sample image and the transformed key point heat map under multiple channels corresponding to the second sample image, the transformation adaptability loss is determined.

4. A method for identifying wild animals according to claim 3, characterized in that: The adjusting of the parameters of the key point prediction model based on the transformation adaptability loss, the key point classification loss, and the key point heat map loss specifically includes: Adjusting parameters of the key point prediction model based on the transformation consistency loss, the transformation adaptability loss, the key point classification loss, and the key point heat map loss; The transformation consistency loss is determined based on the following steps: The key point representations of multiple sample key points in the second sample image are spliced ​​to obtain a complete key point vector of the second sample image; after performing image transformation on the second sample image based on the second image transformation function to obtain a second transformed sample image, the key point heat map under multiple channels corresponding to the second transformed sample image is obtained using the key point prediction model, and the key point heat map under multiple channels corresponding to the second transformed sample image is spliced ​​to obtain a complete key point vector of the second transformed sample image; based on the difference between the complete key point vector of the second sample image and the complete key point vector of the second transformed sample image, the transformation adaptability loss is determined.

5. A method for identifying wild animals according to claim 1, characterized in that: Determining the type of the animal in the image to be identified based on the key point matching result specifically includes: Determining, based on the key point matching results, a degree of matching between each key point in the image to be identified and its matching key point in the first sample image; Based on the matching degree between each key point in the image to be identified and its matching key point in the first sample image and a first preset matching threshold, it is determined whether the type of animal in the image to be identified is any of the wild animal types.

6. A method for identifying wild animals according to claim 5, characterized in that: The determining, based on the degree of matching between each key point in the image to be identified and its matching key point in the first sample image and a first preset matching threshold, whether the species of the animal in the image to be identified is any of the wild animal species specifically includes: determining positions of key points in the image to be identified and the first sample image based on key point heat maps under multiple channels corresponding to the image to be identified and the first sample image respectively; Based on the position of each key point in the image to be identified, intercepting the key point image block of each key point in the image to be identified, and obtaining the image features of the key point image block of each key point in the image to be identified; Based on the position of each key point in the first sample image, intercepting a key point image block of each key point in the first sample image, and obtaining image features of the key point image block of each key point in the first sample image; determining an image matching degree between the image to be identified and the first sample image based on image features of key point image blocks of each key point in the image to be identified and image features of key point image blocks of each key point in the first sample image; Based on the matching degree between each key point in the image to be identified and its matching key point in the first sample image and a first preset matching threshold, as well as the image matching degree between the image to be identified and the first sample image and a second preset matching threshold, determine whether the type of animal in the image to be identified is any of the wild animal species.

7. A method for identifying wild animals according to claim 6, characterized in that: Capture key point image blocks of each key point in any image, including: Determining a proportion of animals in any one of the images based on edge key points of each key point in any one of the images; Determining the size of the key point image block to be intercepted based on the proportion of animals in the any one image; Based on the size of the key point image block to be intercepted, the key point image block of each key point in any one image is intercepted with the position of each key point in any one image as the center.

8. A method for identifying wild animals according to claim 6, characterized in that: The determining of the image matching degree between the image to be identified and the first sample image based on the image features of the key point image blocks of each key point in the image to be identified and the image features of the key point image blocks of each key point in the first sample image specifically includes: Based on the prior knowledge of the any one wild animal species, determining a distinguishing key point in the any one wild animal species, and determining a degree of match between image features of a key point image block of the distinguishing key point in the image to be identified and image features of a key point image block of the distinguishing key point in the first sample image; An image matching degree between the image to be identified and the first sample image is determined based on a matching degree between image features of a key point image block of a distinctive key point in the image to be identified and image features of a key point image block of a distinctive key point in the first sample image.

9. A wild animal identification device, characterized in that: include: a feature extraction unit configured to sequentially extract features from the image to be processed based on multiple feature extraction layers of different scales of the key point prediction model to obtain multiple feature maps of different scales, and select a plurality of feature maps to be activated from the multiple feature maps of different scales; wherein the image to be processed includes an image to be identified and a first sample image corresponding to any wild animal species; A key point prediction unit, configured to perform key point prediction on the plurality of feature maps of different scales based on the key point prediction layer of the key point prediction model, to obtain key point heat maps under a plurality of channels corresponding to the image to be processed; A key point representation unit, configured to fuse the feature map to be activated and the key point heat maps under the multiple channels based on the key point representation layer of the key point prediction model to obtain key point representations of multiple key points in the image to be processed; an animal identification unit, configured to pairwise match key point representations of a plurality of key points in the image to be identified with key point representations of a plurality of key points in the first sample image corresponding to any one of the wild animal species, to obtain a key point matching result of the image to be identified, and to determine the species of the animal in the image to be identified based on the key point matching result; The key point representation layer based on the key point prediction model fuses the feature map to be activated and the key point heat maps under the multiple channels to obtain key point representations of multiple key points in the image to be processed, specifically including: Upsampling the plurality of feature maps to be activated to a scale consistent with the scale of the key point heat map based on an upsampling layer, and splicing the upsampled feature maps to be activated to obtain a spliced ​​feature map to be activated; Performing feature extraction on the stitched front activation feature map based on a convolutional layer to obtain a fused feature map, and fusing the fused feature map with the key point heat maps under the multiple channels to obtain activation feature maps under the multiple channels; Based on the maximum pooling layer, pooling processing is performed on the activation feature maps under the multiple channels to obtain key point representations of multiple key points in the image to be processed.

Images