Wild animal identification method fusing fine-grained features and context features

Abstract

The invention provides a wildlife identification method fusing fine-grained features and context features, and belongs to the technical field of wildlife identification. The problems that wild animals present multi-scale features due to body types and shooting distances and only depend on high-level semantic features, small animals are prone to losing key details, large animals are difficult to comprehensively express various features and cannot adapt to multi-scale changes, and the recognition precision is affected are solved. The method comprises the following steps: constructing an image wild animal identification network based on a Swin Transfomer-ConvNextv2 algorithm and an HCFF module, training the constructed identification network, carrying out model optimization on the trained wild animal identification network, and carrying out animal image species identification on the optimized network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wild animal recognition, in particular to a wild animal recognition method fusing fine-grained features and context features. BACKGROUND

[0002] Currently, there are mainly two mainstream methods for wild animal species recognition. The first one is an end-to-end target detection method, which can identify the location and category of the target object in the image and return all results, thus improving the processing capability of camera data. Its advantage lies in its relatively fast detection speed, which can process a large amount of image data in a short time. However, it also has obvious disadvantages, such as low recognition accuracy, which cannot meet the demand for accurate recognition of wild animal species. This method requires a large amount of manual annotation frames for model training. The annotation personnel need to spend a lot of time and effort to accurately annotate the target object in each image, which not only has high cost, but also the accuracy and consistency of the annotation are difficult to guarantee.

[0003] Studies have shown that in complex natural environments, detecting the location of animals first may be the basis for improving classification ability. Specifically, this method first uses a target detection network to locate animals in an image, and then inputs the cropped animal image into a species classification network for species classification. However, through in-depth research on related literature, it is found that this method is not perfect. Njathi et al. discussed the influence of background on wild animal image classification performance, showing that the classification accuracy of antelope images will significantly decrease after completely removing the background, and the accuracy will increase when replaced with a similar natural background, verifying that the background has a significant auxiliary role in wild animal image classification. SUMMARY

[0004] Therefore, the present application aims to provide a wild animal recognition method fusing fine-grained features and context features to solve the problem that wild animals present multi-scale features due to body size and shooting distance, rely only on high-level semantic features, small animals easily lose key details, large animals are difficult to fully represent diverse features, and cannot adapt to multi-scale changes, affecting recognition accuracy.

[0005] To achieve the above purpose, the present application adopts the following technical solution. According to one aspect of the present application, a wild animal recognition method fusing fine-grained features and context features is provided, comprising the following steps: S1, image acquisition, an infrared camera acquires original image data of wild animals; S2, target positioning, inputting the collected original image into a target detection network to extract four scales of context features; S3, feature acquisition, adjusting the pixels of the cropped original image to input into a classification network to obtain four scales of fine-grained features. S4, hierarchical complementary feature fusion, the context features and the fine-grained features are sequentially subjected to hierarchical complementary feature fusion; S5, classification output, the features after fusion are subjected to global average pooling, flattening and full connection layer processing to complete the identification and classification of wild animal species.

[0006] Further, the detection network adopts a Swin-Transformer algorithm.

[0007] Further, the original image is cropped according to the positioning frame on the original image.

[0008] Further, the adjusted pixels are 256x256 pixels.

[0009] Further, the classification network is ConvNeXtv2.

[0010] Further, S4 is that after obtaining the multi-scale context features and the fine-grained features in the previous step, the features are sequentially subjected to a hierarchical complementary feature fusion module, and the fusion module includes a feature interaction module, a context guide module, a residual inverting multi-layer perception module and a shortcut connection.

[0011] Further, the feature interaction module (FIM) concatenates the context features and the fine-grained features L in the channel dimension, realizes feature mapping through a 1x1 convolution, a ReLU rectified linear unit and a 1x1 convolution in series, and introduces a Sigmoid function at the end to generate an interaction weight map.

[0012] Further, the context guide module (CGM) generates a channel dimension dynamic convolution kernel with a size of bxcxk2x1x1 through a continuous 1x1 convolution, a nonlinear activation and a 1x1 convolution, at the same time, the input features G' are also mapped through a layer of 1x1 convolution to obtain a spatial dimension dynamic convolution kernel with a size of bk2xh xw.

[0013] Further, the fused features are input into the IRMLP.

[0014] Further, the IRMLP is composed of a 3x3 convolution, a ReLU activation, two layers of 1x1 convolution and BN.

[0015] Beneficial effects: 1. The feature interaction module dynamically adjusts the weights of the context and the fine-grained features, effectively suppresses the interference of complex habitat background on the fine-grained features, and improves the feature discriminability.

[0016] By the context guidance module, the complementarity of the context features and the fine-grained detail features is fully utilized to solve the problem of insufficient utilization of single features.

[0017] The dynamic convolution modulation under the guidance of the context information enhances the fine-grained feature expression of the small wild animal target and relieves the recognition error caused by the too small target.

[0018] The IRMLP optimizes the gradient flow through residual connection and nonlinear transformation to avoid gradient disappearance or network degradation in high-dimensional features and ensure effective aggregation of context and fine-grained features.

[0019] The residual optimization strategy combined with the shortcut connection reduces invalid feature interference and improves the robustness of species identification. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and are used to interpret the illustrative embodiments of the present application and their descriptions, and do not constitute improper limitations on the present application. In the drawings: Figure 1 is a flowchart of the present application; Figure 2 is a schematic diagram of the overall framework; Figure 3 is a schematic diagram of hierarchical complementary feature fusion (HCFF); Figure 4 is a schematic diagram of the context guidance module (GCM) flow; Figure 5 is a schematic diagram of the results compared with other classification models; Figure 6 is a schematic diagram of the ablation experiment results; DETAILED DESCRIPTION The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict, and the described embodiments are only part of the embodiments of the present application, not all the embodiments.

[0021] It should be noted that the descriptions of "left", "right", "left side", "right side", "upper part", "lower part", "top", "bottom", etc. in the present application are defined based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the described structure must be constructed and operated in a particular orientation, therefore, it cannot be understood as a limitation on the present application. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0022] In the description of the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connecting" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integrally connected; it can be directly connected, or indirectly connected through an intermediate medium, or internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0023] According to one aspect of the present application, a wild animal recognition method fusing fine-grained features and context features is provided, comprising the following steps: S1, image acquisition, an infrared camera acquires raw image data of wild animals.

[0024] S2, target positioning, inputting the collected raw image into a target detection network to extract four scales of context features.

[0025] The collected raw image is input into the target detection network, and the Swin-Transformer algorithm is used as the detection network. The detection network first generates candidate target regions through the region proposal network (RPN), and then the classification branch discriminates the target categories (such as animals or non-animals) of the candidate regions, while the regression branch finely adjusts the position of the candidate frame, so as to obtain accurate target positioning results. On this basis, the positioning results are mapped to the FPN feature map, and four scales of context features are extracted as the input of the HCFF module.

[0026] S3, feature acquisition, after the raw image is cropped, the pixel is adjusted to input the classification network to obtain four scales of fine-grained features.

[0027] After the target detection network obtains the target positioning result, we crop the raw image according to the positioning frame. Since the image classification network needs fixed size input, and the size of the cropped area is different, we use bilinear interpolation to uniformly adjust all the cropped images to 256x256 pixels, without considering the original aspect ratio. The image after cropping and scaling is input into the ConvNeXt v2 classification network. To extract features, and finally obtain four scales of fine-grained features and input them into the HCFF module.

[0028] S4, hierarchical complementary feature fusion, the context features and fine-grained features are sequentially subjected to hierarchical complementary feature fusion.

[0029] The multi-layer complementary feature fusion module HCFF includes: a feature interaction module (FIM), a context guidance module (CGM), an inverted residual multi-layer perception module (IRMLP), and a shortcut connection.

[0030] The image wild animal recognition network based on the Swin Transfomer-ConvNextv2 algorithm and the HCFF module is constructed. The input original image first enters the Swin Transformer target detection network, and the context features of four different scales are obtained based on the position information at the FPN feature map layer. The input animal image enters the ConvNextv2 network to obtain multi-scale fine-grained features. The constructed recognition network is trained, and the trained wild animal recognition network is optimized. The optimized recognition network accepts the original image input by the image acquisition and the animal image cropped based on the position information in the original image by the target detection network.

[0031] As shown in Figure 2 , first, a dual-branch feature extraction mechanism of context features and fine-grained features is constructed for the input original image. In the detection branch, the input image is processed by the initial block (Patch Partition), linear embedding (Linear Embedding), and multi-stage Swin Transformer Block in turn, and the positioning box of the target on the whole image is obtained through the precision regression. At the same time, the positioning box is mapped to the corresponding FPN feature map region through the target detection positioning box, so that the context features of multiple animals in a single image can be extracted and correspond to the extracted fine-grained features. In the classification branch, the target region obtained by positioning is first subjected to cropping and scaling operations, and then through convolution layers, layer normalization, and multi-stage processing, the fine-grained detail features of the target region are extracted. In the feature fusion stage, a hierarchical complementary feature fusion module (HCFF) is set at each stage of Stage1 to Stage4, and the context features and fine-grained features of the same scale mapped by the positioning box are fused. The fused features are then subjected to element-by-element addition operation with the fine-grained branch features to form enhanced features and passed backward, and the complementary enhancement of context semantics and fine-grained details is realized through iterative optimization.

[0032] As shown in Figure 3 , the multi-scale context features and fine-grained features are fused in the HCFF module. In the feature interaction module (FIM), the input context features and fine-grained features are spliced in the channel dimension, and the feature mapping is realized through the concatenation of 1×1 convolution, ReLU rectified linear unit, and 1×1 convolution, and the Sigmoid function is introduced at the end to generate the interaction weight map. The weight map is used to guide the element-by-element multiplication of the context features and the fine-grained features to obtain the enhanced context features G' and fine-grained features L', respectively. This process effectively models the dependency between context and fine-grained features and strengthens the interaction between features.

[0033] AsFigure 4 As shown, subsequently, in the context guidance module (CGM), the enhanced context features G' and the fine-grained features L' are taken as inputs, wherein the context features G' are globally averaged to obtain channel-level description vectors, and then, through a continuous 1x1 convolution, a nonlinear activation, and a 1x1 convolution, a channel-dimension dynamic convolution kernel with a size of bxcxk2x1x1 is generated; at the same time, the input features G' are also mapped through a layer of 1x1 convolution to obtain a spatial-dimension dynamic convolution kernel with a size of b x k2xh x w. The two are fused through element-wise addition to model the inter-channel dependency and the spatial position difference at the same time. On the other hand, the fine-grained features L' are first extracted through a 3x3 deep separable convolution to obtain basic fine-grained information, and then, element-wise convolution operation is performed with the fused dynamic convolution kernel to achieve adaptive enhancement of the features. Finally, the above enhanced features and the original input features are concatenated in the channel dimension to obtain b x 2c x h x w, and then output through 1x1 convolution dimension reduction to obtain the final context and fine-grained complementary features bxcxhwx. This design generates spatial dynamic convolution kernels and channel dynamic convolution kernels guided by context features, which are applied to fine-grained features to achieve multi-scale complementary fusion in spatial and channel dimensions. This way not only enhances the context awareness of fine-grained representation, but also avoids information conflicts caused by semantic alignment errors. The fused features are input to the IRMLP. The IRMLP consists of 3x3 convolution, ReLU activation, two layers of 1x1 convolution, and BN, wherein the residual connection structure is used to keep the gradient stable and reduce the risk of information loss. This module can further extract and reconstruct the feature distribution to achieve higher-level nonlinear modeling capability. The output of the IRMLP and the fine-grained features L are residual fused through element-wise addition to obtain the final output feature representation. This design ensures that fine-grained details will not be covered in the fusion process, while preserving the original discriminative features, effectively improving the overall recognition accuracy. Through the organic combination of feature interaction, context guidance module, and residual inverting multi-layer perceptron, this overall module realizes efficient complementarity of context and detail information, and can effectively improve the accuracy and robustness of wild animal species recognition in complex scenes.

[0034] S5, classification output, the fused features are processed through global average pooling, flattening, and fully connected layer to complete the recognition and classification of wild animal species.

[0035] As Figure 5As shown, with the introduction of each module, the classification accuracy (Acc%) and F1 score are continuously improved. The accuracy and F1 score of the benchmark model ConvNeXtV2 are 94.42% and 94.40% respectively; after adding Swin Transformer, they are improved to 95.70% and 95.67%, which proves that the context features effectively enhance the feature expression; after introducing FIM+CGM, the performance is significantly improved to 96.98% and 96.97%, which shows that the channel and spatial dynamic convolution fusion plays a key role in feature enhancement; further combined with IRMLP, the accuracy and F1 score reach 97.05% and 97.03%; finally, after adding the shortcut connection, the performance reaches 97.10% and 97.06%, which is the best result. In summary, each module in HCFF can effectively improve the model performance, especially the contribution of FIM+CGM and IRMLP is the most outstanding, and the final model breaks through 97% in accuracy and F1 score, which verifies the effectiveness and robustness of the method of the present application.

[0036] In the above description, the sensors, controllers and control programs that may be involved are all prior art and are not described in detail.

[0037] The above disclosed embodiments of the present application are only used to help explain the present application. The embodiments do not describe all the details, nor limit the application to the specific embodiments described. Based on the content of the present description, many modifications and changes can be made. The present description selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can well understand and utilize the present application.

Claims

1. A method of wild animal recognition fusing fine-grained features and contextual features, characterized in that, The method comprises the following steps: S1, image acquisition, an infrared camera collects original image data of wild animals; S2, target positioning, the collected original image is input into a target detection network to extract four-scale context features; S3, feature acquisition, after the original image is cropped, pixel adjustment is performed to input a classification network to finally obtain four-scale fine-grained features; S4, hierarchical complementary feature fusion, the context features and the fine-grained features are sequentially subjected to hierarchical complementary feature fusion; S5, classification output, the fused features are subjected to global average pooling, flattening and full connection layer processing to complete the identification and classification of wild animal species.

2. The method of claim 1, wherein the method comprises: The detection network adopts a Swin-Transformer algorithm.

3. The method of claim 2, wherein the method further comprises: The original image is cropped according to a positioning frame.

4. The method of claim 3, wherein the method further comprises: The adjusted pixels are 256x256 pixels.

5. The wild animal identification method of claim 4, wherein the classification network is a ConvNeXtv2.

6. The method according to claim 5, wherein, The S4 is specifically that after the multi-scale context features and the fine-grained features are obtained in the previous step, the features are sequentially subjected to a hierarchical complementary feature fusion module, and the fusion module comprises a feature interaction module, a context guide module, a residual inverting multi-layer perception module and a shortcut connection.

7. The method according to claim 6, wherein, The feature interaction module (FIM) concatenates the context features and the fine-grained features in the channel dimension, realizes feature mapping through a 1x1 convolution, a ReLU rectified linear unit and a 1x1 convolution in series, and introduces a Sigmoid function at the end to generate an interaction weight map.

8. The method of claim 7, wherein the method further comprises: The context guide module (CGM) generates a channel dimension dynamic convolution kernel with a size of bxcxk2x1x1 through a continuous 1x1 convolution, a nonlinear activation and a 1x1 convolution, at the same time, the input feature G' is also mapped through a layer of 1x1 convolution to obtain a spatial dimension dynamic convolution kernel with a size of bk2xhwx.

9. The method of claim 8, wherein the method further comprises: The fused features are input into an IRMLP.

10. The wild animal identification method of claim 9, wherein the IRMLP is composed of a 3x3 convolution, a ReLU activation, two layers of 1x1 convolution and BN.

Images