Wild animal image cross-domain identification method based on fusion perception adversarial network

Abstract

The invention provides a wild animal image cross-domain identification method based on a fusion perception adversarial network, and belongs to the technical field of image identification. The method comprises the following steps that ResNet50 is adopted as a backbone network to extract image bottom layer features, and semantic information of the image features is obtained by a model; an adversarial learning strategy is adopted to realize extraction of domain invariant features by the model; introducing a maximum mean value difference constraint between the fully connected layers, and expanding the domain invariant feature extraction capability of the adversarial network under a complex background; domain invariant features are extracted through a two-layer domain adaptation strategy, positive migration of the model is enhanced, and cross-domain recognition of wild animal images is achieved; compared with the prior art, the method effectively improves the processing efficiency of large-scale image data of wild animals and the recognition accuracy of wild animal images in different environments.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a cross-domain recognition method for wildlife images based on a fusion perception adversarial network. Background Art

[0002] Wildlife images collected at different time and spatial scales have domain shift problems such as different backgrounds and animal postures, which makes the existing models unable to adapt well to new out-of-distribution datasets, which greatly limits the effectiveness of wildlife recognition technology in practical applications. In addition, in wildlife recognition tasks, existing domain adaptation methods are easily disturbed by complex wild environments, affecting recognition performance.

[0003] Based on the above problems, it is necessary to propose a high-performance domain adaptation method that can ensure the correct extraction of wildlife domain-invariant features under the influence of complex backgrounds in order to achieve cross-domain recognition of wildlife images. Summary of the invention

[0004] The purpose of the present invention is to propose a cross-domain wildlife image recognition method based on a fusion perceptual adversarial network to solve the problem that wildlife images are affected by different camera locations, data acquisition times and acquisition devices, resulting in distribution differences.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A cross-domain recognition method for wildlife images based on a fusion perceptual adversarial network includes the following contents:

[0007] S1. Build a wildlife image recognition model and use ResNet50 as the backbone network to extract the underlying features of the image to obtain the semantic information of the image features;

[0008] S2, using adversarial learning strategy and using adversarial network to extract domain invariant features;

[0009] S3, using a distribution alignment module to expand the ability of adversarial networks to extract domain-invariant features in complex backgrounds;

[0010] S4. A fusion-perceptual adversarial network is used to extract domain-invariant features to achieve cross-domain recognition of wildlife images.

[0011] Preferably, the use of ResNet50 as the backbone network in S1 specifically refers to: selecting ResNet50 as the feature extractor, in the ResNet50 structure, the input image size is set to 224×224×3 pixels, first passing through a preprocessing convolution layer, the preprocessing convolution layer includes a 7×7 convolution layer, a batch normalization layer (BN), an activation layer (Relu) and a maximum pooling layer (Max pool); secondly, passing through four residual stages, each residual stage is composed of a different number (3, 4, 6, 3) of residual blocks.

[0012] Preferably, S2 specifically includes the following contents:

[0013] Let f represent the features extracted by the feature extractor and c represent the prediction results of the classifier. Then the training function of the domain discriminator D is expressed as:

[0014]

[0015] Among them, (f,c) represents the joint variable of the feature f extracted by the feature extractor and the prediction result c of the classifier; Represents a linear mapping, and its calculation formula is as follows:

[0016]

[0017] Among them, d f and d c Represent the dimensions of vectors f and c respectively;

[0018] If a multilinear map The dimension is greater than 4096, the model chooses randomized multilinear mapping The specific function is expressed as:

[0019]

[0020] Where d satisfies d≤d f ×d c , R f and R c Represents a random matrix that is sampled once during training.

[0021] Preferably, S3 specifically includes the following contents:

[0022] The distribution alignment metric forces the model to further minimize the difference between the source and target domains, making the model more generalizable between the source and target domains and classifying unlabeled target data; the distribution alignment loss measured on the fully connected layer is used to guide transfer learning in the convolutional layer to extract more transferable sample feature representations and reduce the impact of irrelevant background information.

[0023] Preferably, the S4 specifically includes the following contents:

[0024] Firstly, an adversarial domain adaptation network is used to preliminarily extract domain invariant features. Then, a distribution alignment module is introduced between the last fully connected layer of the source and target feature extraction networks to minimize the distribution difference between the source and target data again. The fusion-aware adversarial network model enhances the adaptability of the model by weighting the loss of the domain discriminator, the cross-entropy classification loss and the maximum average difference metric for each category.

[0025] Compared with the prior art, the present invention provides an image cross-domain recognition method based on a fusion perceptual adversarial network, which has the following beneficial effects:

[0026] The method of the invention realizes cross-domain identification of wild animals, provides a reliable method for the automatic identification of wild animal images, and effectively improves the processing efficiency of large-scale image data of wild animals and the recognition accuracy of wild animal images in different environments. In the face of complex and changeable outdoor environments, this technology can overcome challenges such as lighting and angle changes to achieve accurate identification of wild animals. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flowchart of the cross-domain identification method for wildlife images based on a fusion perception adversarial network mentioned in Example 1 of the present invention;

[0028] Figure 2 This is a structural diagram of the feature extractor based on ResNet50 mentioned in Example 1 of the present invention. DETAILED DESCRIPTION

[0029] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0030] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0031] Embodiment 1:

[0032] See also Figure 1 The present invention proposes a cross-domain identification method for wildlife images based on a fusion perception adversarial network, comprising the following steps:

[0033] S1. Build a wildlife image recognition model, use ResNet50 as the backbone network to extract the underlying features of the image, and enable the model to obtain the semantic information of the image features;

[0034] See also Figure 2 , Using ResNet50 as the backbone network in S1 specifically means: selecting ResNet50 as the feature extractor, such as Figure 2 As shown in the figure, in the ResNet50 structure, the input image size is set to 224×224×3 pixels. It first passes through a preprocessing convolution layer, which includes a 7×7 convolution layer, a batch normalization layer (BN), an activation layer (Relu) and a maximum pooling layer (Max pool); secondly, it passes through four residual stages, each of which is composed of a different number (3, 4, 6, 3) of residual blocks.

[0035] S2. Adopt adversarial learning strategy to realize the extraction of domain invariant features by the model, which includes the following contents:

[0036] Let f represent the features extracted by the feature extractor and c represent the prediction results of the classifier. Then the training function of the domain discriminator D is expressed as:

[0037]

[0038] Among them, (f,c) represents the joint variable of the feature f extracted by the feature extractor and the prediction result c of the classifier; Represents a linear mapping, and its calculation formula is as follows:

[0039]

[0040] Among them, d f and d c Represent the dimensions of vectors f and c respectively;

[0041] If a multilinear map The dimension is greater than 4096, the model chooses randomized multilinear mapping The specific function is expressed as:

[0042]

[0043] Where d satisfies d≤d f ×d c , R f and R c Represents a random matrix that is sampled once during training.

[0044] S3, using the distribution alignment module to expand the ability of adversarial networks to extract domain-invariant features in complex backgrounds, including the following:

[0045] The distribution alignment metric forces the model to further minimize the difference between the source and target domains, making the model more generalizable between the source and target domains and classifying unlabeled target data; the distribution alignment loss measured on the fully connected layer is used to guide transfer learning in the convolutional layer to extract more transferable sample feature representations and reduce the impact of irrelevant background information.

[0046] S4. Use a fusion-perceptual adversarial network to extract domain-invariant features to achieve cross-domain recognition of wildlife images, including the following:

[0047] Firstly, an adversarial domain adaptation network is used to preliminarily extract domain invariant features. Then, a distribution alignment module is introduced between the last fully connected layer of the source and target feature extraction networks to minimize the distribution difference between the source and target data again. The fusion-aware adversarial network model enhances the adaptability of the model by weighting the loss of the domain discriminator, the cross-entropy classification loss and the maximum average difference metric for each category.

[0048] Embodiment 2:

[0049] Based on Example 1, but different in that the comparison of experimental results of different models based on the self-built North American wildlife data set is shown in Table 1. According to the experimental results, the FAN method proposed in the present invention has achieved the best results in each task, which means that the model of the present invention can better extract domain invariant features under the influence of complex backgrounds and achieve better classification results. In the S1→S2 task, the classification accuracy of the method of the present invention reached 47.4%, which is 1.9 percentage points higher than the suboptimal comparison method CDAN; in terms of F1 score, the F1 score of the method of the present invention is 39.1%, which is 1.3 percentage points higher than the suboptimal comparison method DALN. In the S2→S1 task, the classification accuracy of the method of the present invention reached 64.1%, which is 4 percentage points higher than the suboptimal comparison method BNM; the F1 score of the method of the present invention is 51.1%, which is 0.4 percentage points higher than the suboptimal comparison method BNM. In terms of average accuracy and average F1 score, the method of the present invention achieved an average accuracy of 55.8% and an average F1 score of 45.1%, which are significantly improved compared with the comparison method. This verifies that the model can better balance the precision and recall rate on the basis of improving the classification accuracy.

[0050] Table 1 Comparison of experimental results of different domain adaptation methods based on North American wildlife dataset

[0051]

[0052] In order to verify the generalization of the FAN method, the present invention conducted an experimental comparison based on the self-built RC wildlife dataset, and the experimental results are shown in Table 2. In the R→C task, the classification accuracy reached 90.0%, which was 0.3 percentage points higher than the DSAN method; the F1 score was 86.4%, which was 2.4 percentage points higher than the DSAN method; in the C→R task, the method in this chapter achieved a classification accuracy of 83.2%, which was 0.6 percentage points higher than the DSAN method. The F1 score of the method in this chapter was lower than the best performing DSAN method, but higher than other comparison methods. In terms of average accuracy and average F1 score, the method in this chapter achieved the best results, with an average accuracy of 86.6% and an average F1 score of 80.3%, which verified the generalization of the method and showed that the method can effectively handle transfer learning tasks between different fields and enhance the recognition performance of the model.

[0053] Table 2 Comparison of experimental results of different models based on RC wildlife dataset

[0054]

[0055] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A cross-domain recognition method for wildlife images based on a fusion perceptual adversarial network, characterized in that: It includes the following: S1. Build a wildlife image recognition model and use ResNet50 as the backbone network to extract the underlying features of the image to obtain the semantic information of the image features; S2, using adversarial learning strategy and using adversarial network to extract domain invariant features; S3, using a distribution alignment module to expand the ability of adversarial networks to extract domain-invariant features in complex backgrounds; S4. A fusion-perceptual adversarial network is used to extract domain-invariant features to achieve cross-domain recognition of wildlife images.

2. The cross-domain identification method of wildlife images based on fusion perceptual adversarial network according to claim 1 is characterized in that: The use of ResNet50 as the backbone network in S1 specifically refers to: selecting ResNet50 as a feature extractor, in the ResNet50 structure, the input image size is set to 224×224×3 pixels, and first passes through a preprocessing convolution layer, the preprocessing convolution layer includes a 7×7 convolution layer, a batch normalization layer, an activation layer, and a maximum pooling layer; Secondly, it goes through four residual stages, each of which consists of a different number of residual blocks.

3. The cross-domain identification method of wildlife images based on fusion perceptual adversarial network according to claim 2 is characterized in that: The S2 specifically includes the following contents: Let f represent the features extracted by the feature extractor and c represent the prediction results of the classifier. Then the training function of the domain discriminator D is expressed as: Among them, (f,c) represents the joint variable of the feature f extracted by the feature extractor and the prediction result c of the classifier; Represents a linear mapping, and its calculation formula is as follows: Among them, d f and d c Represent the dimensions of vectors f and c respectively; If a multilinear map The dimension is greater than 4096, the model chooses randomized multilinear mapping The specific function is expressed as: Where d satisfies d≤d f ×d c , R f and R c Represents a random matrix that is sampled once during training.

4. The cross-domain identification method of wildlife images based on fusion perceptual adversarial network according to claim 3 is characterized in that: The S3 specifically includes the following contents: The distribution alignment metric forces the model to further minimize the difference between the source and target domains, making the model more generalizable between the source and target domains and classifying unlabeled target data; the distribution alignment loss measured on the fully connected layer is used to guide transfer learning in the convolutional layer to extract more transferable sample feature representations and reduce the impact of irrelevant background information.

5. The cross-domain identification method of wildlife images based on fusion perceptual adversarial network according to claim 4 is characterized in that: The S4 specifically includes the following contents: Firstly, an adversarial domain adaptation network is used to preliminarily extract domain invariant features. Then, a distribution alignment module is introduced between the last fully connected layer of the source and target feature extraction networks to minimize the distribution difference between the source and target data again. The fusion-aware adversarial network model enhances the adaptability of the model by weighting the loss of the domain discriminator, the cross-entropy classification loss and the maximum average difference metric for each category.

Images