A remote sensing image feature extraction method based on hierarchical feature autonomous learning

Abstract

The present invention discloses a remote sensing image feature extraction method based on hierarchical feature autonomous learning, comprising: performing two random cropping operations on an input remote sensing image x to obtain two different views of the input remote sensing image x, denoted as x a ,x b ; for view x a Perform random masking to obtain the mask image x m ; for view x b Perform random image transformation to obtain the transformed image x g ; The mask image x m Input into the student model to get the mask image x m The feature map f m ; Transform the image x g Input into the teacher model to get the transformed image x g The feature map f g ; According to the feature map f m , learn pixel-level image features; based on the feature map f m and feature map f g , learning object-level target separability features and learning scene-level features.

Description

Technical Field

[0001] The present invention relates to the technical field of image feature extraction, and in particular to a remote sensing image feature extraction method based on autonomous hierarchical feature learning without relying on manually labeled data. Background Art

[0002] The primary goal of feature extraction is to extract effective features from raw input imagery and use these features to represent the raw data. The extracted features can better describe the inherent structure that influences the data and effectively suppress noise and other information irrelevant to the task. Furthermore, the extracted features are invariant, remaining stable despite changes in imaging conditions. This makes them suitable for intelligent interpretation of remote sensing images under diverse conditions. In recent years, supervised and self-supervised learning methods, with their powerful feature learning capabilities, have gradually replaced handcrafted feature extraction methods, achieving breakthrough progress in the field of feature extraction.

[0003] Supervised learning methods learn image features from large-scale data in an "end-to-end" manner, saving the time and effort of manually designing features. In addition, the features extracted by supervised learning are highly robust and can be applied to image understanding tasks in different scenarios.

[0004] Despite the tremendous success of supervised learning-based feature extraction methods, this form of supervised learning still has shortcomings and limitations. First, from a data perspective, the success of deep neural networks lies in their ability to fit large amounts of annotated samples without significantly sacrificing generalization ability. However, in the field of remote sensing, annotating remote sensing data requires considerable time and manpower. Remote sensing data inherently features multiple sensors, multiple temporal phases, and multiple resolutions. Therefore, constructing a large-scale, high-quality, and comprehensive remote sensing scene classification dataset presents numerous challenges. Furthermore, supervised learning methods learn from a limited number of samples, which can easily lead to performance degradation due to the closed nature of the samples and the dynamic variability of their features. Furthermore, supervised learning relies on semantic support provided by human annotations as the sole learning signal for model learning. If human annotations are used as prior knowledge, the annotation process effectively constrains the machine's knowledge. However, the inherent information in massive remote sensing data should theoretically be much richer than the semantic information provided by sparse labels. Therefore, over-reliance on manual annotations risks inducing "inductive bias" in the resulting models.

[0005] Self-supervised learning methods can effectively solve the above problems. They use artificially designed self-supervised learning signals to mine their own supervisory information from massive unlabeled data to form pseudo-labels, thereby replacing manually labeled data to drive the model for feature learning. Currently, commonly used self-supervised learning methods can be divided into two categories: generative and contrastive. The basic idea of ​​the generative self-supervised learning method is to achieve the purpose of feature learning by restoring the image that is considered to be damaged. The motivation behind it is that once the model can completely restore the original image, it means that the model has the ability to extract the key features of the original image. The core idea of ​​the contrastive self-supervised learning method is to shorten the distance between different transformed views of the same image in the feature space and increase the distance between different images in the feature space, so as to achieve the purpose of learning feature expressions that are both invariant and distinguishable.

[0006] At present, the main implementation schemes for image feature extraction based on self-supervised learning are:

[0007] Step 1: Build a large-scale unlabeled remote sensing image dataset.

[0008] Step 2: Based on the needs of downstream applications, design appropriate self-supervised learning signals (such as instance discrimination signals for scene classification tasks and mask reconstruction signals for semantic segmentation tasks), and pre-train on the constructed unlabeled remote sensing image dataset to extract rich image features.

[0009] Step 3: Fine-tune the self-supervised pre-trained model to specific downstream tasks of remote sensing images (such as scene classification, semantic segmentation, change detection, object detection, etc.) using a small amount of labeled data.

[0010] The performance of the pre-trained model is evaluated based on the evaluation metrics used in downstream tasks, which serves as a criterion for evaluating the quality of self-supervised learning methods. Finally, the trained model is applied to real-world remote sensing imagery applications.

[0011] Current self-supervised learning methods in remote sensing are limited to using either generative or contrastive self-supervised learning signals. These signals, however, have limited representational capabilities for the features they extract. Generative signals model inter-pixel contextual features, limiting their ability to extract pixel-level features; contrastive signals model global semantic relationships between images, limiting their ability to extract scene-level features. The hierarchical construction of features in remote sensing images, from microscopic to macroscopic, relies on a comprehensive, multi-layered feature representation to enhance scene perception and recognition. Therefore, a feature extraction method that autonomously learns hierarchical features is urgently needed. Summary of the Invention

[0012] Purpose of the invention: To address the problem that current self-supervised feature learning methods are unable to extract hierarchical features, the present invention proposes a remote sensing image feature extraction method based on autonomous learning of hierarchical features. By combining multiple self-supervised learning signals through multi-dimensional projection space and teacher-student knowledge distillation network, the semantic confusion problem faced by different self-supervised learning signals in the combination process is solved, the intrinsic information in massive unlabeled remote sensing data is mined, and pixel-object-scene hierarchical image features are extracted.

[0013] Technical solution: A remote sensing image feature extraction method based on hierarchical feature autonomous learning, including the following steps:

[0014] The input remote sensing image x is randomly cropped twice to obtain two different views of the input remote sensing image x, denoted as x a ,x b ; for view x a Perform random masking to obtain the mask image x m ; for view x b Perform random image transformation to obtain the transformed image x g ; The mask image x m Input into the student model to get the mask image x m The feature map f m ; Transform the image x g Input into the teacher model to get the transformed image x g The feature map f g ;

[0015] The feature map f m Input to the reconstruction decoder, through the reconstructed view x a To learn pixel-level image features;

[0016] According to the cropping parameters of random cropping, the feature map f is calculated m and feature map f g The position encoding of each feature vector in , and based on the position encoding, calculate the feature map f m With the feature map f g The Euclidean distance between feature vectors is calculated, and N feature vectors with the closest Euclidean distance are selected as sample pairs for position matching. The sample pairs are projected into another feature space through local projection. The projected sample pairs are respectively mapped into learnable prototypes and the probability distribution of each feature vector with respect to the prototypes is obtained through the Softmax operation. The semantic information between the two feature vectors in the sample pair is aligned by calculating the cross entropy loss function between the probability distributions corresponding to the sample pairs to learn the target separability features at the object level.

[0017] Feature map f m and feature map f gObtain the transformed image x by global pooling g With the mask image x m The scene-level global features of the student model are optimized by the InfoNCE loss function to shorten the distance between the global features of the student model and the teacher model in the feature space and to further distance them from other features in the visual dictionary to learn scene-level features.

[0018] Further, view x a and view x b The overlapping area between them is greater than 50% of the area size of the input remote sensing image x.

[0019] Furthermore, the view x a Perform random masking to obtain the mask image x m , specifically including:

[0020] Random mask view x a 60% of the pixels in the image are replaced by the spectral mean to obtain the mask image x m .

[0021] Furthermore, the student model and the teacher model adopt the same network architecture;

[0022] During the student model training process, the parameters of the student model are updated through the optimization algorithm;

[0023] During the teacher model training process, the parameters of the teacher model are updated by the sliding exponential average method:

[0024] The sliding exponential averaging method comprises:

[0025] Assuming that the parameters of the student model are θ, the parameters of the teacher model are δ, and m is the momentum parameter in the sliding exponential average method, the parameters of the teacher model are updated as follows:

[0026] δ t =(1-m)θ t +mδ t-1 (1)

[0027] Where t represents the tth iteration process.

[0028] Furthermore, the reconstruction decoder is a multi-layer perceptron having one hidden layer.

[0029] Furthermore, the feature map f m Input to the reconstruction decoder, through the reconstructed view x a To learn pixel-level image features, specifically including:

[0030] By calculating and minimizing the reconstruction result and the reconstructed view xa The l2 loss function in the spatial domain and frequency domain is used to obtain the reconstruction results in the spatial domain and frequency domain.

[0031]

[0032]

[0033] Among them, LL, L f Respectively, the reconstruction results and reconstructed view x in spatial domain and frequency domain a The l2 loss function of , Fourier(·) is the Fourier transform.

[0034] Furthermore, the feature map f is calculated based on the cropping parameters of random cropping. m and feature map f g The position encoding of each feature vector in , specifically includes:

[0035] Assume that the cropping parameters are: the coordinates of the upper left corner are (i2, j2), and the width and height of the crop are w2, h2;

[0036] The position encoding (l1, l2) of the feature vector at the feature map (u, v) is:

[0037]

[0038]

[0039] Where H input 、W input is the height and width of the input of the student model teacher model, and H and W are the height and width of the feature map.

[0040] Furthermore, the projected sample pairs are mapped to learnable prototypes and the probability distribution of each feature vector with respect to the prototypes is obtained through the Softmax operation, specifically including:

[0041]

[0042]

[0043] Among them, p i ,q i are the probability distributions of the sample pairs with matching positions between the student model and the teacher model on Prototypes; τ s ,τ tare the temperature parameters of the student model and the teacher model respectively, <·,·> is the cosine similarity; C is the learnable prototypes; the prototypes are obtained through the learnable parameters of a linear fully connected layer;

[0044] The method of aligning the semantic information between the two feature vectors in the sample pair by calculating the cross entropy loss function between the probability distributions corresponding to the sample pair to learn the target separability features at the object level specifically includes:

[0045]

[0046] Among them, L local is the cross entropy loss function, and N is the number of matching samples.

[0047] Furthermore, the InfoNCE loss function is expressed as:

[0048]

[0049] Where <·,·> represents the cosine similarity; τ is the temperature coefficient that controls the distance between positive and negative samples; k - is the feature vector in the visual dictionary; q and k are respectively the mask image x m With the transformed image x g The feature map f m , f g Scene-level global features of the transformed image and mask image obtained by global pooling.

[0050] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0051] (1) The method of the present invention can mine the intrinsic features of massive unlabeled remote sensing data, fully utilize the advantages of massive remote sensing image data, and extract the all-round hierarchical features of "pixel-object-scene";

[0052] (2) Using the model pre-trained by the method of the present invention as the backbone network for various remote sensing image interpretation tasks (scene classification, semantic segmentation, target detection, change detection), the accuracy of remote sensing image interpretation can be greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Schematic diagram of the autonomous learning method for the "pixel-object-scene" hierarchical features of remote sensing images;

[0054] Figure 2 Schematic diagram of random transformation operation;

[0055] Figure 3 Schematic diagram for pixel-level feature learning;

[0056] Figure 4 Learning schematics for object-level features;

[0057] Figure 5 Learning schematics for scene-level features;

[0058] Figure 6 This is a comparison chart of scene classification accuracy;

[0059] Figure 7 is the semantic segmentation result map;

[0060] Figure 8 is the target detection result map;

[0061] Figure 9 This is a comparison chart of change detection results;

[0062] Figure 10 Visualize the results for t-SNE;

[0063] Figure 11 Visualize the results for the attention map;

[0064] Figure 12 This is the feature point matching result diagram of the method of the present invention. DETAILED DESCRIPTION

[0065] The technical solution of the present invention will now be further described with reference to the accompanying drawings and embodiments.

[0066] Example:

[0067] like Figure 1 As shown, this embodiment discloses a remote sensing image feature extraction method based on hierarchical feature autonomous learning, which mainly includes the following steps:

[0068] Step 1: Perform two random cropping operations on the input remote sensing image x to obtain two different views x of the input remote sensing image x. a ,x b , and record their respective cropping parameters. When cropping, ensure that the two views x a ,x b The overlapping area is greater than 50% of the input remote sensing image area x. The cropping parameters include the coordinates of the upper left corner, the width and height of the crop.

[0069] Step 2: For two different views x a ,x b Perform random mask and random image transformation to obtain mask image x m and the transformed image x g; For the random mask operation, 60% of the pixels in the random mask image are randomly masked and the pixel values ​​of the masked pixels are replaced by the spectral mean; for the random transformation operation, the image is flipped, randomly rotated, randomly colored, Gaussian blurred, Gaussian noised, grayscaled, etc., see Figure 2 .

[0070] Step 3: Mask image x m and the transformed image x g Input to the student model and teacher model respectively, and the student model and teacher model output mask image x respectively m With the transformed image x g The feature map f m ,f g ; The student model and teacher model of this embodiment use the same network architecture (convolutional neural network or Transformer). During the model training process, the student model updates its parameters through an optimization algorithm. The teacher model parameters are updated using the Exponential Moving Average (EMA) method: Assuming that the parameters of the student model are θ, the parameters of the teacher model are δ, and m is the momentum parameter in the EMA method, the parameters of the teacher model are updated using the following formula:

[0071] δ t =(1-m)θ t +mδ t-1 (1)

[0072] The subscript t represents the parameter in the t-th iteration process.

[0073] Step 4: Pixel-level feature learning: feature map f output by the student model m Input to the reconstruction decoder to predict the pixel value of the masked pixel in step 2 and then reconstruct the image x a , and thereby learn pixel-level image features. The reconstruction decoder is a multi-layer perceptron (MLP) with one hidden layer. There are a large number of small targets in remote sensing images and they are closely arranged. Large-scale masking operations may cause the loss of small targets in the image, thereby increasing the difficulty of reconstruction and causing feature semantic loss. Compared with other self-supervised learning methods that directly use zero values ​​to fill the masked pixels, it is proposed in step 2 to use the spectral mean to fill the masked pixels. The spectral mean is the DC component of the image and carries part of the semantic information about the image. Therefore, it can provide semantic priors for the reconstruction task and make up for the problem of missing image semantics. On the other hand, image reconstruction in the original spatial domain can only capture the short-distance dependencies between pixels. It is proposed to reconstruct the consistency of the image and the original image in the frequency domain to further capture the long-distance semantic dependencies between pixels and improve the expression ability of the pixel-level feature map. See. Figure 3 Specifically, the reconstruction decoder outputs the reconstruction results in the spatial domain and frequency domain The consistency between the predicted results and the real results is achieved by calculating and minimizing the l2 loss function between the reconstructed image and the real image in the spatial domain and frequency domain:

[0074]

[0075]

[0076] Among them, LL, L f where is the l² loss function between the reconstructed image and the original image in the spatial and frequency domains, respectively, and Fourier(·) is the Fourier transform. Spatial domain consistency focuses on learning the semantic relationships between pixel neighborhoods within an image. In the frequency domain, each pixel value in the Fourier transformed image is the result of the interaction between that pixel and all other pixels in the entire image. Therefore, frequency domain consistency can capture long-range semantic relationships between pixels within an image.

[0077] Step 5: Object-level feature learning: First, calculate the feature map f according to the cropping parameters of the cropped image in step 1 m ,f g Assume that the random cropping parameters are: the coordinates of the upper left corner (i2, j2) and the width and height of the cropped image w2, h2; the input size of the student model and the teacher model is H input ×W input , the feature map size is H×W. Then the position encoding (l1,l2) of the feature vector at the position (u,v) of the feature map is:

[0078]

[0079]

[0080] According to the position encoding of each feature vector in the feature map, the Euclidean distance between the feature vectors of the output feature maps of the student model and the teacher model is calculated, and the N feature vectors with the closest distance are selected as the position matching sample pairs. The sample pairs as position matching are projected into another feature space through local projection to prevent semantic confusion with the features learned by the other two branches (pixel-level feature learning and scene-level feature learning). The projected sample pairs as position matching are mapped to learnable Prototypes and the Softmax operation is used to obtain the probability distribution p of each feature matching vector corresponding to the sample pairs as position matching with respect to Prototypes. i ,q i :

[0081]

[0082]

[0083] Among them, p i ,q i are the soft probability distributions of the feature vectors of position matching between the student model and the teacher model with respect to Prototypes; τ s ,τ t are the temperature parameters of the student model and the teacher model, respectively, and the temperature parameter τ of the teacher model t smaller than the temperature coefficient τ of the Student model s To ensure that the probability distribution of different types of objects in the teacher model is more concentrated, thereby guiding the student model to learn object-level separability features; <·,·> is the cosine similarity; C is the learnable prototypes.

[0084] By calculating the cross entropy loss function L between the probability distributions corresponding to the sample pairs as position matching local To align the semantic information between the two feature vectors in the sample pair as position matching to learn the object-level target separability features ( Figure 4 ):

[0085]

[0086] Where N is the number of matching samples.

[0087] Prototypes are obtained through the learnable parameters of a linear fully connected layer. During the training process, the network optimizes the parameters of the prototypes while backpropagating to optimize the model parameters. This allows the prototypes to autonomously optimize and fit the features of different land objects in the dataset during training to guide the student network to extract high-level semantic features of the land objects.

[0088] Step 6: Scene-level feature learning: This step uses contrastive self-supervised learning to learn scene-level invariance and separability features. The random image transformation method in step 2 generates a transformed image, simulating the spatiotemporal differences of images caused by different imaging conditions, and transforms the mask image x m With the transformed image x g The feature map f m , f g The scene-level global features q and k of the transformed image and the mask image are obtained by global pooling, and finally the InfoNCE loss function L is optimized. NCE To narrow the distance between the global features of the student model and the teacher model in the feature space and push the distance between them and other features (negative samples) in the visual dictionary, and learn scene-level features that are both globally separable and spatiotemporally invariant ( Figure 5 ).

[0089] The specific expression of InfoNCE loss function is as follows:

[0090]

[0091] Where <·,·> represents the cosine similarity; τ is the temperature coefficient that controls the distance between positive and negative samples; k - is the feature vector in the visual dictionary.

[0092] During the training process, the visual dictionary is updated in each iteration through the global features output by the teacher model, thereby increasing the richness of negative samples and thus improving the separation of global features between different scenes.

[0093] The self-supervised learning method CMID proposed in this embodiment can mine the intrinsic features of massive unlabeled remote sensing data, make full use of the advantages of massive remote sensing image data, and extract the all-round hierarchical features of "pixel-object-scene". The model pre-trained by this method is used as the backbone network for various remote sensing image interpretation tasks (scene classification, semantic segmentation, target detection, change detection), which can greatly improve the accuracy of remote sensing image interpretation. For example, the convolutional neural networks ResNet50 (CMID-ResNet50) and Transformer (CMID-Swin-B) pre-trained by CMID have shown outstanding performance in various remote sensing downstream tasks: In the scene classification task, this method (CMID-Swin-B and CMID-ResNet50) achieved the highest accuracy, especially when the training samples were small, the classification accuracy of this method was much higher than that of other mainstream self-supervised learning methods ( Figure 6 ); In the semantic segmentation task, compared with other self-supervised pre-training models, the models pre-trained by this method (CMID-Swin-B and CMID-ResNet50) achieved more accurate segmentation results ( Figure 7 ); In the target detection results, the target detection model (CMID-Swin-B) pre-trained by this method can accurately detect the ground objects in the image ( Figure 8 ); In the change detection task, the change detection model pre-trained by this method (CMID-Swin-B and CMID-ResNet50) can accurately locate the changed objects in the image and eliminate the false changes in the image ( Figure 9 ).

[0094] The representation ability of the features extracted by the method is verified by various feature visualization methods: t-SNE feature visualization results show that the method of the present invention extracts more separable scene-level features ( Figure 10 ); The attention map visualization results show that the features extracted by this method pay more attention to the main objects in the image ( Figure 11); The feature point matching results show that the invented method can accurately match the feature similarities between two different images. When the same image undergoes significant changes, the method can also accurately locate the same-name points between the images before and after the transformation ( Figure 12 ).

[0095] In summary, the present invention can effectively extract the “pixel-object-scene” hierarchical features and significantly improve the performance of the model in various remote sensing image downstream tasks.

Claims

1. A remote sensing image feature extraction method based on hierarchical feature autonomous learning, characterized by: The following steps are involved: The input remote sensing image x is randomly cropped twice to obtain two different views of the input remote sensing image x, denoted as x a ,x b ; for view x a Perform random masking to obtain the mask image x m ; for view x b Perform random image transformation to obtain the transformed image x g ; The mask image x m Input into the student model to get the mask image x m The feature map f m ; Transform the image x g Input into the teacher model to get the transformed image x g The feature map f g ; The feature map f m Input to the reconstruction decoder, through the reconstructed view x a To learn pixel-level image features; According to the cropping parameters of random cropping, the feature map f is calculated m and feature map f g The position encoding of each feature vector in , and based on the position encoding, calculate the feature map f m With the feature map f g The Euclidean distance between feature vectors is calculated, and N feature vectors with the closest Euclidean distance are selected as sample pairs for position matching. The sample pairs are projected into another feature space through local projection. The projected sample pairs are respectively mapped into learnable prototypes and the probability distribution of each feature vector with respect to the prototypes is obtained through the Softmax operation. The semantic information between the two feature vectors in the sample pair is aligned by calculating the cross entropy loss function between the probability distributions corresponding to the sample pairs to learn the target separability features at the object level. Feature map f m and feature map f g Obtain the transformed image x by global pooling g With the mask image x m The scene-level global features of the student model are optimized by the InfoNCE loss function to shorten the distance between the global features of the student model and the teacher model in the feature space and to further distance them from other features in the visual dictionary to learn scene-level features.

2. The remote sensing image feature extraction method of hierarchical feature autonomous learning according to claim 1, characterized in that: View x a and view x b The overlapping area between them is greater than 50% of the area size of the input remote sensing image x.

3. The remote sensing image feature extraction method of hierarchical feature autonomous learning according to claim 1, characterized in that: The pair view x a Perform random masking to obtain the mask image x m , specifically including: Random mask view x a 60% of the pixels in the image are replaced by the spectral mean to obtain the mask image x m .

4. The remote sensing image feature extraction method of hierarchical feature autonomous learning according to claim 1, characterized in that: The student model and the teacher model use the same network architecture; During the student model training process, the parameters of the student model are updated through the optimization algorithm; During the teacher model training process, the parameters of the teacher model are updated by the sliding exponential average method: The sliding exponential averaging method comprises: Assuming that the parameters of the student model are θ, the parameters of the teacher model are δ, and m is the momentum parameter in the sliding exponential average method, the parameters of the teacher model are updated as follows: d t =(1-m)θ t +mδ t-1 (1) Where t represents the tth iteration process.

5. The remote sensing image feature extraction method of hierarchical feature autonomous learning according to claim 1, characterized in that: The reconstruction decoder is a multi-layer perceptron with one hidden layer.

6. The remote sensing image feature extraction method of hierarchical feature autonomous learning according to claim 1, characterized in that: The feature map f m Input to the reconstruction decoder, through the reconstructed view x a To learn pixel-level image features, specifically including: By calculating and minimizing the reconstruction result and the reconstructed view x a The l2 loss function in the spatial domain and frequency domain is used to obtain the reconstruction results in the spatial domain and frequency domain. Among them, L, L f Respectively, the reconstruction results and reconstructed view x in spatial domain and frequency domain a The l2 loss function of , Fourier(·) is the Fourier transform.

7. The remote sensing image feature extraction method of hierarchical feature autonomous learning according to claim 1, characterized in that: The feature map f is calculated based on the cropping parameters of random cropping. m and feature map f g The position encoding of each feature vector in , specifically includes: Assume that the cropping parameters are: the coordinates of the upper left corner are (i2, j2), and the width and height of the crop are w2, h2; The position encoding (l1, l2) of the feature vector at the feature map (u, v) is: Where H input 、W input is the height and width of the input of the student model teacher model, and H and W are the height and width of the feature map.

8. The remote sensing image feature extraction method of hierarchical feature autonomous learning according to claim 7, characterized in that: The projected sample pairs are mapped to learnable prototypes and the probability distribution of each feature vector with respect to the prototypes is obtained through the Softmax operation, specifically including: Among them, p i ,q i are the probability distributions of the sample pairs with matching positions between the student model and the teacher model on Prototypes; τ s ,τ t are the temperature parameters of the student model and the teacher model respectively, <·,·> is the cosine similarity; C is the learnable prototypes; the prototypes are obtained through the learnable parameters of a linear fully connected layer; The method of aligning the semantic information between the two feature vectors in the sample pair by calculating the cross entropy loss function between the probability distributions corresponding to the sample pair to learn the target separability features at the object level specifically includes: Among them, L local is the cross entropy loss function, and N is the number of matching samples.

9. The remote sensing image feature extraction method of hierarchical feature autonomous learning according to claim 1, characterized in that: The InfoNCE loss function is expressed as: Where <·,·> represents the cosine similarity; τ is the temperature coefficient that controls the distance between positive and negative samples; k - is the feature vector in the visual dictionary; q and k are respectively the mask image x m With the transformed image x g The feature map f m , f g Scene-level global features of the transformed image and mask image obtained by global pooling.

Images