Rural building detection method based on multi-constraint heterogeneous deep network

Abstract

The invention discloses a rural building detection method based on a multi-constraint heterogeneous deep network, and the method comprises the following steps: S1, carrying out the aerial photographing of a rural region, and obtaining the high-resolution image data of a building; s2, marking a built building and key information such as center point coordinates, width, height and category of the building on the collected image data by using a rectangular frame through a Labelimg tool to construct a remote sensing image data set; s3, the constructed remote sensing image data set is sent to a designed multi-constraint heterogeneous deep network MHDN for training, and a weight file obtained through training is stored to serve as a pre-training weight model in the reasoning process; s4, loading the stored pre-training weight into the model obtained in the step S3, and verifying the detection effect of the model by using the test set; and S5, detecting a target building through the model obtained in the step S4.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, in particular to remote sensing image processing technology, and specifically to a model and method for rural building detection based on a multi-constrained heterogeneous deep network. Background Art

[0002] Rural buildings are the infrastructure for the development of rural areas. The continuous expansion of their number and scale has put forward higher requirements for the management of land resources. In order to prevent the unreasonable use of rural land and the illegal construction of buildings, remote sensing technology has been widely used in the detection and monitoring of rural buildings. However, the existing building detection technology only focuses on whether the target belongs to a building, ignoring the importance of distinguishing between buildings under construction and completed buildings, which plays an important role in regulating rural residential construction and avoiding the loss of arable land resources.

[0003] In order to achieve scientific management and unified planning of rural land, the present invention divides rural buildings into completed buildings and buildings under construction to achieve more fine-grained rural building detection. However, these two types of buildings are highly similar visually, especially in low-resolution remote sensing images, and are difficult to distinguish in appearance. This similarity makes it easy for existing target detection models to misidentify buildings under construction as completed buildings, or vice versa. In the prior art, although the detection method based on convolutional neural network (CNN) performs well in extracting local features, it cannot effectively capture the global correlation between buildings, resulting in the model being unable to fully distinguish between the two types of similar buildings.

[0004] In real-world scenarios, the number of completed buildings far exceeds that of buildings under construction. This imbalance in data poses a huge challenge to detection tasks. During the training process, traditional deep learning models tend to learn the features of a larger number of completed buildings, while ignoring the features of buildings under construction. This class imbalance problem seriously affects the model's ability to detect buildings under construction, which in turn leads to a significant increase in the missed detection rate.

[0005] In addition to buildings, remote sensing images also contain a lot of background information, such as fields, trees, roads, etc. These background information may have certain visual similarities with the features of buildings, especially when the boundary between buildings and background areas is blurred, which can easily lead to false detection problems. Existing convolutional neural networks (CNNs) cannot effectively distinguish between buildings and backgrounds when extracting local features, resulting in a high false detection rate in the detection results.

[0006] Although existing convolutional neural networks (CNNs) have achieved some success in target detection, their limitation is that they cannot effectively capture the dependencies between distant targets. For example, in a remote sensing image containing multiple buildings, there may be some spatial correlation between the buildings, which can help the model better understand the structure of the buildings and their relative positions. However, since convolutional neural networks (CNNs) mainly focus on local feature extraction and lack the integration of global information, the detection effect in complex building scenes is poor.

[0007] In recent years, graph neural networks (GNNs) have been gradually introduced into the field of remote sensing image processing as a technology that can effectively capture global features. By constructing a graph structure of nodes and edges, GNNs can capture the global correlation between different regions in an image. Although some studies have attempted to combine GNNs with convolutional neural networks (CNNs) to improve feature representation capabilities, these methods still face problems such as unclear distinction between under-construction and completed buildings, class imbalance, and background interference when dealing with complex multi-scale target detection tasks. In addition, existing feature extraction methods based on GNNs often have high computational overhead and are not suitable for efficient processing of large-scale remote sensing image data.

[0008] The present invention aims to solve the problems of a large number of false detections and missed detections caused by the similarity and imbalance between the buildings under construction and the completed buildings and the interference of the background.

[0009] Therefore, the applicant proposed a rural building detection method based on a multi-constrained heterogeneous deep network. Summary of the invention

[0010] The purpose of this invention is to solve the technical problems of poor detection effect caused by high similarity of building categories, category imbalance, background interference and lack of global features in the existing remote sensing image building detection technology, and to propose a rural building detection method based on a multi-constrained heterogeneous deep network.

[0011] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0012] A rural building detection method based on a multi-constrained heterogeneous deep network includes the following steps:

[0013] S1. Use drones to take aerial photos of rural areas and obtain high-resolution image data including buildings;

[0014] S2. Use the Labelimg tool to construct a remote sensing image dataset by using rectangular boxes to mark the built buildings and key information such as the center point coordinates, width, height and category of the buildings;

[0015] S3. Send the constructed remote sensing image dataset to the designed multi-constrained heterogeneous deep network MHDN for training, and save the trained weight file as a pre-trained weight model in the reasoning process;

[0016] S4. Load the saved pre-trained weights into the model obtained in step S3, and use the test set to verify the detection effect of the model;

[0017] S5. Detect the target building using the model obtained in step S4.

[0018] In step 3, the multi-constrained heterogeneous deep network MHDN is specifically:

[0019] The input image is input into Stage1 and Stage2, which are composed of three C2F convolution modules, for preliminary feature extraction and dimension upgrading to obtain feature map Feature0.

[0020] The feature map Feature0 is input into Stage3 composed of 6 C2F convolution modules to extract shallow detail information and upgrade the dimension to obtain the feature map Feature1;

[0021] Input feature map Feature1 into the first GAM module to strengthen the global correlation of shallow features and obtain feature map Feature2;

[0022] The feature map Feature2 is input into Stage4 composed of 6 C2F convolution modules to extract the middle-level spatial information and upgrade the dimension to obtain the feature map Feature3;

[0023] Input feature map Feature3 into the second GAM module to strengthen the global correlation of the middle-level features, and obtain feature map Feature4;

[0024] The feature map Feature4 is input into Stage5 composed of three C2F convolution modules to extract deep semantic information, and the feature map Feature5 is obtained;

[0025] Input feature map Feature5 into the third GAM module to strengthen the global correlation of deep features and obtain feature map Feature6;

[0026] After upsampling the feature map Feature6, it is concatenated with the feature map Feature4 by channel to achieve feature fusion, and the feature map Feature7 is obtained;

[0027] After upsampling the feature map Feature7, it is concatenated with the feature map Feature2 by channel to achieve feature fusion, and the feature map Feature_S is obtained for small-scale target detection;

[0028] After downsampling the feature map Feature_S, it is concatenated with the feature map Feature7 by channel to achieve feature fusion, and the feature map Feature_M is obtained for medium-scale object detection;

[0029] After downsampling the feature map Feature_M, it is concatenated with the feature map Feature6 by channel to achieve feature fusion, and the feature map Feature_L is obtained for large-scale object detection.

[0030] The GAM module is specifically:

[0031] Relative position encoding is added to the input feature Feature_in to obtain feature map F1. After the KNN-based graph generator calculates the similarity of each feature point in feature map F1, the position information Pos of the nearest K feature points is obtained. The feature map F1 and the position information Pos are input into the graph attention layer Graph Attention Layer for weighted update of the attention value to obtain feature map F2. The feature map F2 is residually linked with F1 and normalized to obtain feature map F3. The feature map F3 is passed through two fully connected feedforward layers to obtain feature map F4. The feature map F4 is residually linked with F3 and normalized to obtain feature map Feature_out.

[0032] The graph attention layer in the GAM module is specifically:

[0033] Through the obtained nearest position information Pos, each feature point h in the feature map F_in i and its K-1 adjacent nodes h i Input the multidimensional unbiased convolution layer W to map the features into a tensor Tensor1 of 1*K*number of channels, input the tensor Tensor1 into the multidimensional convolution layer a to learn the attention value to obtain the eigenvalue A1, and pass the eigenvalue A1 through the normalization function Softmax to obtain the attention weight vector α of dimension 1*K ij ;

[0034] The attention weight vector α ij With feature point h i and h j Perform weighted addition to obtain the updated feature point h′ i , each feature point in the feature map F_in is updated in this way to obtain the output feature map F_out.

[0035] In step S3, when calculating the loss, any pixel feature vector i∈I in the sample set I is regarded as an anchor point, and features with the same category are regarded as positive samples i +, otherwise it is a negative sample i - ; By introducing the supervised contrast loss function to compare the similarity between positive and negative samples, the positive pairs are pulled closer and the negative pairs are pushed away in the embedding space; the formula is as follows:

[0036]

[0037] in, represents the set of all positive samples except i, is a set of negative samples, and τ>0 is a temperature hyperparameter.

[0038] Compared with the prior art, the present invention has the following technical effects:

[0039] 1) This paper proposes a novel network structure. By introducing a graph attention mechanism into a convolutional neural network, the connection relationship between different nodes in the graph structure can be learned to establish the global feature correlation in the image. This correlation not only emphasizes the representation of the saliency of the target area, but also suppresses the interference from similar but semantically different features. The GAM module is placed after the last three stages of the 5-level CNN backbone network to form a grid-based CNN and GNN hybrid model (GCGH). Through this CNN-GNN hybrid heterogeneous model, the local detail features and global structural relationship of the object can be fully considered, thereby obtaining a more robust feature representation;

[0040] 2) This paper innovatively introduces the idea of ​​contrastive learning into the supervised object detection algorithm, and alleviates the classification bias in imbalanced situations by learning more discriminative representations. Specifically, this paper introduces supervised contrast loss and proposes a cross-image multi-scale contrast constraint loss branch (CMCC) to guide the model to learn more discriminative features. By adding a contrast constraint branch to the traditional supervised learning object detection network, the tilt of the feature distribution learned by the network can be reduced, which is beneficial to detector learning in imbalanced situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0042] Figure 1 is a flow chart of the present invention;

[0043] Figure 2 It is a schematic diagram of the structure of the grid-based CNN and GNN hybrid model (GCGH) in the present invention;

[0044] Figure 3 It is a schematic diagram of the structure of the GAM module in the present invention;

[0045] Figure 4It is a schematic diagram of the structure of the Graph Attention Layer in the present invention;

[0046] Figure 5 It is a structural schematic diagram of the cross-image multi-scale contrast constraint loss branch (CMCC) in the present invention. DETAILED DESCRIPTION

[0047] like Figure 1 As shown, a rural building detection algorithm based on a multi-constrained heterogeneous deep network includes the following steps:

[0048] S1. Use drones to take aerial photos of rural areas and obtain high-resolution image data including buildings;

[0049] S2. Use the Labelimg tool to construct a remote sensing image dataset by using rectangular boxes to mark the built buildings and key information such as the center point coordinates, width, height and category of the buildings;

[0050] S3. Send the constructed remote sensing image dataset to the designed multi-constrained heterogeneous deep network MHDN for training, and save the trained weight file as a pre-trained weight model in the reasoning process;

[0051] S4. Load the saved pre-trained weights into the model obtained in step S3, and use the test set to verify the detection effect of the model;

[0052] S5. Detect the target building by the model obtained in step S4;

[0053] In step S3, the construction of the multi-constrained heterogeneous deep network MHDN adopts the following sub-steps:

[0054] In the present invention, a multi-constrained heterogeneous deep network MHDN is constructed with YOLOv8 as the baseline model to construct a grid-based CNN and GNN hybrid model (GCGH) for extracting more robust feature representations.

[0055] like Figure 2 As shown, the grid-based CNN and GNN hybrid model (GCGH) is specifically:

[0056] The input image is input into Stage1 and Stage2, which are composed of three C2F convolution modules, for preliminary feature extraction and dimension upgrading to obtain feature map Feature0.

[0057] The feature map Feature0 is input into Stage3 composed of 6 C2F convolution modules to extract shallow detail information and upgrade the dimension to obtain the feature map Feature1;

[0058] Input feature map Feature1 into the first GAM module to strengthen the global correlation of shallow features and obtain feature map Feature2;

[0059] The feature map Feature2 is input into Stage4 composed of 6 C2F convolution modules to extract the middle-level spatial information and upgrade the dimension to obtain the feature map Feature3;

[0060] Input feature map Feature3 into the second GAM module to strengthen the global correlation of the middle-level features, and obtain feature map Feature4;

[0061] The feature map Feature4 is input into Stage5 composed of three C2F convolution modules to extract deep semantic information, and the feature map Feature5 is obtained;

[0062] Input feature map Feature5 into the third GAM module to strengthen the global correlation of deep features and obtain feature map Feature6;

[0063] After upsampling feature map Feature6 by 2 times, it is concatenated with feature map Feature4 by channel to achieve feature fusion, and feature map Feature7 is obtained.

[0064] The feature map Feature7 is upsampled by 2 times and then concatenated with the feature map Feature2 by channel to achieve feature fusion, and the feature map Feature_S with a size of 1 / 8 of the input image is obtained for small-scale target detection.

[0065] The feature map Feature_S is downsampled by 2 times and then concatenated with the feature map Feature7 by channel to achieve feature fusion, and the feature map Feature_M with a size of 1 / 16 of the input image is obtained for medium-scale object detection.

[0066] The feature map Feature_M is downsampled by 2 times and then concatenated with the feature map Feature6 by channel to achieve feature fusion, and the feature map Feature_L with a size of 1 / 32 of the input image is obtained for large-scale object detection.

[0067] like Figure 3 As shown, the GAM module is specifically:

[0068] Relative position encoding is added to the input feature Feature_in to obtain feature map F1. After the KNN-based graph generator calculates the similarity of each feature point in feature map F1, the position information Pos of the nearest K feature points is obtained. The feature map F1 and the position information Pos are input into the graph attention layer Graph Attention Layer for weighted update of the attention value to obtain feature map F2. The feature map F2 is residually linked with F1 and normalized to obtain feature map F3. The feature map F3 is passed through two fully connected feedforward layers to obtain feature map F4. The feature map F4 is residually linked with F3 and normalized to obtain feature map Feature_out.

[0069] The KNN graph generator treats each grid of the input image as a graph node and establishes connections based on the Euclidean distance between nodes. These connections are used to create a sparse graph structure that captures the global associations between different regions in the image. On this basis, the Graph Attention Layer calculates the attention weights between nodes, updates the node features, strengthens the saliency of the target area, and suppresses background interference.

[0070] like Figure 4 As shown, the graph attention layer in the GAM module is specifically:

[0071] Through the obtained nearest position information Pos, each feature point h in the feature map F_in i and its K-1 adjacent nodes h i Input the 2D unbiased convolution layer W to map the features into a tensor Tensor1 of 1*K*number of channels, input the tensor Tensor1 into the 2D convolution layer a to learn the attention value to obtain the eigenvalue A1, and pass the eigenvalue A1 through the normalization function Softmax to obtain the attention weight vector α of dimension 1*K ij , attention weight α ij The formula is as follows:

[0072]

[0073] The attention weight vector α ij With feature point h i and h j Perform weighted addition to obtain the updated feature point h′ i , each feature point in the feature map F_in is updated in this way to obtain the output feature map F_out, where the update process of a feature point can be expressed as:

[0074]

[0075] The cross-image multi-scale contrast constraint loss branch (CMCC) is added to the YOLOv8 network, aiming to simplify the learning pressure of the detector in the case of class imbalance by learning more discriminative features.

[0076] like Figure 5 As shown in Figure 2, the cross-image multi-scale contrast constraint loss branch (CMCC) is specifically:

[0077] For the multi-scale feature maps Feature_S, Feature_M and Feature_L output by the grid-based CNN and GNN hybrid model (GCGH), each feature point of each feature map is regarded as a basic unit, and the foreground samples are screened out through the task alignment strategy in the baseline model YOLOv8, realizing the interaction between tasks, while considering the category information and location information. The feature vectors of these foreground samples in the projected features are used as the sample set I for comparison, and the category information is added to the samples by introducing labels to achieve multi-scale sampling across images within the mini-batch.

[0078] When calculating the loss, this paper considers any pixel feature vector i∈I in the sample set I as an anchor point, and considers features of the same category as positive samples i + , otherwise it is a negative sample i - By introducing the supervised contrast loss function to compare the similarity between positive and negative samples, the positive pairs are pulled closer and the negative pairs are pushed away in the embedding space. The formula is as follows:

[0079]

[0080] in, represents the set of all positive samples except i, is the set of negative samples, and τ>0 is the temperature hyperparameter. The push-out operation of negative pairs prompts the network to distinguish samples with similar features from different categories in the embedding space. At the same time, the pull-in operation of positive pairs can also enable the model to learn the common features of targets with large differences within the class. This method only participates in the training phase, and promotes the discriminative representation learning of the model by comparing the whole and parts between objects, which can reduce the learning bias of the detector. The final detection loss is defined as follows:

[0081]

[0082] In the present invention, the proposed grid-based CNN and GNN hybrid model (GCGH) captures local context information and global dependency information from the perspectives of CNN and GNN respectively to enhance the representation ability of the model, thereby obtaining a more robust feature representation. On this basis, the cross-image multi-scale contrast constraint loss branch (CMCC) supervises the grid-based CNN and GNN hybrid model (GCGH) for representation learning by introducing the displayed feature contrast constraint and detection loss together, and the main purpose is to achieve better detection effect through better feature representation. By seamlessly integrating the grid-based CNN and GNN hybrid model (GCGH) and the cross-image multi-scale contrast constraint loss branch (CMCC) into the YOLOv8 network, the present invention constructs a multi-constrained heterogeneous deep network (MHDN), which can obtain more discriminative feature representations, thereby achieving more accurate rural building detection, and providing strong technical support for rural construction, building monitoring and other fields.

[0083] Example:

[0084] 1) Parameter settings

[0085] In this paper, the present invention uses YOLOv8s as a baseline to build the proposed multi-constrained heterogeneous deep network (MHDN). The multi-constrained heterogeneous deep network (MHDN) was trained for 300 epochs using Pytorch on a single NVIDIA 3090 GPU using the stochastic gradient descent (SGD) algorithm with an initial learning rate of 0.01, a momentum of 0.937, and a weight decay of 0.0005. The proposed method was subjected to an ablation study using the same settings to verify its effectiveness.

[0086] 2) Experimental indicators

[0087] In the present invention, the average precision of each category and the average precision calculated at a cross-over-union threshold (mAP50) of 0.5 were used to evaluate the performance of different methods.

[0088] 3) Comparative test

[0089] In order to verify the performance of multi-constrained heterogeneous deep networks (MHDN), this paper adopts nine mainstream methods as comparison objects, including four CNN-based methods, namely YOLOv8-L, Faster-RCNN, RTMDet, and YOLOv8 (with LSKNet-T as the backbone); four Transformer-based methods, namely DINO, RT-DETR, Deformable-DETR, and YOLOv8 (with Swin-T as the backbone), and one GNN-based method, namely YOLOv8 (with Pyramid VIG-S as the backbone).

[0090] Table 1 Quantitative evaluation of current mainstream algorithms on the dataset

[0091]

[0092] As shown in Table 1, the proposed multi-constrained heterogeneous deep network (MHDN) achieves higher mAP50 detection results at a lower computational cost, which is 3.5% higher than the second-best method (RTMDet) in the building under construction category and 1.8% higher in the overall mAP50. These results show that the multi-constrained heterogeneous deep network (MHDN) is able to improve the accuracy of building detection under normal circumstances.

[0093] 4) Ablation experiment

[0094] In order to verify the effectiveness of the proposed grid-based CNN and GNN hybrid model (GCGH) and the cross-image multi-scale contrast constrained loss branch (CMCC), the present invention conducted an ablation experiment to compare the detection performance of the model and the number of model parameters.

[0095] Table 2 Ablation results of each part in Multi-Constrained Heterogeneous Deep Network (MHDN)

[0096]

[0097] The proposed multi-constrained heterogeneous deep network (MHDN) takes YOLOv8s as the baseline, combining YOLOv8 with a grid-based CNN and GNN hybrid model (GCGH) and a cross-image multi-scale contrastive constrained loss branch (CMCC), namely Base+GCGH+CMCC. As shown in Table 2, the present invention compares the effects of adding the GCGH model and the CMCC branch to the baseline, namely Base+GCHF and Base+CMCC. The experimental results show that the performance of MHDN is better than Base+GCGH and Base+CMCC, proving the complementarity of the two methods. In addition, with a slight increase in GFLOPs and parameters, the proposed multi-constrained heterogeneous deep network (MHDN) improves the overall mAP50 by 8.1% compared with the baseline model, and significantly improves by 14.8% in the under-construction building (UB) category, proving the effectiveness of the multi-constrained heterogeneous deep network (MHDN).

[0098] The present invention provides a construction method of a rural building detection algorithm, which is composed of a grid-based CNN and GNN hybrid model (GCGH) and a cross-image multi-scale contrast constraint loss branch (CMCC) seamlessly added to the YOLOv8 network to form a multi-constrained heterogeneous deep network (MHDN), which can improve the impact of complex architectural styles and construction stages and category imbalance on the construction status detection of rural buildings. The experimental results given in this paper show that through the representation learning of rich local and global information and less category deviation, the proposed multi-constrained heterogeneous deep network (MHDN) is significantly superior to other methods in detection performance and computational cost, and achieves more accurate detection of rural under-construction and completed buildings in remote sensing images, proving the effectiveness of the multi-constrained heterogeneous deep network (MHDN).

Claims

1. A rural building detection method based on multi-constrained heterogeneous deep network, characterized in that: The following steps are involved: S1. Use drones to take aerial photos of rural areas and obtain high-resolution image data including buildings; S2. Use the Labelimg tool to use rectangular boxes to mark the built buildings and the key information such as the center point coordinates, width, height and category of the buildings to construct a remote sensing image dataset; S3. Send the constructed remote sensing image dataset to the designed multi-constrained heterogeneous deep network MHDN for training, and save the trained weight file as a pre-trained weight model in the reasoning process; S4. Load the saved pre-trained weights into the model obtained in step S3, and use the test set to verify the detection effect of the model; S5. Detect the target building using the model obtained in step S4.

2. The method according to claim 1, characterized in that In step 3, the multi-constrained heterogeneous deep network MHDN is specifically: The input image is input into Stage1 and Stage2, which are composed of three C2F convolution modules, for preliminary feature extraction and dimension upgrading to obtain feature map Feature0. The feature map Feature0 is input into Stage3 composed of 6 C2F convolution modules to extract shallow detail information and upgrade the dimension to obtain the feature map Feature1; Input feature map Feature1 into the first GAM module to strengthen the global correlation of shallow features and obtain feature map Feature2; The feature map Feature2 is input into Stage4 composed of 6 C2F convolution modules to extract the middle-level spatial information and upgrade the dimension to obtain the feature map Feature3; Input feature map Feature3 into the second GAM module to strengthen the global correlation of the middle-level features, and obtain feature map Feature4; The feature map Feature4 is input into Stage5 composed of three C2F convolution modules to extract deep semantic information, and the feature map Feature5 is obtained; Input feature map Feature5 into the third GAM module to strengthen the global correlation of deep features and obtain feature map Feature6; After upsampling the feature map Feature6, it is concatenated with the feature map Feature4 by channel to achieve feature fusion, and the feature map Feature7 is obtained; After upsampling the feature map Feature7, it is concatenated with the feature map Feature2 by channel to achieve feature fusion, and the feature map Feature_S is obtained for small-scale target detection; After downsampling the feature map Feature_S, it is concatenated with the feature map Feature7 by channel to achieve feature fusion, and the feature map Feature_M is obtained for medium-scale object detection; After downsampling the feature map Feature_M, it is concatenated with the feature map Feature6 by channel to achieve feature fusion, and the feature map Feature_L is obtained for large-scale object detection.

3. The method according to claim 2, characterized in that The GAM module is specifically: Relative position encoding is added to the input feature Feature_in to obtain feature map F1. After the KNN-based graph generator calculates the similarity of each feature point in feature map F1, the position information Pos of the nearest K feature points is obtained. The feature map F1 and the position information Pos are input into the graph attention layer Graph Attention Layer for weighted update of the attention value to obtain feature map F2. The feature map F2 is residually linked with F1 and normalized to obtain feature map F3. The feature map F3 is passed through two fully connected feedforward layers to obtain feature map F4. The feature map F4 is residually linked with F3 and normalized to obtain feature map Feature_out.

4. The method according to claim 3, characterized in that The graph attention layer GraphAttention Layer in the GAM module is specifically: Through the obtained nearest position information Pos, each feature point h in the feature map F_in i and its K-1 adjacent nodes h i Input the multidimensional unbiased convolution layer W to map the features into a tensor Tensor1 of 1*K*number of channels, input the tensor Tensor1 into the multidimensional convolution layer a to learn the attention value to obtain the eigenvalue A1, and pass the eigenvalue A1 through the normalization function Softmax to obtain the attention weight vector α of dimension 1*K ij ; The attention weight vector α ij With feature point h i and h j Perform weighted addition to obtain the updated feature point h′ i , each feature point in the feature map F_in is updated in this way to obtain the output feature map F_out.

5. The method according to claim 1, characterized in that In step S3, when calculating the loss, any pixel feature vector i∈I in the sample set I is regarded as an anchor point, and features with the same category are regarded as positive samples i + , otherwise it is a negative sample i - ; By introducing the supervised contrast loss function to compare the similarity between positive and negative samples, the positive pairs are pulled closer and the negative pairs are pushed away in the embedding space; the formula is as follows: in, represents the set of all positive samples except i, is a set of negative samples, and τ>0 is a temperature hyperparameter.

Images