A convolutional neural network based on neighborhood and grid attention mechanism

Abstract

The present invention relates to the field of computer technology, and in particular to a convolutional neural network based on a neighborhood and grid attention mechanism, wherein the convolutional neural network divides the subsequent part of the res_conv4_1 residual block into three branches: a global branch, a local branch one, and a local branch two; the global branch embeds a grid relationship guidance GRG module, disperses the attention at the target pixel to multiple grids, and prompts the pixels in the grid to interact to learn the global dependency between pixels; the local branch two embeds a neighborhood relationship guidance NRG module, interacts the target pixel with the nearest neighbor pixel to capture the pairwise dependency between the nearest neighbor pixels, and captures the correlation information between the nearest neighbor pixels. The present invention can fully learn the local detail information of the input feature map and obtain richer and more diverse feature representations.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a convolutional neural network based on neighborhood and grid attention mechanisms. Background Art

[0002] Convolutional neural networks have played a core role in promoting the innovation of computer vision technology, exploring cutting-edge fields such as instance segmentation, scene understanding, and feature learning. Among them, the self-attention mechanism has played a huge potential in computer vision. Existing attention methods mainly include global self-attention and local self-attention. The computational complexity of global self-attention is quadratic, and the long-distance connections in the global context captured by global self-attention are weak, which is not conducive to learning important overall information of the input feature map. Although local self-attention has reduced computational complexity, this method does not refine to the nearest neighbor of the pixel and cannot fully learn the local detail information of the feature map. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a convolutional neural network based on neighborhood and grid attention mechanism.

[0004] The technical solution adopted by the present invention to solve the technical problem is:

[0005] A convolutional neural network based on neighborhood and grid attention mechanism, wherein the convolutional neural network uses the residual block before res_conv4_2 of ResNet-50 as the backbone network, and divides the subsequent part of the res_conv4_1 residual block into three branches: a global branch, a local branch one, and a local branch two, wherein the global branch is used to extract discriminative global features of the input feature map, such as color features, shape features, etc.;

[0006] The global branch embedding grid relationship guides the GRG module, which divides the feature map into multiple grids, generates pairwise dependencies through the interaction between grids, generates attention weights using compression and normalization, and weights the attention weights to all pixels in the feature map through a dot product operation to obtain the output feature map of the GRG module, thereby reducing the computational cost of the module and strengthening those important remote connections that are ignored or weakly connected;

[0007] The local branch two embedding neighborhood relationship guides the NRG module, which locates the attention range of each pixel to its nearest neighborhood, and interacts the target pixel with its nearest neighbor pixels to generate a pairwise dependency relationship, and then generates an attention map through dimensionality reduction and normalization. Finally, the attention map is dot-producted with the original feature map to complete the weighted operation and obtain the output feature map of NRG, thereby reducing the computational complexity and capturing the discriminative local context.

[0008] Furthermore, in the NRG module, the specific process of generating paired dependencies is as follows:

[0009] Given a feature map X∈R C×H×W As the input of the NRG module, C, H, and W represent the number of channels, height, and width of the feature map, respectively; the query feature map Q∈R is obtained by performing three 1×1 convolutions on X. C×H×W , key feature map K∈R C ×H×W Sum value feature graph V∈R C×H×W :

[0010] ( Q , K , V ) = Conv2d(X);

[0011] Define the query feature graph Q l The query for pixels is the position in X l The eigenvector at q l ∈R l×C , l ={1,2, ...,HW}; in the key feature graph K, the distance l The nearest pixel c 2 The pixels are recorded as the neighborhood of the pixel m ∈ c 2 , taking the feature vector of each pixel in the neighborhood as the first l The key set of target pixels k l ∈R (γ×γ)×C ;calculate q l and k T l The matrix product of l The pairwise affinity of a pixel with its neighborhood pixels, thus generating a relationship vector A i ∈R 1×(γ×γ) , i ={1, 2, ...,HW}:

[0012] ;

[0013] The relationship vector z The element represents the l Pixels and m Neighborhood z Attention scores for pixels;

[0014] In order to calculate the pairwise dependencies between each pixel and the pixels in its neighborhood, the Unfold operation is used to generate a dynamic shift window around each pixel and slide it on the key feature map K to extract the nearest neighbors of each pixel. The pairwise dependencies between all target pixels and their nearest neighbor pixels are generated through matrix multiplication operations.

[0015] Furthermore, the queries of all pixels in the query feature map Q are stacked together to generate a query tensor Q ’ ∈R HW ×1×C , by applying a kernel size of c An Unfold operation with a step size of 1 and a Transform operation generates a HW×C× c 2 The key tensor K ’ , so that the key sets of HW target pixels are stacked together; the query tensor Q ’ and the key tensor K ’ Perform matrix multiplication to obtain the relational tensor A ∈R HW×1×(γ×γ) :

[0016] ;

[0017] Among them, the vector 1×( c × c ) indicates that a pixel and its m ∈ c 2 The tensor A reflects the pairwise affinity between each pixel and its nearest neighbor.

[0018] Furthermore, in the NRG module, the specific process of generating the attention map is as follows:

[0019] Based on the pairwise dependency between the target pixel and its nearest neighbor pixel, the tensor A is transformed to obtain the matrix A ’ ∈R HW×(γ×γ) ;right A ’ The last dimension of is compressed to calculate the average features of the pixels in the neighborhood, and the vectorW ∈R HW×1 ; Each value in W is obtained by calculating the mean of the pairwise dependencies between each pixel and the pixels in its neighborhood:

[0020] W = Mean(Reshape( A ));

[0021] After transforming the dimension of W, a 1D convolution operation is applied, and the attention weight of each pixel is generated through a Softmax normalization operation to obtain the weight vector W ’ ∈R 1×HW :

[0022] W ’ = Softmax(1D( W ));

[0023] 1D convolution captures local patterns at different positions in the relationship vector in a local interactive manner. Compared with full connection, one-dimensional convolution has smaller parameters and faster calculation speed. In order to reconstruct the representation of each pixel and capture the refined local context, W ’ Reshape to get the attention map Y ∈R 1×H×W This process efficiently aggregates the rich details in the nearest neighbor information through weighted operations.

[0024] Furthermore, the attention map Y is element-wise multiplied with the value feature map V to generate a feature map F ∈R C×H×W :

[0025] F = Y☉V;

[0026] The feature map F Add it to the input feature map X, apply batch normalization (BN) and GELU activation function to the added feature map in turn, and get the output feature map of NRG F ’ :

[0027] F ’ = GELU(BN(F + X)).

[0028] Furthermore, in the GRG module, the specific process of generating paired dependencies is as follows:

[0029] The output X∈R of the last stage of ResNet-50 C×H×WAs GRG input, C, H, W represent the number of channels, height, and width of the feature map respectively; perform three 1×1 convolutions on X to obtain three feature maps: query feature map Q∈R C×H×W , key feature map K∈R C×H×W , value feature map V∈R C×H×W :

[0030] ( Q , K , V ) = Conv2d(X);

[0031] In order to distribute the attention scores at the target pixels to different grids, a uniform grid of size sh×sw is used along the spatial dimension to query the feature map. Q Divide and transform the divided tensor to obtain the query tensor Q ’ ∈R sh ×sw×(HW / (sh*sw))×C , where sh and sw represent the height and width of each grid, sh*sw represents the number of pixels in a grid, and HW / (sh*sw) represents the number of grids obtained after grid division. Similarly, a uniform grid of size sh×sw is used to map the key feature map. K Perform grid division and apply a deformation operation to the divided tensor to obtain the key tensor K ’ ∈R sh×sw×(HW / (sh*sw))×C :

[0032] Q ’ = Reshape(grid_partition( Q ));

[0033] K ’ = Reshape(grid_partition( K ));

[0034] To compute pairwise dependencies between pixels, the query tensor Q ’ and key tensors K ’ Perform matrix multiplication to obtain the relational matrix A ∈R sh×sw×(HW / (sh*sw))×(HW / (sh*sw)) :

[0035] ;

[0036] In the matrix A In the , each row represents the i The pairwise dependencies between pixels in a grid and pixels in other grids.

[0037] Furthermore, in the GRG module, the specific process of generating attention weights is as follows:

[0038] Based on the relationship matrix A , transform it to generate a tensor A ’ ∈R HW×(HW / (sh*sw)) ,right A ’ The last dimension of P ∈R HW×1 , which captures robustness information while reducing dimensionality by aggregating the above pairwise dependencies; then, P Transform to get vector W ∈R 1×HW :

[0039] W = Reshape(Mean( A ’ )).

[0040] Next, W Implement a 1D convolution operation and perform a Softmax normalization operation on the last dimension of the resulting tensor to output the attention weight of each pixel, thereby obtaining a weight vector W ’ ∈R 1×HW :

[0041] W ’ = Softmax(1D( W )).

[0042] Furthermore, the weight vector W ’ Deformation to obtain attention map G ∈R 1×H×W , then, G Characteristic graph with value V Perform element-by-element multiplication to complete the weighting operation on the original features, thereby highlighting the most significant dependencies in the feature graph:

[0043] G ’ = V☉Reshape( W ’ );

[0044] This process generates a feature map G ’ The size of the space is C×H×W;

[0045] Finally, the feature map G’ Add it to the input feature map X and perform BN and GELU activation functions on the added feature map to finally generate the output feature map F of GRG:

[0046] F = GELU(BN( G ’ + X)).

[0047] Furthermore, the output feature maps of the three branches are globally average pooled to generate a 2048-dimensional feature vector, and then the dimension is reduced to 256 dimensions. The reduced 256-dimensional feature vector is used to calculate the triplet loss in the training phase, and the 256-dimensional feature is input into the fully connected layer to calculate the cross entropy loss.

[0048] Furthermore, the reduced 256-dimensional feature vectors of the three branches are concatenated together as the final feature representation in the testing phase.

[0049] Technical effects of the present invention:

[0050] Compared with the prior art, the present invention proposes two complementary attention modules: a GRG module and an NRG module. The NRG module interacts the target pixel with the nearest neighbor pixel to capture the pairwise dependencies between the nearest neighbor pixels, and generates a weighted attention map by averaging and one-dimensional convolution, thereby capturing the correlation information between the nearest neighbor pixels. The GRG module disperses the attention at the target pixel to multiple grids, and prompts the pixels within the grid to interact to learn the global dependencies between pixels, thereby learning enhanced global context information. Based on the above two modules, the present invention constructs a convolutional neural network with neighborhood and grid attention mechanisms to learn the discriminative local and global features of the input feature map, which can fully learn the local detail information of the feature map and obtain richer and more diverse feature representations. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A convolutional neural network based on neighborhood and grid attention mechanism of the present invention;

[0052] Figure 2 This is a structural diagram of the NRG module of the present invention;

[0053] Figure 3 This is a structural diagram of the GRG module of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention is clearly and completely described below in conjunction with the accompanying drawings.

[0055] Embodiment 1:

[0056] like Figure 1 As shown, the present embodiment involves a convolutional neural network based on a neighborhood and grid attention mechanism, wherein the convolutional neural network includes a backbone network and three branches. Specifically, the residual block before res_conv4_2 of ResNet-50 is used as the backbone network, and the subsequent part of the res_conv4_1 residual block is divided into three branches: a global branch, a local branch one, and a local branch two; the features of the three branches are integrated into a unified framework to achieve end-to-end training, thereby obtaining a more complete, richer, and more diverse feature representation;

[0057] The global branch is used to extract the discriminative global features of the entire input feature map, such as color features, shape features, etc.;

[0058] The global branch is embedded in a grid relationship guidance GRG (Grid Relationship Guidance) module to learn robust global features of the feature map, so that the network can enhance its attention to important features;

[0059] The local branch 2 embeds a neighborhood relationship guidance NRG (Neighborhood Relationship Guidance) module, which is intended to discriminate and identify the features of the detail parts of the feature map. By adding the NRG module to this branch, more discriminative local information of the feature map can be extracted;

[0060] The input feature map is processed by the feature extraction backbone to generate a 2048-dimensional feature map of size 16×16, and then applied to the three branches for feature learning. The feature map of each branch is first processed by the global average pooling layer to generate a 2048-dimensional feature vector, and then it is reduced to 256 dimensions by a 1×1 convolution layer, a batch normalization (BN) layer, and a rectified linear unit (ReLU).

[0061] During the training phase, the reduced 256-dimensional feature vector is used to calculate the triplet loss, and the 256-dimensional feature is input into the fully connected layer to calculate the cross entropy loss; during the testing phase, the 256-dimensional feature vectors output by all branches are connected together as the final feature representation.

[0062] 1. NRG module

[0063] This embodiment locates the attention range of each pixel to its nearest neighbor through the NRG module, and generates a pairwise dependency relationship by interacting the target pixel with its nearest neighbor pixels, and then generates an attention map through dimensionality reduction and normalization. Finally, the attention map is dot-producted with the original feature map to complete the weighted operation and obtain the output feature map of NRG, thereby reducing the computational complexity and efficiently capturing the discriminative local information of the feature map.

[0064] Specifically, in the NRG module, given the feature map X∈R C×H×W As the input of the NRG module, C, H, and W represent the number of channels, height, and width of the feature map, respectively; the query feature map Q∈R is obtained by performing three 1×1 convolutions on X. C×H×W , key feature map K∈R C×H×W Sum value feature graph V∈R C×H×W :

[0065] ( Q , K , V ) = Conv2d(X);

[0066] Define the query feature graph Q l The query for pixels is the position in X l The eigenvector at q l ∈R l×C , l ={1,2, ...,HW}; in the key feature graph K, the distance l The nearest pixel c 2 The pixels are recorded as the neighborhood of the pixel m ∈ c 2 , taking the feature vector of each pixel in the neighborhood as the first l The key set of target pixels k l ∈R (γ×γ)×C ;like Figure 2 As shown, in this embodiment, c =3. Calculation q l and k T l The matrix product of l The pairwise affinity of a pixel with its neighborhood pixels, thus generating a relationship vector A i ∈R 1×(γ×γ) , i ={1, 2, ...,HW};

[0067] ;

[0068] The relationship vector z The element represents the l Pixels and m Neighborhood z Attention scores for pixels;

[0069] In order to calculate the pairwise dependency between each pixel and its neighboring pixels, an Unfold operation is used to generate a dynamic shift window around each pixel and slide it on the key feature map K to extract the nearest neighbor of each pixel. Then, a matrix multiplication operation is performed to generate the pairwise dependency between all target pixels and their nearest neighbor pixels. Specifically, the query feature map Q is transformed to generate the query tensor Q ’ ∈R HW×1×C , so that the queries of all pixels in Q are stacked together. By applying a kernel size of c An Unfold operation with a step size of 1 and a Transform operation to generate a HW×C× c 2 The key tensor K ’ , so that the key sets of HW target pixels are stacked together; the query tensor Q ’ and the key tensor K ’ Perform matrix multiplication to obtain the relational tensor A ∈R HW×1×(γ×γ) :

[0070] ;

[0071] Among them, the vector 1×( c × c ) indicates that a pixel and its m ∈ c 2 The tensor A reflects the pairwise affinity between each pixel and its nearest neighbor.

[0072] Based on the pairwise dependency between the target pixel and its nearest neighbor pixel, the tensor A is transformed to obtain the matrix A ’ ∈R HW×(γ×γ) Next, A ’ The last dimension of is compressed to calculate the average features of the pixels in the neighborhood, and the vector W ∈R HW×1 ; Each value in W is obtained by calculating the mean of the pairwise dependencies between each pixel and the pixels in its neighborhood:

[0073] W = Mean(Reshape( A ));

[0074] After transforming the dimension of W, a 1D convolution operation is applied, and the attention weight of each pixel is generated through a Softmax normalization operation to obtain the weight vector W ’ ∈R 1×HW :

[0075] W ’ = Softmax(1D( W ));

[0076] 1D convolution captures local patterns at different positions in the relationship vector in a local interactive manner. Compared with full connection, one-dimensional convolution has smaller parameters and faster calculation speed.

[0077] In order to reconstruct the representation of each pixel and capture the refined local context, W ’ Reshape to get the attention map Y ∈R 1×H×W , and multiply Y by the value feature map V element by element to generate the feature map F ∈R C×H×W :

[0078] F = Y☉V;

[0079] This process efficiently aggregates the rich detail information in the nearest neighbor information through weighted operations.

[0080] Finally, perform the feature map F Add it to the input feature map X. Then, batch normalization (BN) and GELU activation function are applied to the added feature map to obtain the output feature map of NRG F ’ :

[0081] F ’ = GELU(BN(F + X));

[0082] The above process aims to fuse the input features and the representations learned inside the module to produce a richer output feature map.

[0083] The NRG module learns local features with subtle differences. In NRG, the feature map is divided into multiple grids, and the relationship between pixels in each sub-grid is considered by calculating the attention score of the pixels in the grid. Since it contains valid information of other sub-grids in addition to its own information, NRG can effectively strengthen those meaningful but weak connections and reduce computational complexity.

[0084] 2. GRG module

[0085] This embodiment divides the feature map into multiple grids through the GRG module, then generates pairwise dependencies through the interaction between grids, uses compression and normalization to generate attention weights, and weights the attention weights to all pixels in the feature map through a dot multiplication operation to obtain the output feature map of the GRG module, thereby reducing the computational cost of the module and strengthening important remote connections that are ignored or weakly connected.

[0086] Specifically, in the GRG module, the output X∈R C×H×W As GRG input, C, H, W represent the number of channels, height, and width of the feature map respectively; perform three 1×1 convolutions on X to obtain three feature maps: query feature map Q∈R C×H×W , key feature map K∈R C×H×W , value feature map V∈R C×H×W :

[0087] ( Q , K , V ) = Conv2d(X);

[0088] In order to distribute the attention scores at the target pixels to different grids, a uniform grid of size sh×sw is used along the spatial dimension to query the feature map. Q Divide, such as Figure 3 As shown, in this embodiment, sh=sw=2. Next, the divided tensor is transformed to obtain the query tensor Q ’ ∈R sh×sw×(HW / (sh*sw))×C , where sh and sw represent the height and width of each grid, sh*sw represents the number of pixels in a grid, and HW / (sh*sw) represents the number of grids obtained after grid division. Similarly, a uniform grid of size sh×sw is used to map the key feature map. K Perform mesh division, and then apply a deformation operation to the divided tensor to obtain the key tensor K ’ ∈R sh×sw×(HW / (sh*sw))×C :

[0089] Q ’ = Reshape(grid_partition( Q ));

[0090] K ’ = Reshape(grid_partition( K ));

[0091] To compute pairwise dependencies between pixels, the query tensor Q ’ and key tensors K ’ Perform matrix multiplication to obtain the relational matrix A ∈R sh×sw×(HW / (sh*sw))×(HW / (sh*sw)) :

[0092] ;

[0093] In the matrix A In the , each row represents the i The pairwise dependencies between pixels in a grid and pixels in other grids.

[0094] Based on the relationship matrix A , transform it to generate a tensor A ’ ∈R HW×(HW / (sh*sw)) Next, A ’ The last dimension of P ∈R HW×1 , which captures robustness information while reducing dimensionality by aggregating the above pairwise dependencies; then, P Transform to get vector W ∈R 1×HW :

[0095] W = Reshape(Mean( A ’ )).

[0096] Next, W Implement a 1D convolution operation and perform a Softmax normalization operation on the last dimension of the resulting tensor to output the attention weight of each pixel, thereby obtaining a weight vector W ’ ∈R 1×HW :

[0097] W ’ = Softmax(1D( W )).

[0098] Will W ’ Deformation to obtain attention map G ∈R 1×H×W , then, G Characteristic graph with value V Perform element-by-element multiplication to complete the weighting operation on the original features, thereby highlighting the most significant dependencies in the feature graph:

[0099] G ’ = V☉Reshape( W ’ );

[0100] This process generates a feature map G ’ The size of the space is C×H×W.

[0101] Finally, the feature map G ’ Add it to the input feature map X and perform BN and GELU activation functions on the added feature map to finally generate the output feature map F of GRG:

[0102] F = GELU(BN( G ’ + X)).

[0103] The GRG module learns local features of feature map refinement. It locates the attention receptive field of each query (target pixel) in its nearest neighbor pixel. The key pair is selected from the neighborhood centered on q. The attention weight is obtained by performing self-attention calculation on it, and the weight is weighted to the original feature map to reconstruct the pixel, thereby capturing rich detail information and reducing the amount of calculation, overcoming the disadvantage of insufficient learning of detail information by global self-attention.

[0104] 3. Loss Function

[0105] The present invention chooses to use cross entropy loss and triplet loss in the training stage. Cross entropy loss is mainly used for input feature map ID prediction. The role of triplet loss is to make the distance between features of the same feature map category smaller during network training.

[0106] The present invention proposes an NRG module and a GRG module to reduce the amount of computation, while extracting more discriminative local features and more robust global features. The two modules complement each other and jointly improve the learning effect of the model. Specifically, the NRG module reduces the computational complexity of attention learning and captures rich detail information by interacting with the target pixel and its nearest neighbor pixels and compressing pairwise dependencies. The GRG module strengthens the dependencies between pixels that are far away and meaningful from each other and reduces the computational cost through interactions and dimensionality reduction operations between grids. Finally, a convolutional neural network with a neighborhood and grid attention mechanism for extracting discriminative local features and robust global appearance features of the input feature map is proposed in combination with the NRG module and the GRG module, and a richer and more diverse feature representation is obtained through end-to-end training.

Claims

1. A convolutional neural network based on neighborhood and grid attention mechanism, characterized by: The convolutional neural network uses the residual block before res_conv4_2 of ResNet-50 as the backbone network, and divides the subsequent part of the res_conv4_1 residual block into three branches: a global branch, a local branch one, and a local branch two, wherein the local branch one is used to extract discriminative global features in the input feature map; The global branch embedding grid relationship guides the GRG module, which divides the feature map into multiple grids, then generates pairwise dependencies through the interaction between grids, generates attention weights using compression and normalization, and weights the attention weights to all pixels in the feature map through a dot product operation to obtain the output feature map of the GRG module; The local branch embeds the neighborhood relationship to guide the NRG module. The NRG module locates the attention range of each pixel to its nearest neighborhood, and interacts the target pixel with its nearest neighbor pixels to generate a pairwise dependency relationship. Then, an attention map is generated by dimensionality reduction and normalization. Finally, the attention map is dot-producted with the original feature map to complete the weighted operation, and the output feature map of NRG is obtained. In the NRG module, the specific process of generating paired dependencies is as follows: Given a feature map X∈R C×H×W As the input of the NRG module, C, H, and W represent the number of channels, height, and width of the feature map, respectively; the query feature map Q∈R is obtained by performing three 1×1 convolutions on X. C×H×W , key feature map K∈R C×H×W Sum value feature graph V∈R C×H×W : ( Q , K , V ) = Conv2d(X); Define the query feature graph Q l The query for pixels is the position in X l The eigenvector at q l ∈R l×C , l ={1, 2,...,HW}; in the key feature graph K, the distance l The nearest pixel γ 2 The pixels are recorded as the neighborhood of the pixel μ ∈ γ 2 , taking the feature vector of each pixel in the neighborhood as the first l The key set of target pixels k l ∈R (γ×γ)×C ;calculate q l and k T l The matrix product of l The pairwise affinity of a pixel with its neighborhood pixels, thus generating a relationship vector A i ∈R 1×(γ×γ) , i ={1,2, ...,HW}: ； The relationship vector z The element represents the l Pixels and μ Neighborhood z Attention scores for pixels; Use the Unfold operation to generate a dynamic shift window around each pixel and slide it on the key feature map K to extract the nearest neighbor of each pixel. Generate the pairwise dependency relationship between all target pixels and their nearest neighbor pixels through matrix multiplication operation; In the GRG module, the specific process of generating paired dependencies is as follows: The output X∈R of the last stage of ResNet-50 C×H×W As GRG input, C, H, W represent the number of channels, height, and width of the feature map respectively; perform three 1×1 convolutions on X to obtain three feature maps: query feature map Q∈R C×H×W , key feature map K∈R C×H×W , value feature map V∈R C×H×W : ( Q , K , V ) = Conv2d(X); Use a uniform grid of size sh×sw along the spatial dimension to query the feature map Q Divide and transform the divided tensor to obtain the query tensor Q ’ ∈R sh×sw×(HW / (sh*sw))×C , where sh and sw represent the height and width of each grid, sh*sw represents the number of pixels in a grid, and HW / (sh*sw) represents the number of grids obtained after grid division. Similarly, a uniform grid of size sh×sw is used to key feature maps. K Perform grid division and apply a deformation operation to the divided tensor to obtain the key tensor K ’ ∈R sh×sw×(HW / (sh*sw))×C : Q ’ = Reshape(grid_partition( Q )); K ’ = Reshape(grid_partition( K )); The query tensor Q ’ and key tensors K ’ Perform matrix multiplication to obtain the relational matrix A ∈R sh ×sw×(HW / (sh*sw))×(HW / (sh*sw)) : ； In the matrix A In the , each row represents the i The pairwise dependencies between pixels in a grid and pixels in other grids.

2. The convolutional neural network based on neighborhood and grid attention mechanism according to claim 1, characterized in that: The queries for all pixels in the query feature map Q are stacked together to generate a query tensor Q ’ ∈R HW×1×C , by applying a kernel size of γ An Unfold operation with a step size of 1 and a Transform operation generates a HW×C× γ 2 The key tensor K ’ , so that the key sets of HW target pixels are stacked together; the query tensor Q ’ and the key tensor K ’ Perform matrix multiplication to obtain the relational tensor A ∈R HW×1×(γ×γ) : ； Among them, the vector 1×( γ × γ ) indicates that a pixel and its μ ∈ γ 2 The tensor A reflects the pairwise affinity between each pixel and its nearest neighbor.

3. The convolutional neural network based on neighborhood and grid attention mechanism according to claim 2, characterized in that: In the NRG module, the specific process of generating the attention map is: Based on the pairwise dependency between the target pixel and its nearest neighbor pixel, the tensor A is transformed to obtain the matrix A ’ ∈R HW ×(γ×γ) ; right A ’ The last dimension of is compressed to calculate the average features of the pixels in the neighborhood, and the vector W ∈R HW×1 ; Each value in W is obtained by calculating the mean of the pairwise dependencies between each pixel and the pixels in its neighborhood: W = Mean(Reshape( A )); After transforming the dimension of W, a 1D convolution operation is applied, and the attention weight of each pixel is generated through a Softmax normalization operation to obtain the weight vector W ’ ∈R 1×HW : W ’ = Softmax(1D( W )); W ’ Reshape to get the attention map Y ∈R 1×H×W .

4. The convolutional neural network based on neighborhood and grid attention mechanism according to claim 3, characterized in that: The attention map Y is multiplied element by element with the value feature map V to generate a feature map F ∈R C×H×W : F = Y☉V; The feature map F Add it to the input feature map X, apply batch normalization BN and GELU activation function to the added feature map in turn, and get the output feature map of NRG F ’ : F ’ = GELU(BN(F + X))。 5. The convolutional neural network based on neighborhood and grid attention mechanism according to claim 1, characterized in that: In the GRG module, the specific process of generating attention weights is: Based on the relationship matrix A , transform it to generate a tensor A ’ ∈R HW×(HW / (sh*sw)) ,right A ’ The last dimension of P ∈R HW×1 , then, P Transform to get vector W ∈R 1×HW : W = Reshape(Mean( A ’ )); Next, W Implement a 1D convolution operation and perform a Softmax normalization operation on the last dimension of the resulting tensor to output the attention weight of each pixel, thereby obtaining a weight vector W ’ ∈R 1×HW : W ’ = Softmax(1D( W ))。 6. The convolutional neural network based on neighborhood and grid attention mechanism according to claim 5, characterized in that: The weight vector W ’ Deformation to get attention map G ∈R 1×H×W ,Will G Characteristic graph with value V Perform element-by-element multiplication to complete the weighting operation on the original features, thereby highlighting the most significant dependencies in the feature graph: G ’ = V☉Reshape( W ’ ); The feature map G ’ Add it to the input feature map X and perform BN and GELU activation functions on the added feature map to finally generate the output feature map F of GRG: F = GELU(BN( G ’ + X))。 7. The convolutional neural network based on neighborhood and grid attention mechanism according to any one of claims 1 to 6, characterized in that: The output feature maps of the three branches are globally averaged pooled to generate a 2048-dimensional feature vector, and then the dimension is reduced to 256 dimensions. The reduced 256-dimensional feature vector is used to calculate the triplet loss in the training phase, and the 256-dimensional feature is input into the fully connected layer to calculate the cross entropy loss.

8. The convolutional neural network based on neighborhood and grid attention mechanism according to claim 7, characterized in that: The reduced 256-dimensional feature vectors of the three branches are concatenated together as the final feature representation during the testing phase.

Images