A traditional pattern image retrieval method based on self-attention mechanism
By using the traditional pattern sub-map search method with self-attention mechanism in image retrieval, combined with convolutional neural network and Transformer, the feature extraction problem in sub-map search is solved, and the retrieval efficiency and accuracy are improved.
Patent Information
- Application Number
- CN202210105755.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-28
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-01-28
AI Technical Summary
The prior art is difficult to effectively extract specific regional features in the sub-picture in image retrieval, resulting in low retrieval efficiency, especially in sub-picture retrieval tasks, which are difficult to meet user needs.
Using the traditional pattern map retrieval method based on the self-attention mechanism, the data set is constructed, the feature maps of different levels are extracted using convolutional neural networks and feature pyramid fusion is performed, and combined with Transformer and Vision Transformer, attention weights are calculated and features are aggregated. Implement the extraction and similarity calculation of subgraph features.
The efficiency and accuracy of image retrieval is improved. By integrating high-level semantic features and underlying spatial features, smaller detailed information is captured, unnecessary information is ignored, and better search results are achieved.
Smart Images

Figure CN114510594B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing and computer vision technology, and in particular to a traditional texture-like image retrieval method based on a self-attention mechanism. Background Art
[0002] With the development of the Internet and digital economy, multimedia data is also growing rapidly. More and more data can be saved in the form of images, and the types are more complex. How to find the images you need in a large number of images has become a difficult problem. Image retrieval has been a very active research field since the 1970s. Image retrieval methods can be divided into two categories: text-based image retrieval (TBIR) and content-based image retrieval (CBIR). The entire visual content in an image is sometimes difficult to express in words, so TBIR will produce irrelevant results. CBIR refers to retrieving images using information such as color, texture, contour, and spatial relationship of the image.
[0003] With the development of deep learning, the focus of research has begun to shift to methods based on deep learning. Early research on image retrieval using DCNN generally used fully connected layer features as descriptors. However, this method has problems such as high feature dimension and limited image input. With the increase in the complexity of retrieval tasks and the deepening of research, it was found that compared with the fully connected layer, the convolutional layer features contain certain position feature information, making them more suitable as image descriptors. Therefore, the research on image feature descriptors has gradually shifted from the fully connected layer to the convolutional layer. Convolutional neural networks have three important characteristics: a certain degree of scaling and rotation invariance; shared weights and local receptive fields; and capture of hierarchical features from details to the whole. These characteristics make convolutional neural networks very suitable for processing computer vision tasks and have become the cornerstone of the CV field in the era of deep learning.
[0004] After the Transformer was proposed in 2017, it was successfully applied to many NLP tasks. In 2020, Vision Transformer applied Transformer to the CV field with minimal changes. The core of Transformer is the Attention mechanism. Attention imitates the internal process of biological observation behavior. It can filter out a small amount of important information from a large amount of information, ignore most of the unimportant information, and focus on these important information. A sub-image refers to an image that the user extracts from the original image by segmentation. The background of the original image has been removed. Only specific areas in the sub-image help to construct distinctive global features. Sub-image retrieval is a new challenge and a new exploration. How to extract specific areas in a sub-image, better meet user needs, solve the problem of sub-image retrieval, and improve retrieval efficiency are problems that need to be solved urgently in the industry. Summary of the invention
[0005] In view of the problems in the related art, the present invention proposes a traditional texture sub-image retrieval method based on the self-attention mechanism to overcome the above-mentioned technical problems existing in the existing related art.
[0006] To this end, the specific technical solution adopted by the present invention is as follows:
[0007] A traditional texture image retrieval method based on a self-attention mechanism, the method comprising the following steps:
[0008] S1. Build a dataset based on traditional pattern images and use the dataset to train the pre-built sub-image retrieval model;
[0009] S2, using convolutional neural network to extract a series of feature maps at different levels, and fuse the extracted feature maps using feature pyramid;
[0010] S3, using Transformer to extract global and local features from the fused feature map generated based on the database image;
[0011] S4. Use Vision Transformer to calculate the attention weight of the sub-graph input by the user, and aggregate the weighted fusion feature graph to obtain the sub-graph feature;
[0012] S5. Calculate the similarity between the query subgraph features and the database image features, and sort them according to the similarity scores to achieve subgraph retrieval.
[0013] Furthermore, the step of constructing a data set based on traditional pattern images and using the data set to train a pre-constructed sub-image retrieval model includes the following steps:
[0014] S11, obtaining a preset number of traditional pattern pictures, and manually marking the target positions and category information in the traditional pattern pictures to obtain a marked data set;
[0015] S12. Randomly divide the entire database into two sets, a training set and a test set, with a ratio of 8:2. During the training process, use the training set to train the fitting model, use the test set to perform model prediction, measure the performance and ability of the model, and record the accuracy;
[0016] S13, using image processing tools to segment and extract sub-elements in the sorted database image, and remove the background to obtain a query sub-image dataset;
[0017] S14. Construct a subgraph retrieval model, and use the data set to train the subgraph retrieval model.
[0018] Furthermore, the sub-image retrieval model adds a feature fusion module based on the DETR model, uses ResNet as the backbone to extract features of different convolutional layers of the image, and obtains a 1024-dimensional feature map after feature fusion. The feature map is input into the Transformer structure, and the Transformer generates N predictions. The loss function used is as follows:
[0019]
[0020] Among them, y i Represents ground truth, that is, the labeled categories and target locations in the dataset, represents the σ(i)th prediction generated by Transormer, represents the ground truth y i and prediction The sum of the category loss and IoU loss between them. Since the number of N is greater than the number of ground truth, y i Use the empty set to fill N, so that it can be matched with N predictions in a bipartite graph. When the permutation between N elements The loss is minimized, which is the optimal bipartite graph matching between the two sets. Indicates the optimal bipartite graph matching between the prediction set and the ground truth. Further, the method of extracting a series of feature graphs at different levels using a convolutional neural network and fusing the extracted feature graphs using a feature pyramid includes the following steps:
[0021] S21, using the residual network to extract the feature activation output of the last residual structure of each stage of the input image, and using {L1, L2, L3, L4} to represent it, where L1 to L4 represent different residual network convolution groups;
[0022] S22, using a 1*1 convolutional layer to reduce the number of channels of the L4 feature map to the same as the number of channels of the L3 feature map;
[0023] S23, upsampling using the nearest neighbor interpolation method to adjust the dimension of the L4 feature map to be consistent with the dimension of the L3 feature map;
[0024] S24, adding the adjusted L4 feature map to the L3 feature map element by element to obtain a fused feature map;
[0025] S25, using a 1*1 convolutional layer to reduce the number of channels of the fused feature map, and using the nearest neighbor interpolation method to upsample and add it to the L2 feature map to obtain the final fused feature map;
[0026] S26. Use a 3*3 convolutional layer to fuse the generated fused feature map to generate a feature map with a final channel number of 1024 dimensions.
[0027] Furthermore, the calculation formula of the fused feature map during feature fusion is as follows:
[0028]
[0029]
[0030] G f (s, t, j) = G L (s, t, d)*δ(3, 3, d);
[0031] Among them, δ(1, 1, k) represents the k-dimensional 1x1 convolution kernel, δ(3, 3, d) represents the d-dimensional 3x3 convolution kernel, Indicates L i The feature map has a layer width of m, a height of n, and a number of channels of k. Indicates L i The number of layer feature channels is reduced to the feature map of d, Indicates L i-1 The feature map G is s in width, t in height and d in channels. L (s, t, d) represents L i With L i-1 The width of the combined feature map of the layer is s, the height is t, and the number of channels is d. f (s, t, j) represents the final generated fusion feature map with a width of s, a height of t, and a number of channels j. upsample [*] indicates upsampling.
[0032] Furthermore, the extraction of global features and local features from the fused feature map generated based on the database image using Transformer includes the following steps:
[0033] S31, taking the fused feature map as the global feature map of the database image, inputting it into the Transformer to generate a pre-selected box, and mapping the pre-selected box back into the fused feature map to extract a local feature map;
[0034] S32, using the R-MAC algorithm to aggregate the global feature map and the local feature map, generate the global features and local features of the database image and save them to the database.
[0035] Furthermore, the Transformer does not need to predict when generating the pre-selected box, but maps the coordinates of the pre-selected box back to the fusion feature map, extracts the local convolution feature map, and each local convolution feature map is obtained by the following formula:
[0036]
[0037] Among them, X represents the convolution feature map of the image, which is essentially a three-dimensional matrix. w and h represent the width and height of the convolution feature respectively. min , x max ,y min ,y max Indicates the coordinate position of the generated pre-selected box in the original image, I w , I h Represents the width and height of the image. When the image in the database is input into the Transformer, multiple target areas will be generated. The subscript i represents the i-th pre-selected box generated. Indicates rounding up. Indicates rounding down.
[0038] Furthermore, the method of using Vision Transformer to calculate the attention weight of the sub-graph input by the user and aggregating the weighted fusion feature graph to obtain the sub-graph feature includes the following steps:
[0039] S41, extracting a global feature graph after fusing the query subgraph input by the user using a convolutional neural network;
[0040] S42, use Vision Transformer to calculate the self-attention weight, and use the R-MAC algorithm to aggregate the weighted fusion feature map to obtain the sub-graph feature;
[0041] The weighted calculation formula in S42 is as follows:
[0042] F attention (s, t, j) = Attention(G f (s,t,j))*G f (s, t, j);
[0043] Where s and t represent the width and height of the feature map, j represents the number of channels of the feature map, and G f (s, t, j) represents the fusion feature map extracted from S2, Attention (G f (s, t, j)) represents G f The attention weight generated after (s, t, j) is input into Vision Transformer, F attention (s, t, j) represents the f (s, t, j) weighted result.
[0044] Furthermore, the calculation of the similarity between the query subgraph feature and the database image feature, and sorting according to the similarity score to implement subgraph retrieval includes the following steps:
[0045] S51, obtaining the query subgraph features and the global features and local features of the database image, calculating the Euclidean distances one by one, and selecting the minimum distance value as the similarity score;
[0046] S52, selecting a feature vector of a database image that is most similar to a feature vector of a query subgraph as a representation vector of the database image, and sorting them according to the similarity scores to implement subgraph retrieval.
[0047] Furthermore, the calculation formula of the similarity score is as follows:
[0048] sim (D, Q) = min [dist (x0, y), dist (x1, y), ..., dist (x n , y)];
[0049] Where D and Q represent the database image and query subgraph respectively, y represents the global feature vector of the query subgraph, x1, x2, ..., x n Represents the global feature vector and local feature vector of the database image, dist(x n , y) represents the distance between the features of the nth region of the database image and the features of the query subgraph.
[0050] The beneficial effects of the present invention are as follows: the traditional texture sub-image retrieval method based on the self-attention mechanism proposed by the present invention adds a feature fusion module between the convolutional neural network and the Transformer, and fuses the high-level features with rich semantic information with the corresponding low-level feature maps with rich spatial information, so that the pre-selected box generated by the Transformer can capture smaller details. The self-attention weight is calculated for the fused features using the attention mechanism, and the sub-image feature maps are weighted, ignoring most of the unimportant information to obtain better retrieval results. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0052] Figure 1 4 is a flow chart of a traditional texture image retrieval method based on a self-attention mechanism according to an embodiment of the present invention. DETAILED DESCRIPTION
[0053] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention and are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in the field should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.
[0054] According to an embodiment of the present invention, a traditional texture sub-image retrieval method based on a self-attention mechanism is provided.
[0055] The present invention is further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to a traditional texture image retrieval method based on a self-attention mechanism according to an embodiment of the present invention, the method comprises the following steps:
[0056] S1. Build a dataset based on traditional pattern images and use the dataset to train the pre-built sub-image retrieval model;
[0057] The step of constructing a data set based on traditional pattern images and using the data set to train a pre-constructed sub-image retrieval model includes the following steps:
[0058] S11, obtaining a preset number of traditional pattern pictures (collecting and arranging a large number of traditional pattern pictures), and manually marking the target position and category information in the traditional pattern pictures to obtain a marked data set;
[0059] S12. Randomly divide the entire database into two sets, a training set and a test set, with a ratio of 8:2. During the training process, use the training set to train the fitting model, use the test set to perform model prediction, measure the performance and ability of the model, and record the accuracy;
[0060] S13, using image processing tools to segment and extract sub-elements in the labeled database image, and remove the background to obtain a query sub-image dataset;
[0061] S14. Construct a subgraph retrieval model, and use the data set to train the subgraph retrieval model.
[0062] Specifically, the sub-image retrieval model adds a feature fusion module on the basis of the DETR model, uses ResNet as the backbone to extract features of different convolutional layers of the image, and obtains a 1024-dimensional feature map after feature fusion. The feature map is input into the Transformer structure, and the Transformer generates N predictions. The loss function used is as follows:
[0063]
[0064] Among them, y i Represents ground truth, that is, the labeled categories and target locations in the dataset, represents the σ(i)th prediction generated by Transormer, represents the ground truth y i and prediction The sum of the category loss and IoU loss between them. Since the number of N is greater than the number of ground truth, y i Use the empty set to fill N, so that it can be matched with N predictions in a bipartite graph. When the permutation between N elements The loss is minimized, which is the optimal bipartite graph matching between the two sets. Represents the optimal bipartite graph matching between the prediction set and the ground truth.
[0065] S2, using convolutional neural network to extract a series of feature maps at different levels, and fuse the extracted feature maps using feature pyramid;
[0066] The method of extracting a series of feature maps at different levels by using a convolutional neural network and fusing the extracted feature maps by using a feature pyramid includes the following steps:
[0067] S21. Use the residual network (ResNet) to extract the feature activation output of the last residual structure of each stage of the input image, represented by {L1, L2, L3, L4}, where L1 to L4 represent different residual network convolution groups. Since the L1 layer has too much noise, only {L2, L3, L4} is used;
[0068] S22, using a 1*1 convolutional layer to reduce the number of channels of the L4 feature map to the same as the number of channels of the L3 feature map;
[0069] S23, using the nearest neighbor interpolation method to perform upsampling, adjusting the dimension of the L4 feature map to be consistent with the dimension of the L3 feature map. The nearest neighbor interpolation method can retain the semantic information of the feature map to the greatest extent;
[0070] S24, adding the adjusted L4 feature map to the L3 feature map element by element to obtain a fused feature map;
[0071] S25, using a 1*1 convolutional layer to reduce the number of channels of the fused feature map, and using the nearest neighbor interpolation method to upsample and add it to the L2 feature map to obtain the final fused feature map;
[0072] S26. Use a 3*3 convolutional layer to fuse the generated fused feature map to eliminate the overlapping effect caused by the upsampling process and generate a feature map with a final channel number of 1024 dimensions.
[0073] Specifically, the calculation formula of the fused feature map during feature fusion is as follows:
[0074]
[0075]
[0076] G f (s, t, j) = G L (s, t, d)*δ(3, 3, d)
[0077] Among them, δ(1, 1, k) represents the k-dimensional 1x1 convolution kernel, δ(3, 3, d) represents the d-dimensional 3x3 convolution kernel, Indicates L i The feature map has a layer width of m, a height of n, and a number of channels of k. Indicates L i The number of layer feature channels is reduced to the feature map of d, Indicates L i-1 The feature map G is s in width, t in height and d in channels. L (s, t, d) represents L i With L i-1 The width of the combined feature map of the layer is s, the height is t, and the number of channels is d. f (s, t, j) represents the final generated fusion feature map with a width of s, a height of t, and a number of channels j. upsample [*] indicates upsampling.
[0078] S3, using Transformer to extract global and local features from the fused feature map generated based on the database image;
[0079] The method of extracting global features and local features from the fused feature map generated based on the database image using Transformer includes the following steps:
[0080] S31, for the fused feature map generated by the database image, the fused feature map is used as the global feature map of the database image, input into the Transformer to generate a pre-selected box, and the pre-selected box is mapped back to the fused feature map to extract the local feature map;
[0081] S32, use the R-MAC algorithm to aggregate the global feature map and the local feature map, generate the global features and local features of the database image and save them to the database. The R-MAC algorithm is a regional aggregation algorithm extracted in 2015. It uses a variable window method to slide the sampling on the feature plane and selects a region size of R s The convolutional features are sampled with a sliding window, and the overlap area between the two sliding windows needs to be at least 40%. After sampling, R-MAC will perform maximum pooling, L2 normalization and PCA on all regional feature maps, and then use sum pooling to obtain the global feature vector, and then perform L2 normalization again.
[0082] Specifically, when generating the pre-selected box, the Transformer does not need to be predicted by the later FFN layer. The coordinates of the pre-selected box are mapped back to the fusion feature map, and the local convolution feature map is extracted. Each local convolution feature map is obtained by the following formula:
[0083]
[0084] Among them, X represents the convolution feature map of the image, which is essentially a three-dimensional matrix. w and h represent the width and height of the convolution feature respectively. min , X max ,y min ,y max Indicates the coordinate position of the generated pre-selected box in the original image, I w , I h Represents the width and height of the image. When the image in the database is input into the Transformer, multiple target areas will be generated. The subscript i represents the i-th pre-selected box generated. Indicates rounding up. Indicates rounding down.
[0085] S4. Use Vision Transformer to calculate the attention weight of the sub-graph input by the user, and aggregate the weighted fusion feature graph to obtain the sub-graph feature;
[0086] The method of using Vision Transformer to calculate the attention weight of the sub-graph input by the user and aggregating the weighted fusion feature graph to obtain the sub-graph feature includes the following steps:
[0087] S41, extracting a global feature graph after fusion of the query subgraph input by the user using a convolutional neural network;
[0088] S42, since the user input is extracted from the original image, the background of the original image has been removed, and only specific areas in the sub-image are helpful in constructing distinctive global features. Therefore, VisionTransformer is used to calculate the self-attention weight map, and weighted with the fusion feature map, and then aggregated using the R-MAC algorithm to obtain the sub-image features;
[0089] The weighted calculation formula in S42 is as follows:
[0090] F attention (s, t, j) = Attention(G f (s,t,j))*G f (s, t, j)
[0091] Where s and t represent the width and height of the feature map, j represents the number of channels of the feature map, and G f (s, t, j) represents the fusion feature map extracted from S2, Attention (G f (s, t, j)) represents G f The attention weight generated after (s, t, j) is input into Vision Transformer, F attention (s, t, j) represents the f (s, t, j) weighted result.
[0092] The step of calculating the similarity between the query subgraph feature and the database image feature and sorting the similarity scores to implement subgraph retrieval includes the following steps:
[0093] S51, obtaining the query subgraph features and the global features and local features of the database image, calculating the Euclidean distances one by one, and selecting the minimum distance value as the similarity score;
[0094] S52, selecting a feature vector of a database image that is most similar to a feature vector of a query subgraph as a representation vector of the database image, and sorting them according to the similarity scores to implement subgraph retrieval.
[0095] Specifically, the calculation formula of the similarity score is as follows:
[0096] sim (D, Q) = min [dist (x0, y), dist (x1, y), ..., dist (x n , y)];
[0097] Where D and Q represent the database image and query subgraph respectively, y represents the global feature vector of the query subgraph, x1, x2, ..., x n Represents the global feature vector and local feature vector of the database image, dist(xn , y) represents the distance between the features of the nth region of the database image and the features of the query subgraph.
[0098] In summary, with the help of the above technical solution of the present invention, the traditional texture sub-image retrieval method based on the self-attention mechanism proposed by the present invention adds a feature fusion module between the convolutional neural network and the Transformer, and fuses the high-level features with rich semantic information with the corresponding low-level feature maps with rich spatial information, so that the pre-selected box generated by the Transformer can capture smaller details. The self-attention weight is calculated for the fused features using the attention mechanism, and the sub-image feature maps are weighted, ignoring most of the unimportant information to obtain better retrieval results.
[0099] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A traditional texture image retrieval method based on self-attention mechanism, characterized in that: The method comprises the following steps: S1. Build a dataset based on traditional pattern images and use the dataset to train the pre-built sub-image retrieval model; S2, using convolutional neural network to extract a series of feature maps at different levels, and fuse the extracted feature maps using feature pyramid; S3, using Transformer to extract global and local features from the fused feature map generated based on the database image; S4. Use Vision Transformer to calculate the attention weight of the sub-graph input by the user, and aggregate the weighted fusion feature graph to obtain the sub-graph feature; S5, calculating the similarity between the query sub-graph features and the database image features, and sorting them according to the similarity scores to realize sub-graph retrieval; The step of constructing a training data set based on traditional pattern images and using the data set to train a pre-constructed sub-image retrieval model includes the following steps: S11, obtaining a preset number of traditional pattern pictures, and manually marking the target positions and category information in the traditional pattern pictures to obtain a marked data set; S12. Randomly divide the entire database into two sets, a training set and a test set, with a ratio of 8:
2. During the training process, use the training set to train the fitting model, use the test set to perform model prediction, measure the performance and ability of the model, and record the accuracy; S13, using image processing tools to segment and extract sub-elements in the labeled database image, and remove the background to obtain a query sub-image dataset; S14, constructing a subgraph retrieval model, and using the data set to train the subgraph retrieval model; The sub-image retrieval model adds a feature fusion module on the basis of the DETR model, uses ResNet as the backbone to extract features of different convolutional layers of the image, and obtains a 1024-dimensional feature map after feature fusion. The feature map is input into the Transformer structure, and the Transformer generates N predictions. The loss function used is as follows: Among them, y i Represents ground truth, that is, the labeled categories and target locations in the dataset, represents the σ(i)th prediction generated by Transormer, Indicates ground truthy i and prediction The sum of the category loss and IoU loss between them. Since the number of N is greater than the number of ground truth, y i Use the empty set to fill N, so that it can be matched with N predictions in a bipartite graph. When the permutation between N elements The loss is minimized, which is the optimal bipartite graph matching between the two sets. Represents the optimal bipartite graph matching between the prediction set and the ground truth; The method of extracting a series of feature maps at different levels by using a convolutional neural network and fusing the extracted feature maps by using a feature pyramid includes the following steps: S21, using the residual network to extract the feature activation output of the last residual structure of each stage of the input image, and using {L1, L2, L3, L4} to represent it, where L1 to L4 represent different residual network convolution groups; S22, using a 1*1 convolutional layer to reduce the number of channels of the L4 feature map to the same as the number of channels of the L3 feature map; S23, upsampling using the nearest neighbor interpolation method to adjust the dimension of the L4 feature map to be consistent with the dimension of the L3 feature map; S24, adding the adjusted L4 feature map to the L3 feature map element by element to obtain a fused feature map; S25, using a 1*1 convolutional layer to reduce the number of channels of the fused feature map, and using the nearest neighbor interpolation method to upsample and add it to the L2 feature map to obtain the final fused feature map; S26. Use a 3*3 convolutional layer to fuse the generated fused feature map to generate a feature map with a final channel number of 1024 dimensions.
2. According to the traditional texture image retrieval method based on self-attention mechanism according to claim 1, it is characterized in that: The calculation formula of the fused feature map during feature fusion is as follows: G f (s,t,j)=G L (s,t,d)*δ(3,3,d); Among them, δ(1,1,k) represents the k-dimensional 1x1 convolution kernel, δ(3,3,d) represents the d-dimensional 3x3 convolution kernel, Indicates L i The feature map has a layer width of m, a height of n, and a number of channels of k. Indicates L i The number of layer feature channels is reduced to the feature map of d, Indicates L i-1 The feature map G is s in width, t in height and d in channels. L (s,t,d) represents L i With L i-1 The width of the combined feature map of the layer is s, the height is t, and the number of channels is d. f (s, t, j) represents the final generated fusion feature map with a width of s, a height of t, and a number of channels j. upsaMple [*] indicates upsampling.
3. The traditional texture image retrieval method based on self-attention mechanism according to claim 1, characterized in that: The method of extracting global features and local features from the fused feature map generated based on the database image using Transformer includes the following steps: S31, taking the fused feature map as the global feature map of the database image, inputting it into the Transformer to generate a pre-selected box, and mapping the pre-selected box back into the fused feature map to extract a local feature map; S32, using the R-MAC algorithm to aggregate the global feature map and the local feature map, generate the global features and local features of the database image and save them to the database.
4. The traditional texture image retrieval method based on self-attention mechanism according to claim 3, characterized in that: The Transformer does not need to be predicted when generating the pre-selected box. It maps the coordinates of the pre-selected box back to the fusion feature map and extracts the local convolution feature map. Each local convolution feature map is obtained by the following formula: Among them, X represents the convolution feature map of the image, which is essentially a three-dimensional matrix. w and h represent the width and height of the convolution feature respectively. min , x max ,y min ,y max Indicates the coordinate position of the generated pre-selected box in the original image, I w , I h Represents the width and height of the image. When the image in the database is input into the Transformer, multiple target areas will be generated. The subscript i represents the i-th pre-selected box generated. Indicates rounding up. Indicates rounding down.
5. The traditional texture image retrieval method based on self-attention mechanism according to claim 1, characterized in that: The method of using Vision Transformer to calculate the attention weight of the sub-graph input by the user and aggregating the weighted fusion feature graph to obtain the sub-graph feature includes the following steps: S41, extracting a global feature graph after fusing the query subgraph input by the user using a convolutional neural network; S42, use Vision Transformer to calculate the self-attention weight, and use the R-MAC algorithm to aggregate the weighted fusion feature map to obtain the sub-graph feature; The weighted calculation formula in S42 is as follows: F attention (s,t,j)=Attention(G f (s,t,j))*G f (s,t,j); Where s and t represent the width and height of the feature map, j represents the number of channels of the feature map, and G f (s, t, j) represents the fusion feature map, Attention (G f (s,t,j)) represents G f The attention weight generated after (s, t, j) is input into Vision Transformer, F attention (s,t,j) represents the f The weighted result of (s, t, j).
6. The traditional texture image retrieval method based on self-attention mechanism according to claim 1, characterized in that: The method of calculating the similarity between the query subgraph feature and the database image feature and sorting the subgraph features according to the similarity scores to implement subgraph retrieval includes the following steps: S51, obtaining the query subgraph features and the global features and local features of the database image, calculating the Euclidean distances one by one, and selecting the minimum distance value as the similarity score; S52, selecting a feature vector of a database image that is most similar to a feature vector of a query subgraph as a representation vector of the database image, and sorting them according to the similarity scores to implement subgraph retrieval.
7. The traditional texture image retrieval method based on self-attention mechanism according to claim 6, characterized in that: The calculation formula of the similarity score is as follows: sim(d,Q)=min[dist(x0,y),dist(x1,y),...,dist(x n ,y)]; Where D and Q represent the database image and query subgraph respectively, y represents the global feature vector of the query subgraph, x1, x2, ..., x n Represents the global feature vector and local feature vector of the database image, dist(x n , y) represents the distance between the features of the nth region of the database image and the features of the query subgraph.
