Feature-enhanced cross-window lightweight super-resolution method, system and electronic device

Abstract

The present invention relates to the field of image super-resolution technology, and discloses a feature-enhanced cross-window lightweight super-resolution method, system and electronic device. The method comprises the following steps: constructing a self-attention mechanism based on Token dictionary intersection, and introducing a Token grouping mechanism to form a dictionary intersection self-attention module based on Token grouping, i.e., a TDAB module; constructing a feature-enhanced cross-window lightweight super-resolution network based on the TDAB module; and reconstructing a low-resolution image to be reconstructed using the feature-enhanced cross-window lightweight super-resolution network to obtain a reconstruction result. Based on the classical super-resolution method, the method introduces a self-attention mechanism based on Token dictionary intersection and a hybrid attention module based on Token grouping, which makes the model lighter while enhancing the model's ability to capture cross-window dependencies and the reconstruction effect of texture detail features.

Description

Technical Field

[0001] The present invention relates to the field of image super-resolution technology, and in particular to a feature-enhanced cross-window lightweight super-resolution method, system and electronic device. Background Art

[0002] Single image super resolution (SISR) is widely used in the field of computer vision, such as medical images, video surveillance, remote sensing images, and video transmission. SISR generates the corresponding high-resolution (HR) images based on existing low-resolution (LR) images through software processing. With the development of deep learning, methods based on convolutional neural networks (CNN) have far surpassed traditional interpolation algorithms, and can learn more accurate mapping relationships between HR-LR image blocks, and the reconstructed HR images are of higher quality. Therefore, CNN-based methods are the main methods for single image super-resolution research at this stage.

[0003] Super-resolution methods based on deep learning can be roughly divided into two categories. The first category is based on generative adversarial networks. This type of method optimizes the perceptual loss to make the generated HR images more consistent with human subjective visual perception. However, the PSNR and SSIM indicators of the HR images reconstructed by this type of algorithm are low, and the detail texture is quite different from the original image, so the defects in practical applications are also more obvious. The second category pays more attention to the details and texture features of the reconstructed image, and its objective indicators are also higher than the first method. However, this type of method still has some problems. First, in order to improve the quality of image reconstruction, a large number of modules often need to be stacked in the model to increase the depth of the network, but this makes the model training more difficult and time-consuming. Secondly, due to the lack of thinking and research on the feature extraction module in some super-resolution algorithms, the extracted deep features are weak. In addition, these models lack the ability to adaptively distinguish important features from secondary features, and treat all feature information in the image equally, which directly affects the high-frequency features of the reconstructed image. Summary of the invention

[0004] The purpose of the present invention is to solve the problem in the prior art that the feature information extracted by the classical super-resolution model is too limited to the window.

[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide a feature-enhanced cross-window lightweight super-resolution method, comprising the following steps:

[0006] Construct a self-attention mechanism based on Token dictionary intersection and introduce a Token grouping mechanism to form a dictionary cross self-attention module based on Token grouping, namely the TDAB module;

[0007] Construct a feature-enhanced cross-window lightweight super-resolution network based on the TDAB module;

[0008] The feature-enhanced cross-window lightweight super-resolution network is used to reconstruct the low-resolution image to be reconstructed to obtain a reconstruction result.

[0009] Preferably, the internal execution process of the self-attention mechanism based on Token dictionary intersection includes the following steps:

[0010] Introduce an additional dictionary D∈R M×d , initialize it as the network parameter, M represents the spatial dimension of the dictionary, and d represents the channel dimension of the dictionary;

[0011] Generate a key dictionary K using dictionary D D Sum value dictionary V D , and use the input features X∈R N×d The query tokens are generated by 1×1 convolution and 3×3 depth convolution, which is expressed as:

[0012]

[0013] K D =DW K ;

[0014] V D =XW V ;

[0015] Among them, Q X Indicates query Tokens; W d Q represents a 3×3 depth convolution, W c Q represents a 1×1 standard convolution, W K ∈R d×d and W V ∈R d×d are the linear transformations of the key dictionary Tokens and the value dictionary Tokens respectively; N represents the number of Tokens, i.e., the spatial dimension of the input features;

[0016] Use the key dictionary and value dictionary to enhance the query Tokens through cross-attention calculation and get the final output X out , expressed as:

[0017] A=SoftMax(Sim cos (Q X,K D ) / δ);

[0018] X out =TDCA(Q X ,K D ,V D )=A·V D ;

[0019] Among them, δ is a learnable parameter used to adjust the range of similarity values; Sim cos (·,·) indicates the calculation of cosine similarity, and A represents the self-attention map.

[0020] Preferably, M<<N is set to keep the computational cost low.

[0021] Preferably, the internal execution process of the dictionary cross self-attention module based on token grouping includes the following steps:

[0022] Use the Token grouping mechanism to evenly group the input Token X and Token dictionary D and map the grouping results separately;

[0023] Depthwise separable convolutions with different kernel sizes are used to finely group the mapping results;

[0024] The cross-attention calculation is used to enhance the results of fine grouping and obtain the final output.

[0025] Preferably, the method of using the Token grouping mechanism to evenly group the input Token X and the Token dictionary D and mapping the grouping results respectively includes the following steps:

[0026] For the input TokenX∈R N×d And Token dictionary D∈R M×d Grouping is performed, expressed as:

[0027] X i =Split(X),i=0,1,2,3;

[0028] D i =Split(D),i=0,1,2,3;

[0029] Among them, Split represents the grouping operation. represents the i-th group after Token X is grouped. Represents the i-th group after the dictionary D is grouped;

[0030] X is transformed into i Mapped to query TokensQ i, through linear mapping, the four groups of dictionary D are mapped to key dictionaries K Di Sum value dictionary V Di , expressed as:

[0031]

[0032] K Di =D i W K ,i=0,1,2,3;

[0033] V Di =D i W V ,i=0,1,2,3.

[0034] Preferably, the method of finely grouping the mapping results by using depthwise separable convolutions with different kernel sizes comprises the following steps:

[0035] When i=0, no fine grouping is performed, and Q0, K D0 and V D0 Keep its single token mode;

[0036] When i=1,2,3, for Q i , K Di and V Di Perform refined grouping, expressed as:

[0037] Q' i =Re fine i (Q i ),i=1,2,3;

[0038] K' Di =Re fine i (K Di ),i=1,2,3;

[0039] V' Di =Re fine i (V Di ),i=1,2,3;

[0040] in, and Respectively represent Q i , K Di 、V Di The result after refined grouping; i (·) denotes a refined grouping operation, including three DSConv with kernel sizes of 3×3, 5×5, and 7×7 and a Hardswish activation function.

[0041] Preferably, the method of enhancing the result of fine grouping by using cross attention calculation comprises the following steps:

[0042] Using K' Di and V' Di Enhancing Q' by cross-attention computation i , the final output of the dictionary cross self-attention module based on Token grouping is obtained, which is expressed as:

[0043] A=SoftMax(Sim cos (Q',K D ') / δ);

[0044] Y=A·V D ';

[0045] Among them, A∈R N×M For Q' and K D 'The obtained self-attention map, Y∈R N×d Represents the final output.

[0046] Preferably, the feature-enhanced cross-window lightweight super-resolution network includes:

[0047] The shallow feature extraction module uses a 3×3 convolutional layer to perform shallow feature extraction on the input low-resolution image;

[0048] The deep feature extraction module uses four sequentially connected residual grouping attention groups and a 3×3 convolutional layer to perform deep feature extraction on shallow feature extraction; the residual grouping attention group is composed of several sequentially connected TDAB modules and a 3×3 convolutional layer, and the self-attention map A and feature map Y output by each TDAB module will be used for reinforcement learning of each layer of the dictionary;

[0049] The feature connection module uses global residual connection to connect the deep features with the shallow features to obtain the final features;

[0050] The image reconstruction module uses a 3×3 convolutional layer and upsampling to reconstruct the image based on the final features to obtain a high-resolution image.

[0051] The present invention also provides a feature-enhanced cross-window lightweight super-resolution system, comprising:

[0052] The first building block is to construct a self-attention mechanism based on Token dictionary intersection and introduce a Token grouping mechanism to form a dictionary cross self-attention module based on Token grouping, namely the TDAB module.

[0053] The second building module builds a feature-enhanced cross-window lightweight super-resolution network based on the TDAB module;

[0054] The super-resolution module uses the feature-enhanced cross-window lightweight super-resolution network to reconstruct the low-resolution image to be reconstructed to obtain a reconstruction result.

[0055] The present invention also provides an electronic device, comprising:

[0056] one or more processors;

[0057] A storage device for storing one or more programs;

[0058] The one or more programs are executed by the one or more processors, so that the one or more processors implement any of the above methods.

[0059] The present invention has the following beneficial effects:

[0060] The present invention overcomes the problem that the feature information extracted by the original classic super-resolution model is too limited to the window by introducing an attention mechanism and a Token grouping mechanism based on Token dictionary crossover, and provides local, regional and global multi-level feature information; by introducing a Token dictionary to strengthen the prior, the feature extraction accuracy is enhanced, and the model is more lightweight.

[0061] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited to the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A method step diagram of an embodiment of the present invention;

[0063] Figure 2 Schematic diagram of a dictionary cross self-attention module based on Token grouping according to an embodiment of the present invention;

[0064] Figure 3 A schematic diagram of a network framework according to an embodiment of the present invention;

[0065] Figure 4 is a system structure diagram of an embodiment of the present invention;

[0066] Figure 5 The figure is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] See also Figure 1 FIG. 1 is a diagram showing steps of a method according to an embodiment of the present invention, comprising the following steps:

[0068] S101, construct a self-attention mechanism based on Token dictionary intersection, and introduce a Token grouping mechanism to form a dictionary cross self-attention module based on Token grouping, namely the TDAB module;

[0069] S102, constructs a feature-enhanced cross-window lightweight super-resolution network based on the TDAB module;

[0070] S103, reconstructing the low-resolution image to be reconstructed using the feature-enhanced cross-window lightweight super-resolution network to obtain a reconstruction result.

[0071] In order to further explain the technical solution of the present invention, the key points of the present invention are described in detail below through specific embodiments.

[0072] 1. Construct a self-attention mechanism based on Token dictionary intersection.

[0073] The self-attention mechanism based on Token dictionary crossover aims to introduce an additional dictionary D∈R M×d , initialized as network parameters to summarize external priors during the training phase. Use the learned Token dictionary D to generate the key dictionary K D Sum value dictionary V D , and use the input features X∈R N×d Generate query tokens through 1×1 convolution and 3×3 depth convolution:

[0074] K D =DW K , V D =XW V ;

[0075] in, represents a 3×3 depth convolution, W c Q represents a 1×1 standard convolution, W K ∈R d×d , W V ∈R d×d are linear transformations of key dictionary Tokens and value dictionary Tokens, respectively. M<<N is set to keep the computational cost low.

[0076] Use the key dictionary and value dictionary to enhance the query tokens through cross attention calculation to get the final output X of the self-attention out :

[0077] A=SoftMax(Sim cos (Q X ,K D ) / δ);

[0078] TDCA(Q X ,K D ,V D )=A·V D;

[0079] Among them, δ is a learnable parameter used to adjust the range of similarity values; Sim cos (·,·) means calculating the cosine similarity between two tokens, S = Sim cos (Q X ,K D )∈R N×M is the similarity graph between the query token and the key dictionary token. Here, the normalized cosine distance is used instead of the dot product operation in the traditional multi-head self-attention, because it is hoped that each token in the dictionary has an equal chance of being selected, and the similarity magnitude normalization operation is commonly used in previous dictionary learning work. The similarity map S is then converted to the self-attention map A using the SoftMax function for subsequent calculations. The above TDCA operation first selects similar tokens in the key dictionary and obtains the self-attention map A, and then TDCA combines the corresponding tokens in the value dictionary using the similarity values. In this way, TDCA is able to embed external priors into the learned dictionary to enhance the input image features.

[0080] 2. Build a Token grouping mechanism (TC).

[0081] TC aggregates and groups the input tokens to strengthen the correlation between different tokens, and then performs cross-attention operations on the grouped tokens to obtain more representative features.

[0082] 3. The Token grouping mechanism is introduced into the Token dictionary cross-attention mechanism to form a Token grouping-based dictionary cross-attention module (TDAB).

[0083] See also Figure 2 As shown in FIG. 1 , a schematic diagram of a dictionary cross self-attention module based on Token grouping according to an embodiment of the present invention is shown. The internal execution process of this module is as follows:

[0084] First, the proposed TC module is used to transform the input Token X∈R N×d And Token dictionary D∈R M×d As one of the important components of TDAB, how TC divides the input tokens is very critical. For all input tokens, the proposed TC module first divides them evenly into 4 groups according to the number of tokens:

[0085] X i =Split(X),i=0,1,2,3;

[0086] D i=Split(D),i=0,1,2,3;

[0087] in and Represents the i-th group of input X and D. Next, the four groups of X are mapped to the Query (Q) matrix through 3×3 depthwise convolution (DConv) and 1×1 standard convolution, and the four groups of dictionary D are mapped to K through linear mapping. D and V D , then Q, K D 、V D There are also 4 parts, recorded as

[0088]

[0089] K D =DW K ,i=0,1,2,3;

[0090] V D =DW V ,i=0,1,2,3;

[0091] in, represents a 3×3 depth convolution, W c Q represents a 1×1 standard convolution, W K ∈R d×d , W V ∈R d×d They are linear transformations of key dictionary Tokens and value dictionary Tokens respectively.

[0092] Secondly, considering that different sized token groups contain different information and different correlations between token groups, we simply divide the input tokens into the same number of parts (i.e., X i ) is obviously not reasonable enough. In order to explore the association between token groups of different sizes and transcend the window limit, it is also necessary to i and D i Implement more refined grouping. Specifically, for Q i , K Di 、V Di(i=1,2,3), use depthwise separable convolution (DSConv) with different kernel sizes to refine the grouping. Although the correlation between token groups is important, the correlation between token groups and individual tokens, as well as the correlation between individual tokens, cannot be ignored. Therefore, for Q0, K D0 、V D0 , without performing fine grouping, retaining the mode of a single Token, expressed as:

[0093] Q' i =Refine i (Q i ),i=1,2,3;

[0094] K' Di =Refine i (K Di ),i=1,2,3;

[0095] V' Di =Refine i (V Di ),i=1,2,3;

[0096] in Respectively represent Q i , K Di 、V Di The result after refined grouping. i (·) indicates that refined grouping is performed by DSConv and Hardswish activation functions with kernel sizes of 3×3, 5×5, and 7×7, respectively.

[0097] Next, use K D ' and V D 'Enhance Q' by cross-attention calculation (respectively Q' i , K' Di 、V' Di , i = 0, 1, 2, 3), and obtain the final output Y∈R of the module that fully captures the cross-window dependency after being enhanced by the Token dictionary N×d .

[0098] A=SoftMax(Sim cos (Q',K D ') / δ);

[0099] Y=A·V D ';

[0100] Among them, A∈R N×M For Q' and KD 'The obtained self-attention map.

[0101] 4. The dictionary cross-self-attention modules based on token grouping are connected in series to form a residual token grouping attention group, and then the residual token grouping attention groups are connected in series to construct a feature-enhanced cross-window lightweight super-resolution network.

[0102] See also Figure 3 As shown in the figure, the feature-enhanced cross-window lightweight super-resolution network consists of three main modules: Shallow Feature Extraction Module (SFEM), Deep Feature Extraction Module (DFEM) and Image Reconstruction Module (IRM). At the same time, in order to maximize the advantages of shallow and deep features, a global residual connection is introduced to connect shallow features with deep features. In addition, the self-attention map A and feature map Y output by the TDCA of each TDAB will be used for reinforcement learning of each layer of the dictionary, thereby introducing more accurate priors for each layer of TDAB.

[0103] Specifically, the Deep Feature Extraction Module (DFEM) consists of 4 residual group attention groups (RCAGs) and a 3×3 convolutional layer; specifically, RCAG consists of multiple token grouping-based dictionary cross self-attention modules (TDABs) and a 3×3 convolutional layer, which ensures the diversity of deep features.

[0104] See also Figure 4 As shown, it is a system structure diagram of an embodiment of the present invention, including:

[0105] The first construction module 401 constructs a self-attention mechanism based on Token dictionary intersection and introduces a Token grouping mechanism to form a dictionary cross self-attention module based on Token grouping, namely, a TDAB module;

[0106] A second construction module 402 constructs a feature-enhanced cross-window lightweight super-resolution network based on the TDAB module;

[0107] The super-resolution module 403 uses the feature-enhanced cross-window lightweight super-resolution network to reconstruct the low-resolution image to be reconstructed to obtain a reconstruction result.

[0108] See also Figure 5As shown, it is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. The electronic device of this embodiment includes: a processor 501 and a memory 502; wherein the memory 502 is used to store computer-executable instructions; the processor 501 is used to execute the computer-executable instructions stored in the memory to implement the various steps performed by the electronic device in the above-mentioned embodiment. For details, please refer to the relevant description in the aforementioned method embodiment. The memory 502 can be either independent or integrated with the processor 501. When the memory 502 is independently provided, the electronic device also includes a bus 503 for connecting the memory 502 and the processor 501.

[0109] It can be seen that the feature-enhanced cross-window lightweight super-resolution method proposed in the present invention first constructs a self-attention mechanism based on Token dictionary intersection; secondly, constructs a Token grouping mechanism (TC); then introduces the Token grouping mechanism into the self-attention mechanism based on Token dictionary intersection to form a dictionary cross-attention module based on Token grouping (TDAB); finally, the dictionary cross-attention modules based on Token grouping are connected in series to form a residual Token grouping attention group, and then the residual Token grouping attention group is connected in series to construct a feature-enhanced cross-window lightweight super-resolution network. The feature-enhanced cross-window lightweight super-resolution network is used to reconstruct the low-resolution image to be reconstructed to obtain a reconstruction result. On the basis of the classic super-resolution method, this method introduces a self-attention mechanism based on Token dictionary intersection and a hybrid attention module based on Token grouping, which makes the model lighter and enhances the model's ability to capture cross-window dependencies and the reconstruction effect of texture detail features; compared with the original classic single-frame super-resolution method, this method can greatly improve the reconstruction performance of the network model, achieve a high degree of restoration of the texture details of the reconstructed image, and greatly reduce the model calculation amount and parameter amount.

[0110] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A feature-enhanced cross-window lightweight super-resolution method, characterized in that: The following steps are involved: Construct a self-attention mechanism based on Token dictionary intersection and introduce a Token grouping mechanism to form a dictionary cross self-attention module based on Token grouping, namely the TDAB module; Construct a feature-enhanced cross-window lightweight super-resolution network based on the TDAB module; Reconstructing the low-resolution image to be reconstructed using the feature-enhanced cross-window lightweight super-resolution network to obtain a reconstruction result; The internal execution process of the self-attention mechanism based on Token dictionary intersection includes the following steps: Introduce an additional dictionary D∈R M×d , initialize it as the network parameter, M represents the spatial dimension of the dictionary, and d represents the channel dimension of the dictionary; Generate a key dictionary K using dictionary D D Sum value dictionary V D , and use the input features X∈R N×d The query tokens are generated by 1×1 convolution and 3×3 depth convolution, which is expressed as: K D =DW K ; V D =XW V ; Among them, Q X Indicates query Tokens; represents a 3×3 depth convolution, W c Q represents a 1×1 standard convolution, W K ∈R d×d and W V ∈R d×d The key dictionary K D The Tokens and value dictionary V D The linear transformation of Tokens; N represents the number of Tokens, that is, the spatial dimension of the input features; Use the key dictionary and value dictionary to enhance the query Tokens through cross-attention calculation and get the final output X out , expressed as: A=SoftMax(Sim cos (Q X ,K D ) / d); X out =TDCA(Q X ,K D ,V D )=A·V D ; Among them, δ is a learnable parameter used to adjust the range of similarity values; Sim cos (·,·) indicates the calculation of cosine similarity, and A indicates the self-attention map; The internal execution process of the dictionary cross self-attention module based on token grouping includes the following steps: Use the Token grouping mechanism to evenly group the input Token X and Token dictionary D and map the grouping results separately; Depthwise separable convolutions with different kernel sizes are used to finely group the mapping results; The cross-attention calculation is used to enhance the result of fine grouping to obtain the final output; The method of using the Token grouping mechanism to evenly group the input Token X and the Token dictionary D and mapping the grouping results respectively includes the following steps: For the input Token X∈R N×d And Token dictionary D∈R M×d Grouping is performed, expressed as: X i =Split(X),i=0,1,2,3; D i =Split(D),i=0,1,2,3; Among them, Split represents the grouping operation. represents the i-th group after Token X is grouped. Represents the i-th group after the dictionary D is grouped; X is transformed into i Mapped to query TokensQ i , through linear mapping, the four groups of dictionary D are mapped to key dictionaries K Di Sum value dictionary V Di , expressed as: K Di =D i W K ,i=0,1,2,3; V Di =D i W V ,i=0,1,2,3。 2. The feature-enhanced cross-window lightweight super-resolution method according to claim 1, characterized in that: Set M<<N to keep the computational cost low.

3. The feature-enhanced cross-window lightweight super-resolution method according to claim 1, characterized in that: The method of finely grouping the mapping results by using depthwise separable convolutions with different kernel sizes includes the following steps: When i=0, no fine grouping is performed, and Q0, K D0 and V D0 Keep its single token mode; When i=1,2,3, for Q i , K Di and V Di Perform refined grouping, expressed as: Q' i =Refine i (Q i ),i=1,2,3; K' Di =Refine i (K Di ),i=1,2,3; V' Di =Refine i (V Di ),i=1,2,3; in, and Respectively represent Q i , K Di 、V Di The result after refined grouping; Refine i (·) denotes a refined grouping operation, including three DSConv with kernel sizes of 3×3, 5×5, and 7×7 and a Hardswish activation function.

4. The feature-enhanced cross-window lightweight super-resolution method according to claim 3, characterized in that: The method of enhancing the result of fine grouping by using cross attention calculation includes the following steps: Using K' Di and V' Di Enhancing Q' by cross-attention computation i , the final output of the dictionary cross self-attention module based on Token grouping is obtained, which is expressed as: A=SoftMax(Sim cos (Q',K D ') / d); Y=A·V D ′; Among them, Y∈R N×d Represents the final output.

5. The feature-enhanced cross-window lightweight super-resolution method according to claim 1, characterized in that: The feature-enhanced cross-window lightweight super-resolution network includes: The shallow feature extraction module uses a 3×3 convolutional layer to perform shallow feature extraction on the input low-resolution image; A deep feature extraction module uses four sequentially connected residual grouping attention groups and a 3×3 convolutional layer to perform deep feature extraction on shallow feature extraction; the residual grouping attention group is composed of several sequentially connected TDAB modules and a 3×3 convolutional layer, and the self-attention map A and the final output Y output by each TDAB module will be used for reinforcement learning of each layer of the dictionary; The feature connection module uses global residual connection to connect the deep features with the shallow features to obtain the final features; The image reconstruction module uses a 3×3 convolutional layer and upsampling to reconstruct the image based on the final features to obtain a high-resolution image.

6. A feature-enhanced cross-window lightweight super-resolution system, characterized in that: include: The first building block is to construct a self-attention mechanism based on Token dictionary intersection and introduce a Token grouping mechanism to form a dictionary cross self-attention module based on Token grouping, namely the TDAB module. The second building module builds a feature-enhanced cross-window lightweight super-resolution network based on the TDAB module; A super-resolution module, which uses the feature-enhanced cross-window lightweight super-resolution network to reconstruct the low-resolution image to be reconstructed to obtain a reconstruction result; The internal execution process of the self-attention mechanism based on Token dictionary intersection includes the following steps: Introduce an additional dictionary D∈R M×d , initialize it as the network parameter, M represents the spatial dimension of the dictionary, and d represents the channel dimension of the dictionary; Generate a key dictionary K using dictionary D D Sum value dictionary V D , and use the input features X∈R N×d The query tokens are generated by 1×1 convolution and 3×3 depth convolution, which is expressed as: K D =DW K ; V D =XW V ; Among them, Q X Indicates query Tokens; represents a 3×3 depth convolution, W c Q represents a 1×1 standard convolution, W K ∈R d×d and W V ∈R d×d The key dictionary K D The Tokens and value dictionary V D The linear transformation of Tokens; N represents the number of Tokens, that is, the spatial dimension of the input features; Use the key dictionary and value dictionary to enhance the query Tokens through cross-attention calculation and get the final output X out , expressed as: A=SoftMax(Sim cos (Q X ,K D ) / d); X out =TDCA(Q X ,K D ,V D )=A·V D ; Among them, δ is a learnable parameter used to adjust the range of similarity values; Sim cos (·,·) indicates the calculation of cosine similarity, and A indicates the self-attention map; The internal execution process of the dictionary cross self-attention module based on token grouping includes the following steps: Use the Token grouping mechanism to evenly group the input Token X and Token dictionary D and map the grouping results separately; Depthwise separable convolutions with different kernel sizes are used to finely group the mapping results; The cross-attention calculation is used to enhance the result of fine grouping to obtain the final output; The method of using the Token grouping mechanism to evenly group the input Token X and the Token dictionary D and mapping the grouping results respectively includes the following steps: For the input Token X∈R N×d And Token dictionary D∈R M×d Grouping is performed, expressed as: X i =Split(X),i=0,1,2,3; D i =Split(D),i=0,1,2,3; Among them, Split represents the grouping operation. represents the i-th group after Token X is grouped. Represents the i-th group after the dictionary D is grouped; X is transformed into i Mapped to query TokensQ i , through linear mapping, the four groups of dictionary D are mapped to key dictionaries K Di Sum value dictionary V Di , expressed as: K Di =D i W K ,i=0,1,2,3; V Di =D i W V ,i=0,1,2,3。 7. An electronic device comprising: one or more processors; A storage device for storing one or more programs; The one or more programs are executed by the one or more processors, so that the one or more processors implement the method according to any one of claims 1 to 5.

Images