A method, system, device and medium for implementing a global attention mechanism

Abstract

The present invention relates to the field of data processing. The present invention proposes a method, system, device and medium for realizing a global attention mechanism. The method includes: processing the data to be processed to obtain a feature map; calculating the self-attention of the feature map through a preset attention model to obtain a self-attention feature map, and performing global receptive field capture based on the self-attention feature map to obtain an expanded feature map; integrating the self-attention feature map and the expanded feature map to obtain an intermediate feature map; using the intermediate feature map as a new feature map, returning to the step of calculating the self-attention of the feature map until a preset condition is met, and outputting a global feature map. The present invention controls the size of the global receptive field by adjusting the expansion rate, extracts a wider range of image features without increasing the computational complexity, avoids grid problems, and balances model performance and computational efficiency.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method, system, device and medium for implementing a global attention mechanism. Background Art

[0002] At present, in order to achieve a global receptive field, VGGNet proposed that the effect of expanding the receptive field can be achieved by stacking convolutional layers; Ding et al. proposed a method based on stacking convolutional layers to expand the receptive field. The influence of each pixel on the neuron response depends on their distance to the neuron: this is because, for When the distance between the pixel and the neuron is d, ignoring the bias term and activation function, its impact on the neuron output is ,in is the shortest path from the neuron to this pixel. The absolute value of these weights is usually less than So after The influence of pixels on neurons is greatly reduced.

[0003] Based on this, Ding et al. further proposed With the use of super-large convolution kernels, each neuron in the model can obtain a larger receptive field; at the same time, since all pixels covered by the convolution kernel are synchronously perceived by the neurons without going through more convolution layers, the influence of the pixels covered by the convolution kernel on the neurons is not limited by distance. However, compared with normal convolution kernels, the number of parameters is greatly increased, and more complex training techniques are required. In addition, the self-attention mechanism is also a commonly used method to obtain the global receptive field. However, the self-attention mechanism is limited by its The computational limitation and its low rank make it have the characteristics of large computational complexity and high computational redundancy.

[0004] It can be seen that in the prior art, in order to achieve the global receptive field, it is usually achieved through the global self-attention mechanism, and this calculation method has a large computational cost. In contrast, although the convolution operation has a lower computational cost, it needs to be expanded through a large number of convolutional layers to achieve the global receptive field, which is also expensive. The dilated convolution provides a means to quickly expand the receptive field of the convolution layer, but it is affected by the grid problem and there are a lot of gaps, making it difficult to achieve global coverage of all pixels, resulting in incomplete information and reduced performance. Summary of the invention

[0005] In order to solve one or more of the above-mentioned technical problems, the present invention proposes a method for implementing a global attention mechanism, including:

[0006] Processing the data to be processed to obtain a feature map;

[0007] The self-attention of the feature map is calculated by a preset attention model to obtain a self-attention feature map, and a global receptive field is captured based on the self-attention feature map to obtain an expanded feature map;

[0008] Integrating the self-attention feature map and the expansion feature map to obtain an intermediate feature map;

[0009] The intermediate feature map is used as a new feature map, and the step of calculating the self-attention of the feature map is returned until a preset condition is met, and a global feature map is output.

[0010] In some embodiments, the training process of the preset attention model is as follows:

[0011] Obtaining training data, downsampling the training data, and calculating the size of the input feature map;

[0012] Build an expansion self-attention module based on the size of the input feature map, and attach each expansion self-attention module to the corresponding convolution module;

[0013] Setting image parameters, modifying the training data to the image parameter size by interpolation, processing the training data by a convolution module and an expansion self-attention module, and obtaining an output result;

[0014] The output results are used as new training data to restart training until the preset number of times is reached to obtain the attention model.

[0015] In some embodiments, the step of constructing an expanded self-attention module based on the size of the input feature map includes:

[0016] Calculate the number of layers and dilation rate based on the size of the input feature map;

[0017] Stacking the hole self-attention layers corresponding to the number of layers and the expansion rate to obtain an expansion self-attention module;

[0018] Initialize the last layer of the dilated self-attention module to 0.

[0019] In some embodiments, the step of processing the training data through a convolution module and an expansion self-attention module to obtain an output result includes:

[0020] The training data enters the convolution module for processing to obtain a convolution result;

[0021] Selecting a local convolution result based on the connection relationship with the dilated self-attention module and inputting it into the dilated self-attention module to obtain a capture result;

[0022] The convolution result and the capture result are summed and then downsampled;

[0023] The downsampled data is used as the new training data and the next convolution module is used as the new convolution module. The training data is returned to the step of entering the convolution module for processing to obtain the convolution result, until the last convolution module is entered to obtain the output result.

[0024] In some embodiments, the step of calculating the self-attention of the feature map by a preset attention model to obtain a self-attention feature map, and performing global receptive field capture based on the self-attention feature map to obtain an expanded feature map includes:

[0025] Inputting the feature map into a convolution module in a preset attention model, calculating the self-attention of the feature map through a self-attention mechanism in the convolution module, and obtaining a self-attention feature map;

[0026] Select the layer connected to the dilated self-attention module in the convolution module as the preset selection condition;

[0027] The features of the self-attention feature map corresponding to the preset selection conditions are input into the dilated self-attention module to capture the global receptive field. After traversing all layers of the corresponding dilated self-attention module, the dilated feature map is obtained.

[0028] In some embodiments, the step of taking the intermediate feature map as a new feature map and returning to the step of calculating the self-attention of the feature map until a preset condition is met, and the step of outputting the global feature map includes:

[0029] Taking the intermediate feature map as a new feature map, returning to the step of calculating the self-attention of the feature map until a preset number of layers of the attention model is traversed;

[0030] Output the intermediate feature map of the last layer and use it as the global feature map.

[0031] In some embodiments, the step of processing the data to be processed to obtain a feature map includes:

[0032] Acquiring data to be processed, wherein the data to be processed includes image data and non-image data;

[0033] In response to the data to be processed being image data, extracting features to obtain a feature map;

[0034] In response to the data to be processed being non-image data, Fourier transform is performed to obtain a frequency spectrum, and features are extracted from the frequency spectrum to obtain a feature spectrum.

[0035] The present invention proposes a system for implementing a global attention mechanism, comprising:

[0036] An acquisition unit, configured to process the data to be processed to obtain a feature map;

[0037] An expansion unit is configured to calculate the self-attention of the feature map through a preset attention model to obtain a self-attention feature map, and perform global receptive field capture based on the self-attention feature map to obtain an expanded feature map;

[0038] An intermediate unit configured to integrate the self-attention feature map and the expansion feature map to obtain an intermediate feature map;

[0039] The output unit is configured to use the intermediate feature map as a new feature map, return to the step of calculating the self-attention of the feature map, and output the global feature map until a preset condition is met.

[0040] The present invention provides a computer device, comprising:

[0041] At least one processor; and a memory storing a computer program executable on the processor, wherein the processor executes the steps of the method for implementing a global attention mechanism when executing the program.

[0042] The present invention proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the method for implementing a global attention mechanism are performed.

[0043] The present invention has at least the following beneficial technical effects:

[0044] The present invention proposes a method, system, device and medium for realizing a global attention mechanism, the method comprising: processing the data to be processed to obtain a feature map; calculating the self-attention of the feature map through a preset attention model to obtain a self-attention feature map, and performing global receptive field capture based on the self-attention feature map to obtain an expanded feature map; integrating the self-attention feature map and the expanded feature map to obtain an intermediate feature map; using the intermediate feature map as a new feature map, returning to the step of calculating the self-attention of the feature map, until a preset condition is met, and outputting a global feature map.

[0045] The present invention controls the size of the global receptive field by adjusting the expansion rate. Without increasing the computational complexity, it can extract image features in a wider range, improve the ability to understand and recognize data, and have stronger robustness to interference factors such as noise and occlusion. While avoiding grid problems, it can achieve the global receptive field through fewer convolutional layers, thus balancing model performance and computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A flow chart of a method for implementing a global attention mechanism provided by the present invention;

[0048] Figure 2 A system module diagram for implementing a global attention mechanism provided by the present invention;

[0049] Figure 3 A dilated convolutional layer diagram with different dilation rates of an embodiment of a method for implementing a global attention mechanism provided by the present invention;

[0050] Figure 4 An expanded self-attention module diagram of an embodiment of a method for implementing a global attention mechanism provided by the present invention;

[0051] Figure 5 An attention model diagram of an embodiment of a method for implementing a global attention mechanism provided by the present invention;

[0052] Figure 6 A heat map of different models for visualization of an embodiment of a method for implementing a global attention mechanism provided by the present invention;

[0053] Figure 7 A schematic diagram of the structure of an embodiment of a computer device provided by the present invention;

[0054] Figure 8 A schematic diagram of the structure of an embodiment of a computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0055] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0056] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. The subsequent embodiments will not explain this one by one.

[0057] This invention proposes a method for implementing a global attention mechanism, please refer to Figure 1 ,include:

[0058] S1: Process the data to be processed to obtain a feature map;

[0059] S2: Calculate the self-attention of the feature map through a preset attention model to obtain a self-attention feature map, and perform global receptive field capture based on the self-attention feature map to obtain an expanded feature map;

[0060] S3: Integrate the self-attention feature map and the expansion feature map to obtain an intermediate feature map;

[0061] S4: Take the intermediate feature map as a new feature map, return to the step of calculating the self-attention of the feature map, until a preset condition is met, and output a global feature map.

[0062] When faced with complex and ever-changing computer vision tasks, such as scene understanding and dynamic target tracking, the method of the present invention can more effectively handle challenges such as noise, occlusion, and deformation in the input data. The self-attention mechanism helps the model ignore irrelevant or misleading information, while the dilated convolution ensures that even in the absence of information, the model can make accurate judgments by capturing a wider range of contexts.

[0063] In some embodiments, see Figure 1 and Figure 5 , the training process of the preset attention model is as follows:

[0064] Obtaining training data, downsampling the training data, and calculating the size of the input feature map;

[0065] Build an expansion self-attention module based on the size of the input feature map, and attach each expansion self-attention module to the corresponding convolution module;

[0066] Setting image parameters, modifying the training data to the image parameter size by interpolation, processing the training data by a convolution module and an expansion self-attention module, and obtaining an output result;

[0067] The output results are used as new training data to restart training until the preset number of times is reached to obtain the attention model.

[0068] The present invention first implements the self-attention mechanism, so that the model can deeply understand the global dependencies in the input data, that is, the mutual influence between different regions. This global vision provides a strong contextual information foundation for subsequent processing.

[0069] The subsequent dilated convolution operation, while maintaining the resolution of the feature map, effectively increases the receptive field of each pixel by introducing convolution kernels with different expansion rates, allowing the model to capture local details and contextual information in a wider range. This combination not only retains the global insight brought by self-attention, but also enhances the ability to extract local information. At the same time, it avoids the grid problem. By combining the self-attention mechanism and dilated convolution, it is possible to capture global contextual information while retaining sensitivity to detail information. The self-attention mechanism enhances the model's ability to model complex data relationships, while the dilated convolution improves the model's ability to capture a wide range of contextual information by increasing the receptive field.

[0070] This method does not depend on the specific model structure. The output of the backbone model will enter the MDM and capture the global receptive field through the MDM. The feature map after global receptive field capture will be integrated with the original feature map. The attention model is as follows Figure 5 shown.

[0071] The receptive field is defined as the area that affects the firing of a neuron. In convolutional neural networks, the receptive field can be similarly defined for each neuron. For convenience, since the convolution kernels in convolutional neural networks are all square, this paper assumes that the receptive field of any neuron is square. Assume that for the input image or feature map (where H, W, and C represent its height, width, and number of channels, respectively) neuron The output is

[0072] .

[0073] For neurons In terms of

[0074] .

[0075] The receptive field of can be defined as:

[0076]

[0077] ;

[0078] and That is The blind spot is denoted as .

[0079] For example, for the convolutional layer, rewrite the convolution as follows:

[0080]

[0081] It can be seen that if and only if , the weight matrix is ​​not always 0.

[0082] The receptive field size of the convolutional layer is defined as '' means that the receptive field size of each neuron in the convolutional layer is .

[0083] Ignoring the activation function and bias term, for a series of convolutional layers, if all its output neurons are The blind spot set in the receptive field of is an empty set, then this series of convolutional layers can be replaced by a convolution kernel size of And the expansion rate is The convolutional layer is replaced by .

[0084] Proof: To facilitate the proof, we need to rewrite the form of convolution. First, write the convolution parameter matrix as follows:

[0085]

[0086] in is the weight vector, where represents the pixel index of the convolution kernel center, represents the index of the output channel, so express The feature pair pixels at The output feature Contribution in each dimension;

[0087] If and only if . Therefore, the convolution operation can be written as the formula:

[0088]

[0089] in Represents input, is a -dimensional column vector.

[0090] So, after The output after the convolutional layer

[0091] ,

[0092] Indicates that Rearrange to the form in the previous formula. Let ,because The convolutional layers are stacked, and the receptive field size is , and there is no blind spot in the receptive field, so for any Vector in , if and only if or , .then, The stacked convolutional layers can be represented by a weight The convolution layer is replaced by . In this case, the convolution kernel side length of this convolution layer is , expansion rate .

[0093] Furthermore, for continuous dilated convolutional layers, assuming their respective dilation rates are , and satisfies , then:

[0094] ;

[0095] in Represents the radius of the receptive field of the last hole convolution layer on the input. It is worth noting that the input here refers to this The input of the module consists of three atrous convolutional layers, rather than the input of the entire model.

[0096] First consider two consecutive, indexed The hole convolution layer and input And the corresponding expansion rate .for Any neuron in the layer , its output is:

[0097]

[0098] make ,thereby

[0099]

[0100] then right The partial derivative of

[0101] ;

[0102] thereby

[0103]

[0104] so ,therefore .

[0105] Assumptions And the first Any neuron in the layer The receptive field radius on the input feature map satisfies . At this time, we have the formula:

[0106]

[0107] Without loss of generality, let .when ,have:

[0108]

[0109] So for any two pixels , ,have , if and only if and Similarly, If and only if and Take the equal sign when . Thus ,Right now

[0110] According to previous work, when r_1=1 and there are only three dilated convolutional layers, the dilated convolutional layers that satisfy the formula \ref{equ:hdc} will not have a grid problem: .

[0111] in is the size of the convolution kernel of the first convolutional layer. For a dilation rate , the size of the convolution kernel is For the convolutional layer, the receptive field length of each neuron is .

[0112] According to the above, for The dilation rates of the atrous convolution layers are , , ,have Heng established, and before There is no grid problem for the atrous convolutional layer. When the dilation rate satisfies:

[0113]

[0114] There is no grid problem for consecutive n atrous convolutional layers.

[0115] First, when n=3, it degenerates into the formula \ref{equ:hdc}, and the conclusion is obviously valid.

[0116] When n>3, according to Theorems 1 and 2, The atrous convolutional layer can be viewed as a , the expansion rate is The convolutional layer of The atrous convolutional layers can be replaced by a , the expansion rate is The convolutional layer and the other two dilation rates are and The dilated convolutional layer.

[0117] at this time, satisfy

[0118]

[0119] This is equivalent to the three expansion rates being The convolutional layer satisfies the formula

[0120] .

[0121] The method proposed in this invention has a global receptive field. First, rewrite the construction method as follows:

[0122]

[0123] It is easy to get the receptive field radius of the first n convolutional layers is

[0124]

[0125] because , .

[0126] Therefore, except for the extreme edges on each side, no more than Except for the row (column) neurons, the receptive fields of other neurons can cover all pixels. If you want to ensure that all pixels have a global receptive field, you only need to add a dilation rate of The dilated convolutional layer of the previous layer is , and so on. Assume first (Except In addition to the layer, at least three convolutional layers have been stacked. ), obviously

[0127]

[0128] Therefore, this series of dilated convolutional layers does not have a grid problem. , so there is ,Right now , the newly added layers will not have mesh problems.

[0129] Although the above proof is based on convolutional layers, in fact, the dilated self-attention layer can be expressed as

[0130] ;

[0131] in It can be calculated through the self-attention mechanism. It can be seen that in form, this formula is consistent with the convolution operation, so the above proof can be directly extended to the expansion self-attention layer.

[0132] In some embodiments, see Figure 1 , Figure 3 and Figure 4 , the step of constructing an expanded self-attention module based on the size of the input feature map includes:

[0133] Calculate the number of layers and dilation rate based on the size of the input feature map;

[0134] Stacking the dilated convolutional layers corresponding to the number of layers and the dilation rate to obtain a dilated self-attention module;

[0135] Initialize the last layer of the dilated self-attention module to 0.

[0136] The dilated convolutional layer is Figure 3 As shown, the black block represents the pixels covered by the center of the convolution kernel; the gray block represents the other pixels covered by the convolution kernel. Represents the dilation rate of the dilated convolution. When , it is a general convolution operation; when When , the area covered by the convolution kernel is no longer adjacent, but is generated in the horizontal or vertical (or both) direction with the center of the convolution kernel. Specifically, the dilated convolution is written as follows:

[0137]

[0138] in represents the weight matrix. Obviously, when , a single hole convolution layer cannot completely cover an area (that is, there are blind spots in the receptive field). In order to solve the above problem and balance the number of hole convolution layers (computational complexity) and the global receptive quality, assuming that the input feature map size is , and order , the present invention proposes the following hole convolution layer stacking method:

[0139]

[0140] in represents the expansion rate of the i-th atrous convolutional layer, Represents the number of stacked layers.

[0141] The present invention further reduces the amount of computation required for the convolution operation through depth-wise convolution:

[0142]

[0143] in represents the weight vector of the corresponding position, is the weight matrix, are all bias terms, Represents vector element multiplication. The activation function and batch normalization are ignored in the formula.

[0144] Each stack of dilated convolutional layers is called a dilated self-attention module (MDM). Figure 4 , attached to the backbone network to expand its receptive field.

[0145] Taking the ResNet-50 model structure as an example, MDM is added only in the S2 and subsequent model layers. The specific configuration is shown in Table 1 below:

[0146] Table 1

[0147]

[0148] At the same time, initializing the last layer to 0 can avoid the problem of excessive weight updates and gradient explosion after continuous expansion. At the same time, it is convenient to pay attention to the changes and flow of data during training, ensure that the data learns useful features, and accelerate the convergence process of the model. Dilated self-attention can exponentially increase the receptive field of the convolution layer without increasing the number of convolution kernel parameters by introducing holes (i.e. zero padding). This means that the model can capture a wider range of contextual information, which is particularly important for tasks such as image segmentation and object detection.

[0149] In some embodiments, see Figure 1 and Figure 5 The step of processing the training data through a convolution module and an expansion self-attention module to obtain an output result includes:

[0150] The training data enters the convolution module for processing to obtain a convolution result;

[0151] Selecting a local convolution result based on the connection relationship with the dilated self-attention module and inputting it into the dilated self-attention module to obtain a capture result;

[0152] The convolution result and the capture result are summed and then downsampled;

[0153] The downsampled data is used as the new training data and the next convolution module is used as the new convolution module. The training data is returned to the step of entering the convolution module for processing to obtain the convolution result, until entering the last convolution module to obtain the output result.

[0154] Since the backbone model is usually a pre-trained model, in order to efficiently complete the module addition, is the output feature of MDM, is the output feature of the corresponding convolution module in the backbone network, and the input of the next convolution module is:

[0155] ;

[0156] Compared with using a single fully connected layer or deep convolutional network with high computational cost to increase the receptive field, the present invention increases the receptive field while maintaining the size of the feature map through dilated convolution, significantly reducing computational complexity and memory consumption. At the same time, the parallel processing characteristics of the self-attention mechanism also accelerate the training and reasoning speed of the model.

[0157] In some embodiments, see Figure 1 , Figure 4 and Figure 5 The step of calculating the self-attention of the feature map by a preset attention model to obtain a self-attention feature map, and performing global receptive field capture based on the self-attention feature map to obtain an expanded feature map comprises:

[0158] Inputting the feature map into a convolution module in a preset attention model, calculating the self-attention of the feature map through a self-attention mechanism in the convolution module, and obtaining a self-attention feature map;

[0159] Select the layer connected to the dilated self-attention module in the convolution module as the preset selection condition;

[0160] The features of the self-attention feature map corresponding to the preset selection conditions are input into the dilated self-attention module to capture the global receptive field. After traversing all layers of the corresponding dilated self-attention module, the dilated feature map is obtained.

[0161] While expanding the receptive field, dilated self-attention does not reduce the resolution of the output feature map. This is especially important for tasks that need to maintain detailed information (such as semantic segmentation). By presetting the layers connected to the dilated self-attention module, it is possible to ensure that these layers maintain high resolution when processing features, thereby retaining more detailed information and improving the refinement and accuracy of the model. When multiple identical dilated self-attention layers are superimposed, there are a large number of unused pixels (holes) in the receptive field, resulting in a loss of continuity and integrity between the data. Different dilation factors or a mixture of convolution kernels of different sizes are used to reduce this occurrence and improve the model's feature extraction and generalization capabilities.

[0162] In some embodiments, see Figure 1 , Figure 4 and Figure 5 , the step of taking the intermediate feature map as a new feature map and returning to the step of calculating the self-attention of the feature map until a preset condition is met, and the step of outputting the global feature map includes:

[0163] Taking the intermediate feature map as a new feature map, returning to the step of calculating the self-attention of the feature map until the preset number of layers of the attention model is traversed (wherein the preset condition is that the preset number of layers of the attention model is traversed);

[0164] Output the intermediate feature map of the last layer and use it as the global feature map.

[0165] Through the self-attention mechanism, the model can learn richer feature representations that not only contain local information but also integrate global information. This enhanced feature representation capability helps improve the recognition accuracy and generalization ability of the model. Although the self-attention mechanism is relatively complex in computation, by controlling the number and scope of its application layers through preset conditions, it can optimize the utilization of computing resources and improve computing efficiency while ensuring model performance.

[0166] In some embodiments, see Figure 1 The step of processing the data to be processed to obtain a characteristic spectrum includes:

[0167] Acquiring data to be processed, wherein the data to be processed includes image data and non-image data;

[0168] In response to the data to be processed being image data, extracting features to obtain a feature map;

[0169] In response to the data to be processed being non-image data, Fourier transform is performed to obtain a frequency spectrum, and features are extracted from the frequency spectrum to obtain a feature spectrum.

[0170] The conversion process from the spectrum graph to the feature graph is to convert the signal from the time domain to the frequency domain using methods such as Fourier transform (FFT) to obtain the spectrum graph.

[0171] The spectrum diagram shows the energy distribution of the signal at different frequencies. Key features are extracted from the spectrum diagram, including frequency components, energy distribution, and spectrum shape. In audio processing, features such as pitch, timbre, and rhythm are extracted; in mechanical fault diagnosis, the vibration energy at a specific frequency is extracted. The extracted features are encoded and converted into a more compact and easier to process form. Coding methods include vector quantization, hash coding, sparse coding, etc. The encoded feature set can be regarded as a feature spectrum. This spectrum represents the key information of the original signal in a more abstract way. During the entire conversion process, it is necessary to pay attention to maintaining the integrity and accuracy of the information and avoid introducing too much noise or errors in the preprocessing, feature extraction, and encoding steps.

[0172] This invention proposes a system that implements a global attention mechanism, please refer to Figure 2 ,include:

[0173] An acquisition unit 100 is configured to process the data to be processed to obtain a characteristic spectrum;

[0174] The expansion unit 200 is configured to calculate the self-attention of the feature map through a preset attention model to obtain a self-attention feature map, and perform global receptive field capture based on the self-attention feature map to obtain an expanded feature map;

[0175] An intermediate unit 300 is configured to integrate the self-attention feature map and the expansion feature map to obtain an intermediate feature map;

[0176] The output unit 400 is configured to use the intermediate feature map as a new feature map, return to the step of calculating the self-attention of the feature map, and output a global feature map until a preset condition is met.

[0177] The present invention controls the size of the global receptive field by adjusting the expansion rate. Without increasing the computational complexity, it can extract image features in a wider range, improve the ability to understand and recognize data, and have stronger robustness to interference factors such as noise and occlusion. While avoiding grid problems, it can achieve the global receptive field through fewer convolutional layers, thus balancing model performance and computational efficiency.

[0178] In some embodiments, see Figure 1 and Figure 6 , for performance comparison between different models and the present invention;

[0179] As shown in Table 2 below, the accuracy of general convolution is 87%, the accuracy of Mobilevit is 88.1%, and the accuracy of the present invention is 88.7%, which shows that the present invention performs well in image classification tasks and has a high accuracy.

[0180] Table 2

[0181]

[0182] And in Figure 6 In the figure, the heat map of general convolution is (a), the heat map of Mobilevit is (b), and the heat map of the present invention is (c). The heat map can show the response strength and high prediction confidence of the present invention to different parts of the input data.

[0183] Based on the same inventive concept, according to another aspect of the present invention, Figure 7 As shown, an embodiment of the present invention further provides a computer device 30, which includes a processor 310 and a memory 320. The memory 320 stores a computer program 321 that can be run on the processor. When the processor 310 executes the program, the steps of the above method are performed.

[0184] Based on the same inventive concept, according to another aspect of the present invention, Figure 8 As shown, an embodiment of the present invention further provides a computer-readable storage medium 40, which stores a computer program 410 for executing the above method when executed by a processor.

[0185] The embodiment of the present invention may also include a corresponding computer device. The computer device includes a memory, at least one processor, and a computer program stored in the memory and executable on the processor, and the processor executes any one of the above methods when executing the program.

[0186] The memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules in the embodiments of the present application. The processor executes various functional applications and data processing of the device by running the non-volatile software programs, instructions and modules stored in the memory, that is, implementing the above method.

[0187] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In an embodiment, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the local module via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0188] Finally, it should be noted that a person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium of the program can be a disk, an optical disk, a read-only storage memory (ROM) or a random access memory (RAM), etc. The above-mentioned computer program embodiments can achieve the same or similar effects as the corresponding above-mentioned arbitrary method embodiments.

[0189] It will also be appreciated by those skilled in the art that various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given to the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.

[0190] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention. The functions, steps and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.

[0191] It should be understood that, as used herein, the singular forms "a", "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations including one or more of the associated listed items.

[0192] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention is limited to these examples; under the idea of ​​the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A method for implementing a global attention mechanism, characterized in that: include: Processing the data to be processed to obtain a feature map; The self-attention of the feature map is calculated by a preset attention model to obtain a self-attention feature map, and a global receptive field is captured based on the self-attention feature map to obtain an expanded feature map; Integrating the self-attention feature map and the expansion feature map to obtain an intermediate feature map; Taking the intermediate feature map as a new feature map, returning to the step of calculating the self-attention of the feature map until a preset condition is met, and outputting a global feature map; The method also includes that the training process of the preset attention model is as follows: Obtain training data, downsample the training data, and calculate the size of the input feature map; An expansion self-attention module is constructed based on the size of the input feature map, and the number of layers and the expansion rate are calculated based on the size of the input feature map; the hole self-attention layers corresponding to the number of layers and the expansion rate are stacked to obtain an expansion self-attention module. The construction formula of the expansion self-attention module is as follows: ； Where r_i (i=1,2,…,S) represents the expansion rate of the i-th atrous convolutional layer, S represents the number of stacked layers, L=max(H,W), the input feature map size is H×W (H≥2, W≥2), and n is the total number of convolutional layers; the last layer of the dilated self-attention module is initialized to 0, and each dilated self-attention module is attached to the corresponding convolutional module; Setting image parameters, modifying the training data to the image parameter size by interpolation, processing the training data by a convolution module and an expansion self-attention module, and obtaining an output result; The output results are used as new training data to restart training until the preset number of times is reached to obtain the attention model.

2. A method for implementing a global attention mechanism according to claim 1, characterized in that: The step of processing the training data through a convolution module and an expansion self-attention module to obtain an output result comprises: The training data enters the convolution module for processing to obtain a convolution result; Selecting a local convolution result based on the connection relationship with the dilated self-attention module and inputting it into the dilated self-attention module to obtain a capture result; The convolution result and the capture result are summed and then downsampled; The downsampled data is used as new training data and the next convolution module is used as a new convolution module, and the training data is returned to the step of entering the convolution module for processing to obtain the convolution result, until the last convolution module is entered to obtain the output result.

3. The method for implementing a global attention mechanism according to claim 1, characterized in that: The step of calculating the self-attention of the feature map by a preset attention model to obtain a self-attention feature map, and performing global receptive field capture based on the self-attention feature map to obtain an expanded feature map comprises: Inputting the feature map into a convolution module in a preset attention model, calculating the self-attention of the feature map through a self-attention mechanism in the convolution module, and obtaining a self-attention feature map; Select the layer connected to the dilated self-attention module in the convolution module as the preset selection condition; The features of the self-attention feature map corresponding to the preset selection conditions are input into the dilated self-attention module to capture the global receptive field. After traversing all layers of the corresponding dilated self-attention module, the dilated feature map is obtained.

4. The method for implementing a global attention mechanism according to claim 1, characterized in that: The step of taking the intermediate feature map as a new feature map and returning to the step of calculating the self-attention of the feature map until a preset condition is met, and the step of outputting the global feature map comprises: Taking the intermediate feature map as a new feature map, returning to the step of calculating the self-attention of the feature map until a preset number of layers of the attention model is traversed; Output the intermediate feature map of the last layer and use it as the global feature map.

5. The method for implementing a global attention mechanism according to claim 1, characterized in that: The step of processing the data to be processed to obtain a characteristic spectrum comprises: Acquiring data to be processed, wherein the data to be processed includes image data and non-image data; In response to the data to be processed being image data, extracting features to obtain a feature map; In response to the data to be processed being non-image data, Fourier transform is performed to obtain a frequency spectrum, and features are extracted from the frequency spectrum to obtain a feature spectrum.

6. A system implementing a global attention mechanism, characterized in that: include: An acquisition unit, configured to process the data to be processed to obtain a feature map; An expansion unit is configured to calculate the self-attention of the feature map through a preset attention model to obtain a self-attention feature map, and perform global receptive field capture based on the self-attention feature map to obtain an expanded feature map; An intermediate unit configured to integrate the self-attention feature map and the expansion feature map to obtain an intermediate feature map; an output unit, configured to use the intermediate feature map as a new feature map, return to the step of calculating the self-attention of the feature map, until a preset condition is met, and output a global feature map; A training unit is configured to train the preset attention model, and the training process of the preset attention model is as follows: Acquire training data, downsample the training data, and calculate the size of the input feature map; construct an expansion self-attention module based on the size of the input feature map, and calculate the number of layers and the expansion rate based on the size of the input feature map; stack the hole self-attention layers corresponding to the number of layers and the expansion rate to obtain an expansion self-attention module, and the construction formula of the expansion self-attention module is as follows: ； Wherein, r_i (i=1,2,…,S) represents the expansion rate of the i-th hole convolution layer, S represents the number of stacked layers, L=max(H,W), the input feature map size is H×W (H≥2, W≥2), and n is the total number of convolution layers; the last layer of the dilated self-attention module is initialized to 0, and each dilated self-attention module is attached to the corresponding convolution module; the image parameters are set, the training data is modified to the image parameter size by interpolation, the training data is processed by the convolution module and the dilated self-attention module to obtain the output result; the output result is used as the new training data to restart the training until the preset number of times is reached to obtain the attention model.

7. A computer device comprising: at least one processor; and a memory storing a computer program executable on the processor, wherein the processor executes the steps of a method for implementing a global attention mechanism as described in any one of claims 1 to 5 when executing the program.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for implementing a global attention mechanism as described in any one of claims 1 to 5 are performed.

Images