Self-attention image feature extraction method and system based on re-parameters

Abstract

The invention provides a re-parameter-based self-attention image feature extraction method. The method comprises the steps of obtaining a two-dimensional image; respectively inputting the two-dimensional image into a first path, a second path and a third path; respectively inputting the fusion data into a fourth path and a fifth path; the fourth path comprises adjacent context data information; the fifth path is used for outputting the first data matrix; and obtaining image features of the two-dimensional image according to the first data matrix, the second data matrix, the third data matrix and the adjacent context data information. According to the method, the contextual information between pixel neighbors is extracted by using the convolutional layers of different scales, so that the model can better pay attention to the local information of the image, more information interaction exists between the adjacent pixels compared with a traditional self-attention structure, and the model can mine more fine information. The invention further provides a re-parameter-based self-attention image feature extraction system, a computer readable storage medium and electronic equipment, which have the above beneficial effects.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, system, storage medium and electronic device for extracting self-attention image features based on heavy parameters. Background Art

[0002] The traditional self-attention mechanism can be used to express the interaction of features at different spatial locations. However, in the traditional self-attention mechanism, the query matrix Q and the key matrix K are learned independently, which severely limits the self-attention mechanism's ability to extract features from images. Especially in some industrial inspection scenarios, the scale of defects varies, ranging from a few pixels to thousands of pixels, and they need to be accurately identified one by one to avoid a large number of false detections. Summary of the invention

[0003] The purpose of this application is to provide a method, system, computer-readable storage medium and electronic device for self-attention image feature extraction based on heavy parameters, which can promote self-attention learning without excessively affecting computational efficiency.

[0004] In order to solve the above technical problems, this application provides a self-attention image feature extraction method based on heavy parameters, and the specific technical solution is as follows:

[0005] Acquire a two-dimensional image;

[0006] The two-dimensional image is input into a first path, a second path and a third path respectively; the first path is a single-branch convolution layer including several convolution layers of different scales, or obtained by training several convolution layers of different scales, and is used to output fused data; the second path includes a second fully connected layer and is used to output a second data matrix; the third path includes a third fully connected layer and is used to output a third data matrix;

[0007] The fused data is input into a fourth path and a fifth path respectively; the fourth path contains adjacent context data information; the fifth path is used to output a first data matrix;

[0008] The image features of the two-dimensional image are obtained according to the first data matrix, the second data matrix, the third data matrix and the neighboring context data information.

[0009] Optionally, obtaining the image feature of the two-dimensional image according to the first data matrix, the second data matrix, the third data matrix and the neighboring context data information includes:

[0010] multiplying the first data matrix and the second data matrix to obtain a first multiplication result;

[0011] Inputting the first multiplication result into the activation layer to obtain a weight matrix;

[0012] Multiplying the weight matrix and the third data matrix to obtain global context data information;

[0013] The global context data information and the neighboring context data are added pixel by pixel to obtain image features of the two-dimensional image.

[0014] Optionally, after inputting the fused data into the fifth path, the method further includes:

[0015] In the fifth path, the fused data is input into a data flattening layer to reduce the dimension of the fused data;

[0016] The fused data after dimension reduction is input into the first fully connected layer to obtain the first data matrix.

[0017] Optionally, if the first path includes a plurality of convolutional layers of different scales, the two-dimensional image input into the first path includes:

[0018] Inputting the two-dimensional images into convolutional layers of different scales respectively to obtain convolution results of each convolutional layer;

[0019] The convolution results output by each convolution layer are added pixel by pixel to obtain fused data.

[0020] Optionally, if the first path only includes a single-branch convolution layer obtained by training convolution layers of several different scales, the two-dimensional image is input into the first path and includes:

[0021] The two-dimensional image is output to the single-branch convolutional layer to obtain fused data.

[0022] Optionally, the training process of the single-branch convolutional layer includes:

[0023] Determine the weights of the convolution kernels in each convolution layer, take the largest convolution kernel as the center, add the weights at the corresponding center position, and merge them to obtain a single-branch convolution layer.

[0024] Optionally, inputting the first multiplication result into an activation layer to obtain a weight matrix includes:

[0025] The first multiplication result is input into the activation layer, and the weight matrix is ​​iteratively optimized during the training process through forward propagation and back propagation.

[0026] The present application also provides a self-attention image feature extraction system based on heavy parameters, comprising:

[0027] An image acquisition module, used for acquiring a two-dimensional image;

[0028] A matrix output module, used to input the two-dimensional image into a first path, a second path and a third path respectively; the first path is a single-branch convolution layer including several convolution layers of different scales, or obtained by training several convolution layers of different scales, and is used to output fused data; the second path includes a second fully connected layer and is used to output a second data matrix; the third path includes a third fully connected layer and is used to output a third data matrix; the fused data is input into a fourth path and a fifth path respectively; the fourth path includes neighboring context data information; the fifth path is used to output the first data matrix;

[0029] A feature extraction module is used to obtain image features of the two-dimensional image based on the first data matrix, the second data matrix, the third data matrix and the adjacent context data information.

[0030] The present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above-mentioned method when executed by a processor.

[0031] The present application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned method when calling the computer program in the memory.

[0032] The present application provides a self-attention image feature extraction method based on heavy parameters, comprising: obtaining a two-dimensional image; inputting the two-dimensional image into a first path, a second path and a third path respectively; the first path is a single-branch convolution layer including several convolution layers of different scales, or obtained by training several convolution layers of different scales, and is used to output fused data; the second path includes a second fully connected layer and is used to output a second data matrix; the third path includes a third fully connected layer and is used to output a third data matrix; the fused data is input into a fourth path and a fifth path respectively; the fourth path includes neighboring context data information; the fifth path is used to output a first data matrix; and the image features of the two-dimensional image are obtained according to the first data matrix, the second data matrix, the third data matrix and the neighboring context data information.

[0033] In the first path, the present application uses convolutional layers of different scales contained in the first path to extract contextual information between pixel neighbors, which helps the model pay more attention to the local information of the image, so that there is more information interaction between adjacent pixels compared to the traditional self-attention structure, so that the model can mine more subtle information. At the same time, in order to better extract local neighborhood information, the structural re-parameterization is further introduced into the traditional self-attention structure by executing a single-branch convolutional layer after structural re-parameterization, which promotes self-attention learning without excessively affecting computational efficiency.

[0034] The present application also provides a self-attention image feature extraction system based on heavy parameters, a computer-readable storage medium and an electronic device, which have the above-mentioned beneficial effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0036] Figure 1 A flowchart of a self-attention image feature extraction method based on heavy parameters provided in an embodiment of the present application;

[0037] Figure 2 A schematic diagram of the structure of image feature training provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of the structure of image feature reasoning provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of the structural re-parameterization process provided in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of the structure of a self-attention image feature extraction system based on heavy parameters provided in an embodiment of the present application;

[0041] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] See also Figure 1 , Figure 1 A flowchart of a self-attention image feature extraction method based on heavy parameters provided in an embodiment of the present application, the method comprising:

[0044] S101: Acquire a two-dimensional image;

[0045] S102: inputting the two-dimensional image into a first path, a second path and a third path respectively; the first path comprises a plurality of convolutional layers of different scales, or a single-branch convolutional layer obtained by training a plurality of convolutional layers of different scales, and is used to output fused data; the second path comprises a second fully connected layer and is used to output a second data matrix; the third path comprises a third fully connected layer and is used to output a third data matrix;

[0046] S103: inputting the fused data into a fourth path and a fifth path respectively; the fourth path contains adjacent context data information; the fifth path is used to output a first data matrix;

[0047] S104: Obtain image features of the two-dimensional image according to the first data matrix, the second data matrix, the third data matrix and the neighboring context data information.

[0048] The embodiment of the present application is based on the traditional self-attention structure, namely the query vector, key vector and value vector. In the traditional self-attention structure, each input vector generates a query, key and value vector by learning three different linear transformations (i.e., weight matrices). These vectors are used to calculate the attention score (dot product of Query and Key), measure the correlation between inputs, and generate output vectors.

[0049] In the traditional self-attention mechanism, the query vector Q and the key vector K are learned independently, ignoring the rich contextual information between neighbors, which seriously limits the self-attention mechanism's ability to extract features from images. To this end, this application aims to integrate contextual information mining and self-attention learning into a unified architecture, and by making full use of the contextual information between adjacent K, efficiently promote self-attention learning and enhance the expression of image features.

[0050] This embodiment does not limit the number of the acquired two-dimensional images, which may be one or more. For the input two-dimensional images, three different paths are input respectively, namely, the first path, the second path and the third path.

[0051] The first path is a convolution layer including several convolution layers of different scales, or a single branch convolution layer obtained by training several convolution layers of different scales. No matter what form of convolution layer is used in the first path, it is used to output fused data. The second path includes a second fully connected layer and is used to output a second data matrix. The third path includes a third fully connected layer and is used to output a third data matrix. The second data matrix is ​​equivalent to the query vector, and the third data matrix is ​​equivalent to the value vector.

[0052] After the fused data is obtained, the fused data enters the fourth path and the fifth path respectively, wherein the fused data can obtain a first data matrix after passing through the fifth path. The first data matrix is ​​equivalent to the key vector.

[0053] After obtaining the first data matrix, the second data matrix, and the third data matrix, conventional self-attention calculation can be performed. The specific process is as follows:

[0054] The first step is to multiply the first data matrix and the second data matrix to obtain a first multiplication result;

[0055] Step 2: Input the first multiplication result into the activation layer to obtain a weight matrix;

[0056] Step 3: multiplying the weight matrix and the third data matrix to obtain global context data information;

[0057] Step 4: Add the global context data information and the neighboring context data pixel by pixel to obtain the image features of the two-dimensional image.

[0058] First, the first data matrix K and the second data matrix Q are matrix multiplied, and the result is sent to the softmax activation layer to obtain the weight matrix, and then the weight matrix is ​​multiplied with the third data matrix V to obtain the global context information data, and then the neighboring context information data from the fourth path is fused by adding pixel by pixel, and finally the output is obtained. Compared with the traditional self-attention method of directly using the fully connected layer to obtain the matrix K, the present application can use the convolution layer based on heavy parameters to extract adjacent features when generating the first data matrix, i.e. the key vector, which increases the interaction of neighborhood information, and then fuses the neighborhood information with the global information, which helps the model to better extract image features.

[0059] Since this application is designed based on the method of structural re-parameterization, it can be divided into training structure and reasoning structure. The purpose of the training structure is to minimize the prediction error and improve the generalization ability of the model. The reasoning structure has been trained and is used to predict new data, without involving weight updates.

[0060] See also Figure 2 and Figure 3 , Figure 2 This is a schematic diagram of the structure of the image feature training provided in the embodiment of the present application. Figure 3 This is a schematic diagram of the structure of image feature inference provided by the embodiment of the present application. It can be seen that Figure 2 and Figure 3 The main difference is in the first path. The input is one or more two-dimensional images, and the input goes into the first path, the second path, and the third path respectively.

[0061] Figure 2 In the corresponding training structure, the data of the second and third paths will enter the second fully connected layer and the third fully connected layer respectively to obtain the second data matrix Q and the third data matrix V. The data of the first path will enter multiple convolutional layers with different convolution kernel sizes. The first path contains several convolutional layers. Figure 2 Only three convolutional layers are shown in the figure, and those skilled in the art can use more convolutional layers according to actual needs. Among them, (2N+1)×(2N+1)conv represents a convolutional layer with a convolution kernel length and width of (2N+1) and (2N+1), where N≥0. Using convolution kernels of different sizes to extract contextual information between pixel neighbors helps the model to better focus on local information of the image, so that there is more information interaction between adjacent pixels compared to the traditional self-attention structure, so that the model can mine more subtle information.

[0062] The results of the three groups of convolutions are summed pixel by pixel. Figure 2 and Figure 3 The data after data fusion will enter the fourth path and the fifth path respectively. The data of the fifth path will first pass through a data flattening layer Flatten to flatten the three-dimensional data into two-dimensional data, and then be sent to the first fully connected layer to obtain the first data matrix K. It should be noted that other fusion methods can also be used after the three sets of convolutions, such as channel splicing, but it will greatly increase the computational burden, so pixel-by-pixel addition can avoid a large computational burden.

[0063] During inference, the structure merges the convolutional layers of different scales in the first path into one convolutional layer, that is, simplifies the multi-branch structure of the convolution part into a single-branch structure. The single-branch structure runs faster. Since the multi-branch structure is used during training, the single-branch structure after structural re-parameterization also has the powerful feature extraction capability of the multi-branch structure.

[0064] In the training structure, the first path includes several convolution layers of different scales. At this time, after the two-dimensional image is input into the first path, the two-dimensional image is input into the convolution layers of different scales respectively to obtain the convolution results of each convolution layer, and then the convolution results output by each convolution layer are added pixel by pixel to obtain fused data.

[0065] In the inference structure, the first path only includes a single-branch convolution layer obtained by training convolution layers of several different scales. After the two-dimensional image is input to the first path, the two-dimensional image is directly output to the single-branch convolution layer to obtain fused data.

[0066] When generating a single-branch convolutional layer, the weights of the convolution kernels in each convolutional layer can be determined, and the weights are added at the corresponding center position with the largest convolution kernel as the center to obtain a single-branch convolutional layer.

[0067] See also Figure 4 , Figure 4 This is a schematic diagram of the structure re-parameterization process provided in the embodiment of the present application. Figure 4 Taking N=0 as an example, the sizes of convolution kernels during training are 1×1, 3×3, and 5×5, respectively, where w1~w35 are the weights of the convolution kernels. When the structure is re-parameterized, the largest convolution kernel is taken as the center, and the weights are added at the corresponding positions. Finally, after merging, only one convolution layer with the largest convolution kernel remains.

[0068] In the softmax activation layer, the first multiplication result can be input into the activation layer, and the weight matrix is ​​obtained by iterative optimization during the training process through forward propagation and back propagation. The process of iterative optimization of forward propagation and back propagation is not described in detail here.

[0069] See also Figure 5 , Figure 5 A structural schematic diagram of a self-attention image feature extraction system based on heavy parameters provided in an embodiment of the present application, the system comprising:

[0070] An image acquisition module, used for acquiring a two-dimensional image;

[0071] A matrix output module, used to input the two-dimensional image into a first path, a second path and a third path respectively; the first path is a single-branch convolution layer including several convolution layers of different scales, or obtained by training several convolution layers of different scales, and is used to output fused data; the second path includes a second fully connected layer and is used to output a second data matrix; the third path includes a third fully connected layer and is used to output a third data matrix; the fused data is input into a fourth path and a fifth path respectively; the fourth path includes neighboring context data information; the fifth path is used to output the first data matrix;

[0072] A feature extraction module is used to obtain image features of the two-dimensional image based on the first data matrix, the second data matrix, the third data matrix and the adjacent context data information.

[0073] This application applies several paths included in the matrix input module. In the first path, the convolution layers of different scales included in the first path are used to extract contextual information between pixel neighbors, which helps the model pay more attention to the local information of the image, so that there is more information interaction between adjacent pixels compared to the traditional self-attention structure, so that the model can mine more subtle information. At the same time, in order to better extract local neighborhood information, the structural re-parameterization is further introduced into the traditional self-attention structure by executing a single-branch convolution layer after structural re-parameterization, which promotes self-attention learning without excessively affecting computational efficiency.

[0074] Based on the above embodiment, as an optional embodiment, the feature extraction module includes:

[0075] A first multiplication result determining unit, configured to multiply the first data matrix and the second data matrix to obtain a first multiplication result;

[0076] A weight matrix determination unit, used for inputting the first multiplication result into an activation layer to obtain a weight matrix;

[0077] A global context data information determining unit, configured to multiply the weight matrix and the third data matrix to obtain global context data information;

[0078] An image feature determination unit is used to add the global context data information and the neighboring context data pixel by pixel to obtain the image feature of the two-dimensional image.

[0079] Based on the above embodiment, as an optional embodiment, it also includes:

[0080] An input module, used for inputting the fused data into a data flattening layer in the fifth path to reduce the dimension of the fused data;

[0081] The first data matrix determination module is used to input the fusion data after dimension reduction into the first fully connected layer to obtain the first data matrix.

[0082] Based on the above embodiment, as an optional embodiment, the matrix output module includes:

[0083] An input unit, used to input the two-dimensional image into convolution layers of different scales respectively to obtain convolution results of each convolution layer;

[0084] The first fused data determining unit is used to add the convolution results output by each of the convolution layers pixel by pixel to obtain fused data.

[0085] Based on the above embodiment, as an optional embodiment, the matrix output module includes:

[0086] The second fused data determining unit is used to output the two-dimensional image to the single-branch convolutional layer to obtain fused data.

[0087] Based on the above embodiment, as an optional embodiment, the weight matrix determining unit includes:

[0088] The weight matrix determination subunit is used to input the first multiplication result into the activation layer, and obtain the weight matrix by iterative optimization during the training process through forward propagation and back propagation.

[0089] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0090] The present application also provides an electronic device, see Figure 6 , a structural diagram of an electronic device provided in an embodiment of the present application, such as Figure 6 As shown, a processor 1410 and a memory 1420 may be included.

[0091] Among them, the processor 1410 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1410 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1410 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1410 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1410 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0092] The memory 1420 may include one or more computer-readable storage media, which may be non-transitory. The memory 1420 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 1420 is at least used to store the following computer program 1421, wherein, after the computer program is loaded and executed by the processor 1410, it can implement the relevant steps in the method performed by the electronic device side disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 1420 may also include an operating system 1422 and data 1423, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 1422 may include Windows, Linux, Android, etc.

[0093] In some embodiments, the electronic device may further include a display screen 1430 , an input / output interface 1440 , a communication interface 1450 , a sensor 1460 , a power source 1470 , and a communication bus 1480 .

[0094] certainly, Figure 6 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiments of the present application. In actual applications, the electronic device may include Figure 6 More or fewer components than shown, or combinations of certain components.

[0095] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the system provided in the embodiment, since it corresponds to the method provided in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0096] Specific examples are used herein to illustrate the principles and implementation methods of the present application, and the description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

[0097] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A self-attention image feature extraction method based on heavy parameters, characterized in that: include: Acquire a two-dimensional image; The two-dimensional image is input into a first path, a second path and a third path respectively; the first path is a single-branch convolution layer including several convolution layers of different scales, or obtained by training several convolution layers of different scales, and is used to output fused data; the second path includes a second fully connected layer and is used to output a second data matrix; the third path includes a third fully connected layer and is used to output a third data matrix; Inputting the fused data into a fourth path and a fifth path respectively; the fourth path contains neighboring context data information; The fifth path is used to output the first data matrix; The image features of the two-dimensional image are obtained according to the first data matrix, the second data matrix, the third data matrix and the neighboring context data information.

2. The method for extracting image features based on self-attention with heavy parameters according to claim 1, characterized in that: Obtaining the image features of the two-dimensional image according to the first data matrix, the second data matrix, the third data matrix and the adjacent context data information includes: multiplying the first data matrix and the second data matrix to obtain a first multiplication result; Inputting the first multiplication result into the activation layer to obtain a weight matrix; Multiplying the weight matrix and the third data matrix to obtain global context data information; The global context data information and the neighboring context data are added pixel by pixel to obtain image features of the two-dimensional image.

3. The method for extracting image features based on self-attention with heavy parameters according to claim 1, characterized in that: After the fused data is input into the fifth path, the method further includes: In the fifth path, the fused data is input into a data flattening layer to reduce the dimension of the fused data; The fused data after dimension reduction is input into the first fully connected layer to obtain the first data matrix.

4. The method for extracting image features based on self-attention with heavy parameters according to claim 1, characterized in that: If the first path includes a plurality of convolutional layers of different scales, the two-dimensional image is input into the first path and includes: Inputting the two-dimensional images into convolutional layers of different scales respectively to obtain convolution results of each convolutional layer; The convolution results output by each convolution layer are added pixel by pixel to obtain fused data.

5. The method for extracting image features based on self-attention with heavy parameters according to claim 1, characterized in that: If the first path only includes a single-branch convolution layer obtained by training convolution layers of several different scales, the two-dimensional image is input into the first path and includes: The two-dimensional image is output to the single-branch convolutional layer to obtain fused data.

6. The method for extracting image features based on self-attention with heavy parameters according to claim 1 or 5, characterized in that: The training process of the single-branch convolutional layer includes: Determine the weights of the convolution kernels in each convolution layer, take the largest convolution kernel as the center, add the weights at the corresponding center position, and merge them to obtain a single-branch convolution layer.

7. The method for extracting image features based on self-attention with heavy parameters according to claim 2, characterized in that: Inputting the first multiplication result into the activation layer to obtain a weight matrix includes: The first multiplication result is input into the activation layer, and the weight matrix is ​​iteratively optimized during the training process through forward propagation and back propagation.

8. A self-attention image feature extraction system based on heavy parameters, characterized in that: include: An image acquisition module, used for acquiring a two-dimensional image; A matrix output module, used to input the two-dimensional image into a first path, a second path and a third path respectively; the first path is a single-branch convolution layer including several convolution layers of different scales, or obtained by training several convolution layers of different scales, and is used to output fused data; the second path includes a second fully connected layer and is used to output a second data matrix; the third path includes a third fully connected layer and is used to output a third data matrix; the fused data is input into a fourth path and a fifth path respectively; the fourth path includes neighboring context data information; the fifth path is used to output the first data matrix; A feature extraction module is used to obtain image features of the two-dimensional image based on the first data matrix, the second data matrix, the third data matrix and the adjacent context data information.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the self-attention image feature extraction method based on heavy parameters as described in any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: It includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of the self-attention image feature extraction method based on heavy parameters as described in any one of claims 1 to 7 are implemented.

Images