A self-attention calculation method, device, system and storage medium

Abstract

The present application discloses a self-attention calculation method, device, system and storage medium, which relate to the field of machine learning technology. The self-attention calculation method includes: based on sequence data, obtaining first data and a current window; based on the current window, obtaining multiple second data from the sequence data; obtaining the first feature difference value of each second data; if the first feature difference value meets the first preset condition, updating the current window. The present application selects only part of the input data to participate in the attention weight calculation of subsequent target data by setting a dynamic window (that is, the current window) centered on the target data (that is, the first data). It can not only effectively reduce the complexity of attention weight calculation, but also ensure that the self-attention mechanism can capture local information of the input data.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and specifically to a self-attention calculation method, device, system and storage medium. Background Art

[0002] The application scenarios of the self-attention mechanism are very wide, and it is mainly used in the fields of natural language processing and computer vision technology. Specifically, in the technical field of natural language processing, the self-attention mechanism is widely used in tasks such as machine translation, text generation, and speech recognition to help the model better understand the structure and meaning of sentences and improve the performance of language models. For example, in machine translation, the self-attention mechanism can help the model focus on multiple words in the source language and the target language at the same time, thereby improving the accuracy and fluency of translation. Specifically, in the technical field of computer vision, the self-attention mechanism is also applied to tasks such as object detection and image classification. By helping the model better understand the objects in the image and the relationships between them, the self-attention mechanism can improve the accuracy and efficiency of image recognition. For example, in object detection, the self-attention mechanism can help the model better understand the spatial relationship between objects, thereby improving the accuracy and speed of detection.

[0003] The complexity of the self-attention calculation method in the prior art is quadratically related to the length of the sequence data it processes (see the specific implementation for the specific reasons). In other words, the longer the sequence data is, the higher the complexity of the calculation, which means that the calculation takes longer. Summary of the invention

[0004] The purpose of this application is to provide a self-attention calculation method, device, system and storage medium to solve the problem of how to reduce the time consumption when performing self-attention calculation on long sequence data.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] In a first aspect, the present application proposes a technical solution for a self-attention calculation method, the self-attention calculation method comprising:

[0007] Based on the sequence data, first data and a current window are obtained; the sequence data is obtained in advance; the first data is any data in the sequence data; the current window is updated by the historical window;

[0008] Acquire a plurality of second data from the sequence data based on the current window; the current window is centered on the first data, and the plurality of second data are all located within the current window;

[0009] Obtaining a first characteristic difference value of each second data;

[0010] If the first feature difference value meets a first preset condition, the current window is updated; if the first feature difference value meets a second preset condition, the attention weight of the first data is obtained based on each second data.

[0011] As a specific solution in the technical solution of the present application, the step of obtaining the first characteristic difference value of each second data includes:

[0012] Based on each second data, third data and fourth data are acquired; the third data and the fourth data are any two adjacent data in each second data;

[0013] Based on the third data, obtaining a first characteristic value;

[0014] Based on the fourth data, obtaining a second characteristic value;

[0015] A first feature difference value is obtained based on the first feature value and the second feature value.

[0016] As a specific solution in the technical solution of the present application, the step of obtaining the first characteristic difference value of each second data includes:

[0017] Based on each second data, fifth data is obtained; the fifth data is any data in each second data;

[0018] Based on the first data, obtaining a third characteristic value;

[0019] Based on the fifth data, obtaining a fourth characteristic value;

[0020] A first feature difference value is obtained based on the third feature value and the fourth feature value.

[0021] As a specific solution in the technical solution of the present application, if the first feature difference value satisfies a first preset condition, updating the current window includes:

[0022] If the absolute value of the first characteristic difference value is greater than or equal to a first preset value, enlarging the current window;

[0023] If the absolute value of the first characteristic difference value is less than or equal to a second preset value, shrinking the current window;

[0024] The second preset condition includes:

[0025] An absolute value of the first characteristic difference value is greater than the second preset value and less than the first preset value.

[0026] As a specific solution in the technical solution of the present application, obtaining a first feature difference value based on the first feature value and the second feature value includes:

[0027] Obtaining an absolute value of a difference between the first eigenvalue and the second eigenvalue;

[0028] Based on the absolute values ​​of each difference, the average value is obtained;

[0029] Based on the average value, obtaining a first characteristic difference value;

[0030] If the absolute value of the first characteristic difference value is greater than or equal to a first preset value, enlarging the current window includes:

[0031] Based on the first characteristic difference value and the first preset value, obtaining a magnification ratio;

[0032] The current window is enlarged based on the enlargement ratio.

[0033] As a specific solution in the technical solution of the present application, the step of obtaining the attention weight of the first data based on each second data includes:

[0034] Based on each second data, obtaining an input matrix;

[0035] Based on the input matrix, obtaining a dimensionality reduction decomposition matrix;

[0036] Obtaining an attention score based on the input matrix and the dimensionality reduction decomposition matrix;

[0037] Based on the attention score, an attention weight of the first data is obtained.

[0038] As a specific solution in the technical solution of the present application, the step of obtaining a dimensionality reduction decomposition matrix based on the input matrix includes:

[0039] Based on the input matrix, obtaining an initial weight matrix;

[0040] Based on the initial weight matrix, obtaining sixth data; the sixth data is obtained by multiplying the first matrix and the second matrix;

[0041] Based on the initial weight matrix and the sixth data, obtaining a second characteristic difference value;

[0042] If the second characteristic difference value is greater than or equal to a third preset value, the first matrix and the second matrix are updated; otherwise, the sixth data is used as a dimensionality reduction decomposition matrix.

[0043] In a second aspect, the present application proposes a technical solution of a self-attention computing device, the self-attention computing device comprising:

[0044] A reading module, used for acquiring first data and a current window based on sequence data; the sequence data is acquired in advance; the first data is any data in the sequence data; the current window is updated by a historical window;

[0045] A processing module, based on the current window, obtains a plurality of second data from the sequence data; the current window is centered on the first data;

[0046] and obtaining a first characteristic difference value of each second data;

[0047] And, if the first feature difference value meets a first preset condition, the current window is updated; if the first feature difference value meets a second preset condition, the attention weight of the first data is obtained based on each second data.

[0048] As a specific solution in the technical solution of the present application, the reading module is further used to obtain third data and fourth data based on each second data; the third data and the fourth data are any two adjacent data in each second data;

[0049] The processing module is further used to obtain a first characteristic value based on the third data;

[0050] And, based on the fourth data, obtaining a second characteristic value;

[0051] And, based on the first feature value and the second feature value, a first feature difference value is obtained.

[0052] As a specific solution in the technical solution of the present application, the processing module is further used to obtain fifth data based on each second data; the fifth data is any data in each second data;

[0053] And, based on the first data, obtaining a third characteristic value;

[0054] and, based on the fifth data, obtaining a fourth characteristic value;

[0055] And, based on the third feature value and the fourth feature value, a first feature difference value is obtained.

[0056] As a specific solution in the technical solution of the present application, the processing module is further used to, if the absolute value of the first feature difference value is greater than or equal to a first preset value, enlarge the current window;

[0057] and, if the absolute value of the first characteristic difference value is less than or equal to a second preset value, reducing the current window;

[0058] The second preset condition includes: an absolute value of the first characteristic difference value is greater than a second preset value and less than a first preset value.

[0059] As a specific solution in the technical solution of the present application, the processing module is further used to obtain an absolute value of a difference between the first eigenvalue and the second eigenvalue;

[0060] and, based on the absolute values ​​of the respective differences, obtaining an average value;

[0061] And, based on the average value, obtaining a first characteristic difference value;

[0062] The processing module is further configured to obtain an amplification ratio based on the first characteristic difference value and the first preset value;

[0063] And, based on the magnification ratio, the current window is magnified.

[0064] As a specific solution in the technical solution of the present application, the processing module is further used to obtain an input matrix based on each second data;

[0065] And, based on the input matrix, obtaining a dimension reduction decomposition matrix;

[0066] And, obtaining an attention score based on the input matrix and the dimensionality reduction decomposition matrix;

[0067] And, based on the attention score, obtaining the attention weight of the first data.

[0068] As a specific solution in the technical solution of the present application, the processing module is further used to obtain an initial weight matrix based on the input matrix;

[0069] And, based on the initial weight matrix, obtaining sixth data; the sixth data is obtained by multiplying the first matrix and the second matrix;

[0070] and, based on the initial weight matrix and the sixth data, obtaining a second characteristic difference value;

[0071] And, if the second characteristic difference value is greater than or equal to a third preset value, the first matrix and the second matrix are updated; otherwise, the sixth data is used as a dimensionality reduction decomposition matrix.

[0072] In a third aspect, the present application proposes a technical solution for a self-attention computing system, the self-attention computing system comprising:

[0073] A reader, used for acquiring first data and a current window based on sequence data; the sequence data is acquired in advance; the first data is any data in the sequence data; the current window is updated by a historical window;

[0074] The processor acquires a plurality of second data from the sequence data based on the current window; the current window is centered on the first data;

[0075] and obtaining a first characteristic difference value of each second data;

[0076] And, if the first feature difference value meets a first preset condition, the current window is updated; if the first feature difference value meets a second preset condition, the attention weight of the first data is obtained based on each second data.

[0077] As a specific solution in the technical solution of the present application, the reader is further used to obtain third data and fourth data based on each second data; the third data and the fourth data are any two adjacent data in each second data;

[0078] The processor is further configured to obtain a first characteristic value based on the third data;

[0079] And, based on the fourth data, obtaining a second characteristic value;

[0080] And, based on the first feature value and the second feature value, a first feature difference value is obtained.

[0081] As a specific solution in the technical solution of the present application, the processor is further used to obtain fifth data based on each second data; the fifth data is any data in each second data;

[0082] And, based on the first data, obtaining a third characteristic value;

[0083] and, based on the fifth data, obtaining a fourth characteristic value;

[0084] And, based on the third feature value and the fourth feature value, a first feature difference value is obtained.

[0085] As a specific solution in the technical solution of the present application, the processor is further configured to, if the absolute value of the first feature difference value is greater than or equal to a first preset value, enlarge the current window;

[0086] and, if the absolute value of the first characteristic difference value is less than or equal to a second preset value, reducing the current window;

[0087] The second preset condition includes: an absolute value of the first characteristic difference value is greater than a second preset value and less than a first preset value.

[0088] As a specific solution in the technical solution of the present application, the processor is further used to obtain an absolute value of a difference between the first eigenvalue and the second eigenvalue;

[0089] and, based on the absolute values ​​of the respective differences, obtaining an average value;

[0090] And, based on the average value, obtaining a first characteristic difference value;

[0091] The processor is further configured to obtain a magnification ratio based on the first characteristic difference value and the first preset value;

[0092] And, based on the magnification ratio, the current window is magnified.

[0093] As a specific solution in the technical solution of the present application, the processor is further used to obtain an input matrix based on each second data;

[0094] And, based on the input matrix, obtaining a dimension reduction decomposition matrix;

[0095] And, obtaining an attention score based on the input matrix and the dimensionality reduction decomposition matrix;

[0096] And, based on the attention score, obtaining the attention weight of the first data.

[0097] As a specific solution in the technical solution of the present application, the processor is further used to obtain an initial weight matrix based on the input matrix;

[0098] And, based on the initial weight matrix, obtaining sixth data; the sixth data is obtained by multiplying the first matrix and the second matrix;

[0099] and, based on the initial weight matrix and the sixth data, obtaining a second characteristic difference value;

[0100] And, if the second characteristic difference value is greater than or equal to a third preset value, the first matrix and the second matrix are updated; otherwise, the sixth data is used as a dimensionality reduction decomposition matrix.

[0101] In a fourth aspect, the present application proposes a technical solution of a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, a self-attention calculation method as described in any one of the first aspects is implemented.

[0102] Compared with the prior art, the beneficial effects of this application are:

[0103] This application selects only part of the input data to participate in the attention weight calculation of subsequent target data by setting a dynamic window (also referred to as the current window below) centered on the target data (also referred to as the first data below). This can effectively reduce the complexity of attention weight calculation and ensure that the self-attention mechanism can capture local information of the input data. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] Figure 1 A flowchart of a self-attention calculation method proposed in an embodiment of the present application;

[0105] Figure 2 A schematic diagram of the structure of a self-attention calculation device proposed in an embodiment of the present application;

[0106] Figure 3 A schematic diagram of the structure of a self-attention calculation system proposed in an embodiment of the present application;

[0107] Figure 4 A schematic diagram of input data proposed in an embodiment of the present application;

[0108] Figure 5 A schematic diagram of setting an initial window in input data proposed in an embodiment of the present application;

[0109] Figure 6 A schematic diagram of a weight matrix decomposition proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0110] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0111] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0112] The terms "first", "second", etc. in the description of the embodiment of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, such as the first characteristic difference value and the second characteristic difference value proposed below, which belong to different numerical values. It should be understood that the numerical values ​​used in this way can be interchanged where appropriate, so that the embodiments described here can be implemented in an order other than the content illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device including a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The division of the modules that appear in the embodiment of the present application is only a logical division. There may be other division methods when implemented in practical applications, such as multiple modules can be combined or integrated into another system, or some features can be ignored, or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in the embodiment of the present application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules, and some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0113] In order to solve the technical problem that it takes a long time to perform self-attention calculation on long sequence data in the prior art, the present application proposes a self-attention calculation method, such as Figure 1 As shown, the self-attention calculation method includes steps S100 to S500.

[0114] Step S100: based on the sequence data, obtaining the first data and the current window.

[0115] In an embodiment of the present application, the sequence data is acquired in advance. In the technical field of natural language processing, it is necessary to convert the input text data into sequence data in advance for subsequent computer processing. In the technical field of computer vision, it is also necessary to convert the input video data or image data into sequence data in advance for subsequent computer processing. That is to say, in this embodiment, acquiring sequence data in advance is a mature technology and will not be described in detail here.

[0116] As can be seen from the background technology, the application scenarios of the self-attention mechanism are very wide, that is, in the embodiments of the present application, it is impossible to enumerate all the application scenarios of the self-attention mechanism. In the following of the present application, only the application scenario of image recognition is taken as an example to illustrate the self-attention calculation method proposed in the present application, which does not mean that the self-attention calculation method proposed in the present application is only applicable to the application scenario of image recognition, and it is also applicable to other application scenarios that require the use of the self-attention mechanism.

[0117] It should be noted that the self-attention mechanism is a technique used in neural network models that allows the model to pay different degrees of attention to different parts of the input data when processing information. The core of this mechanism is that when the model processes the data at each position, it will consider the correlation between the data at that position and all other positions, thereby determining the importance of each part of the data. That is to say, in the existing technology, such as Figure 4 As shown, assuming that the input data has a total of n data, namely X1 to X n ; If you need to calculate the i-th data X i The attention weight (that is, calculating the i-th data X i and all other data), then we need to calculate the i-th data X i and other data (i.e. X1 to X n ), and then obtain the i-th data X based on the attention score i Since the calculation process of attention weight is a mature technology, we will not go into details.

[0118] It should be noted that in the process of calculating the attention score, the data X needs to be calculated separately i With the attention score of data X1, data X i Attention score with data X2...data X i With data X i Attention score of data X i With data X n That is to say, in the prior art, if there are n data in the sequence data, the number of times the attention score is calculated is n. 2 (That is, the computational complexity of the self-attention calculation method described in the background technology is quadratic to the length of the processed sequence data.) In other words, the larger n is, that is, the longer the length of the sequence data is, the higher the computational complexity of the self-attention calculation method is.

[0119] As can be seen from the foregoing, in the calculation process of self-attention, it is necessary to calculate the attention score of each data in the sequence data, and then obtain the attention weight based on the attention score. That is to say, in an embodiment of the present application, the first data is any data in the sequence data. In other words, the first data is any data in the sequence data that needs to be calculated for attention weight.

[0120] In one embodiment of the present application, in order to reduce the complexity of calculation, an initial window may be set with the first data as the center, and only the attention scores between each data in the initial window and the first data are calculated, and then the attention weight of the first data is calculated based on each attention score. Figure 5 As shown, assuming that the input data has a total of n data, namely X1 to X n , and the first data is X i , the initial window contains 2w+1 data (that is, the data in the initial window is X i-w To X i+w ); then the first data X i The number of attention score calculations is 2w+1, that is, the total number of attention score calculations for all data in the data sequence is (2w+1)*n. In other words, this embodiment uses the initial window to limit each position data (that is, the first data) and only pays attention to other position input data within its local range, thereby significantly reducing the computational complexity.

[0121] It should be noted that in long sequence data, if the initial window is small, the context with a longer distance cannot be focused on, that is, the data in the initial window cannot reflect the true attention weight of the first data, that is, when calculating the attention score, it is easy to ignore other data related to the first data in the input data (that is, the attention mechanism cannot capture local information). In order to obtain the true attention weight that can reflect the first data and reduce the complexity of the calculation, as described in steps S300 to S500 of this embodiment, the window can be updated to obtain the current window suitable for the first data.

[0122] It is easy to understand that in the embodiment of the present application, the current window can be updated from the historical window. That is to say, initially, the current window can be the initial window, and after the current window is updated once, the historical window can be the initial window.

[0123] Step S200: Acquire a plurality of second data from the sequence data based on the current window.

[0124] It should be noted that, in this embodiment, the second data refers to any data located in the current window. Figure 5 As shown, Figure 5The red schematic box represents the current window. The current window is based on the first data (i.e., data X i ) as the center, and the second data, that is, X i-w To X i+w Any data in .

[0125] Step S300: Obtain the first characteristic difference value of each second data.

[0126] In an embodiment of the present application, taking image recognition as an example, if the pixel values ​​of each pixel point in the current window are relatively different, it means that the pixel values ​​of each pixel point in the area of ​​the current window have changed dramatically. In order to avoid the attention mechanism from being unable to capture local information, the current window can be increased. If the pixel values ​​of each pixel point in the current window are not much different, it means that the pixel values ​​of each pixel point in the area of ​​the current window are basically unchanged. In order to avoid high computational complexity, the current window can be appropriately reduced. Of course, if the current window is equal to the initial window, the current window may not be updated.

[0127] In this embodiment, we define the numerical value corresponding to each second data as a characteristic value (for example, gray value or RGB value, etc.); and define the difference of the characteristic values ​​corresponding to each second data as a characteristic difference value. It should be noted that obtaining a characteristic difference value (that is, the difference between two characteristic values) based on two second data is a mature technology and will not be described in detail here.

[0128] As can be seen from the foregoing, if the difference between the pixel values ​​(i.e., eigenvalues) in the current window is too large or too small, the size of the current window can be adjusted. This avoids the inability of the attention mechanism to capture local information and also avoids excessive complexity in subsequent calculations. In an embodiment of the present application, the purpose of obtaining the first eigenvalue difference value is to characterize the eigenvalue difference size corresponding to each data in the current window, and then adjust the size of the current window.

[0129] In the embodiments of the present application, the first characteristic difference value of each second data may be obtained in any manner. For example, the first characteristic difference value of each second data may be obtained by the methods proposed in the following embodiments.

[0130] Method 1 for obtaining the first characteristic difference value

[0131] In this embodiment, step S300, obtaining the first characteristic difference value of each second data, includes steps S310 to S340.

[0132] Step S310: based on each second data, obtain third data and fourth data.

[0133] It should be noted that the third data and the fourth data are any two adjacent data in each second data. In other words, the third data and the fourth data are any two adjacent data in the current window.

[0134] Step S320: Based on the third data, obtain a first eigenvalue.

[0135] It should be clear that obtaining a characteristic value of data (eg, pixel value or RGB value, etc.) based on data is a mature technology and will not be described in detail here.

[0136] Step S330: Based on the fourth data, obtain a second characteristic value.

[0137] It should be clear that obtaining a characteristic value of data (eg, pixel value or RGB value, etc.) based on data is a mature technology and will not be described in detail here.

[0138] Step S340: Obtain a first feature difference value based on the first feature value and the second feature value.

[0139] In this embodiment, the first feature difference value may be the absolute value of the difference between the first feature value and the second feature value (hereinafter referred to as the first difference value). Alternatively, the first feature difference value may be the absolute value of the difference between any two first differences (ie, the difference gradient).

[0140] In order to enable the first feature difference value to comprehensively reflect the degree of change of the feature values ​​of each input data in the current window, in one embodiment of the present application, step S340, based on the first feature value and the second feature value, obtains the first feature difference value, including steps S341 to S343.

[0141] Step S341: Obtain the absolute value of the difference between the first eigenvalue and the second eigenvalue.

[0142] It should be clear that obtaining the absolute value of the difference between two values ​​is a mature technology and will not be described in detail here.

[0143] Step S342: Based on the absolute values ​​of the differences, obtain the average value.

[0144] As can be seen from the foregoing, based on any two adjacent second data in the current window, a corresponding absolute difference value can be obtained. In other words, based on multiple second data in the current window, multiple absolute difference values ​​can also be obtained.

[0145] Specifically, in this embodiment, in step S342, based on the absolute values ​​of the differences, the calculation formula for obtaining the average value is as follows:

[0146] / (n-1)

[0147] Where A represents the average value; n represents the number of second data in the current window; represents the characteristic value of the i+1th second data in the current window; Represents the characteristic value of the i-th second data in the current window; Indicates absolute value.

[0148] In this embodiment, the above average value can comprehensively reflect the degree of change of the characteristic values ​​of each input data in the current window, and the size of the current window can be accurately adjusted based on the average value.

[0149] Step S343: Based on the average value, obtain a first characteristic difference value.

[0150] In the embodiment of the present application, the average value may be directly used as the first characteristic difference value.

[0151] It should be clear that in this embodiment, the first feature difference value reflects the degree of change of the feature values ​​of any two adjacent input data in the current window. It is easy to understand that if the first feature difference value is larger, it means that the difference between the input data in the current window is larger, that is, the current window needs to be expanded to avoid the attention mechanism from failing to capture local information.

[0152] Method 2 for obtaining the first characteristic difference value

[0153] In this embodiment, step S300, obtaining the first characteristic difference value of each second data, includes steps S350 to S380.

[0154] Step S350: Acquire fifth data based on each second data.

[0155] In this embodiment, the fifth data is any data in each second data. That is to say, in this embodiment, the fifth data is any data in the current window.

[0156] Step S360: Based on the first data, obtain a third eigenvalue.

[0157] Based on the first data, obtaining the third characteristic value is to obtain the characteristic value of the first data itself. It should be clear that obtaining the characteristic value of data (such as pixel value or RGB value, etc.) based on a data is a mature technology and will not be described in detail here.

[0158] Step S370: Based on the fifth data, obtain a fourth eigenvalue.

[0159] Based on the fifth data, the fourth characteristic value is obtained, that is, the characteristic value of the fifth data is obtained based on the fifth data. It should be understood that obtaining the characteristic value of a data (eg, pixel value or RGB value, etc.) based on a data is a mature technology and will not be described in detail here.

[0160] Step S380: Obtain a first feature difference value based on the third feature value and the fourth feature value.

[0161] In an embodiment of the present application, the first characteristic difference value may be an average value of the difference between the third characteristic value and the fourth characteristic value. Of course, in order to fully reflect the degree of difference between each input data (i.e., the second data) in the current window and the characteristic value of the first data, in an embodiment of the present application, in step S380, based on the third characteristic value and the fourth characteristic value, the calculation formula for obtaining the first characteristic difference value is as follows:

[0162] / n

[0163] Wherein, B represents the first feature difference value; n represents the number of second data in the current window; represents the characteristic value of the i-th second data in the current window; Y represents the characteristic value of the first data; Indicates absolute value.

[0164] It should be clear that in this embodiment, the purpose of adjusting the current window size is to reduce the complexity of subsequent calculations and to avoid the inability of the attention mechanism to capture local information. It is easy to understand that when calculating the attention score of the first data, only all the second data in the current window corresponding to the first data are focused on. In other words, if the difference in the eigenvalues ​​of all the second data in the current window and the first data is small, the second data in the current window can fully reflect the subsequent attention weight of the first data; if the difference in the eigenvalues ​​of all the second data in the current window and the first data is large, the second data in the current window is likely to be unable to fully reflect the subsequent attention weight of the first data, that is, it is necessary to expand the current window corresponding to the first data.

[0165] Step S400: If the first characteristic difference value satisfies a first preset condition, the current window is updated.

[0166] In an embodiment of the present application, the first preset condition can be set to a suitable condition according to requirements. For example, in an embodiment of the present application, step S400, if the first feature difference value meets the first preset condition, then updating the current window includes step S410 and step S420.

[0167] Step S410: If the absolute value of the first characteristic difference value is greater than or equal to a first preset value, enlarging the current window.

[0168] In the embodiment of the present application, the first preset value can be set to any appropriate value according to the requirements. For example, in the application scenario of image recognition, if the feature value is a gray value, the first preset value can be set to 50 or 100.

[0169] In order to avoid the current window being enlarged without limit, which results in the inability to significantly reduce the complexity of the calculation, in one embodiment of the present application, step S410, if the absolute value of the first feature difference value is greater than or equal to a first preset value, enlarging the current window includes step S411 and step S412.

[0170] Step S411: obtaining a magnification ratio based on the first feature difference value and the first preset value.

[0171] In an embodiment of the present application, the magnification ratio may be the ratio of the first feature difference value to the first preset value. That is, the larger the difference between the first feature difference value and the first preset value, the larger the magnification ratio of the current window; the smaller the difference between the first feature difference value and the first preset value, the smaller the magnification ratio of the current window. Of course, in this embodiment, in order for the attention mechanism to fully capture local information, the first preset value may be set smaller. If the first preset value is smaller, the magnification ratio will be larger. In order to avoid an excessively large magnification ratio, in an embodiment of the present application, the calculation formula for the magnification ratio may be as follows:

[0172] / (E+F)

[0173] Wherein, C represents the magnification ratio; B represents the first characteristic difference value; E represents the first preset value; and F represents the first constant. In this embodiment, the first constant F can be set according to the requirements to increase the denominator, thereby avoiding excessive magnification ratio.

[0174] In another embodiment of the present application, the calculation formula of the magnification ratio may be as follows:

[0175] +G

[0176] Wherein, C represents the magnification ratio; B represents the first characteristic difference value; E represents the first preset value; G represents the second constant; Represents logarithmic value In this embodiment, the second constant G can be set according to the requirements, for example, G can be 1 or 2.

[0177] Step S412: Enlarging the current window based on the enlargement ratio.

[0178] It should be noted that in the embodiment of the present application, the enlarged current window needs to be rounded. For example, in theory, if the enlarged current window has 60.3 data, the current window size can be adjusted to a window with 60 data, or to a window with 61 data.

[0179] Step S420: If the absolute value of the first characteristic difference value is less than or equal to a second preset value, shrinking the current window.

[0180] In the embodiment of the present application, the reduction ratio can be obtained by a similar method as described above, and then the current window can be reduced by the reduction ratio. Of course, if the size of the current window is the same as the size of the initial window, the current window may not be reduced, that is, the reduction ratio is equal to 1.

[0181] In the embodiment of the present application, the second preset value can be set to any appropriate value according to the requirements. For example, in the application scenario of image recognition, if the feature value is a gray value, the second preset value can be set to 5 or 10, etc.

[0182] Step S500: If the first feature difference value satisfies a second preset condition, then the attention weight of the first data is obtained based on each second data.

[0183] In the embodiment of the present application, the second preset condition can be set to a suitable condition according to the requirements. For example, the second preset condition can be that the absolute value of the first characteristic difference value is greater than the second preset value and less than the first preset value.

[0184] In the prior art, the calculation steps for obtaining the attention weight of the first data based on each second data are as follows:

[0185] First, based on each second data, an input matrix is ​​obtained; then, based on the input matrix, a weight matrix (that is, a query (Query, Q) matrix, a key (Key, K) matrix or a value (Value, V) matrix) is obtained; finally, based on the weight matrix, the attention weight of the first data is obtained after multiple iterations. Since this process is a mature technology, it will not be described in detail. It should be noted that the weight matrix is ​​a matrix with the same dimension and size as the input matrix. The larger the input matrix, the larger the weight matrix. If the weight matrix is ​​larger, the amount of calculation for matrix multiplication will be larger in the process of calculating the attention weight, and the more storage space is required.

[0186] In order to further reduce the computational complexity in the process of calculating the attention weight and reduce the storage space required in the calculation process, in one embodiment of the present application, step S500, based on each second data, obtains the attention weight of the first data, including steps S510 to S540.

[0187] Step S510: Based on each second data, an input matrix is ​​obtained.

[0188] As can be seen from the foregoing, obtaining an input matrix based on each second data is a mature technology and will not be described in detail here.

[0189] Step S520: Based on the input matrix, obtain a dimensionality reduction decomposition matrix.

[0190] In the embodiment of the present application, based on the input matrix, obtaining the dimension reduction decomposition matrix refers to decomposing the weight matrix in the prior art to reduce the computational complexity in the process of calculating the attention weight and reduce the storage space required in the calculation process. In order to ensure that the decomposed weight matrix (for example, the first matrix (for example, Figure 6 P matrix in) and a second matrix (e.g., Figure 6 The product of the Q matrix in ) and the initial weight matrix (for example, Figure 6 In one embodiment of the present application, step S520, based on the input matrix, obtains a dimensionality reduction decomposition matrix, including steps S521 to S524.

[0191] Step S521: Based on the input matrix, obtain an initial weight matrix.

[0192] In this embodiment, the initial weight matrix is ​​a weight matrix obtained by inputting a matrix based on the prior art method. In this embodiment, the initial weight matrix can be any one of a query matrix, a key matrix or a value matrix.

[0193] Step S522: Based on the initial weight matrix, obtain sixth data.

[0194] In the present embodiment, the sixth data is obtained by the product of the first matrix and the second matrix. That is to say, in the present embodiment, the initial weight matrix is ​​decomposed into the form of the product of the first matrix and the second matrix. It is easy to understand that if the initial weight matrix is ​​decomposed into the form of the product of the first matrix and the second matrix, then the attention calculation is performed based on the low-dimensional first matrix and the second matrix, rather than the high-dimensional initial weight matrix. It should be noted that the low-dimensional first matrix and the second matrix contain a large amount of redundant information, which can be used for data recovery and feature extraction. In other words, the present embodiment can effectively reduce the amount of matrix multiplication calculation in the process of attention weight calculation through the first matrix and the second matrix, and can also greatly reduce the storage space required for weight matrix storage.

[0195] In order to ensure that the decomposed sixth data (ie, the product of the first matrix and the second matrix) has sufficient accuracy, that is, the difference between the initial weight matrix and the sixth data is small, in the embodiment of the present application, step S523 and step S524 are also required to be performed.

[0196] Step S523: Based on the initial weight matrix and the sixth data, obtain a second characteristic difference value.

[0197] In this embodiment, the second characteristic difference value refers to the difference value between the initial weight matrix and the sixth data. Based on two matrices, obtaining the difference value between the two matrices is a mature technology and will not be described in detail here.

[0198] Step S524: If the second feature difference value is greater than or equal to the third preset value, the first matrix and the second matrix are updated; otherwise, the sixth data is used as a dimensionality reduction decomposition matrix.

[0199] Specifically, in an embodiment of the present application, the dimensionality reduction decomposition matrix is ​​iteratively updated with the help of gradient changes through steps S523 and S524, so that the obtained dimensionality reduction decomposition matrix (that is, the first data iteratively updated multiple times) is extremely similar to the initial weight matrix, which ensures that subsequent calculations can have sufficient accuracy.

[0200] Step S530: Obtain an attention score based on the input matrix and the dimensionality reduction decomposition matrix.

[0201] It should be clear that calculating the attention score based on the input matrix and the weight matrix is ​​a mature technology and will not be described in detail here.

[0202] Step S540: Based on the attention score, obtain the attention weight of the first data.

[0203] It should be clear that calculating the attention weight of a certain data based on the attention score is also a mature technology and will not be elaborated here.

[0204] It should be clear that an embodiment of a self-attention calculation method proposed in this application selects only part of the input data to participate in the attention weight calculation of subsequent target data by setting a dynamic window (i.e., the current window) centered on the target data (i.e., the first data for which attention weight needs to be calculated in the above text). This can effectively reduce the complexity of attention weight calculation and ensure that the self-attention mechanism can capture local information of the input data.

[0205] After introducing an embodiment of a self-attention calculation method proposed in this application, the following introduces an embodiment of a self-attention calculation device proposed in this application. Figure 2 As shown, the self-attention calculation device 10 includes:

[0206] A reading module 11 is used to obtain first data and a current window based on sequence data; the sequence data is obtained in advance; the first data is any data in the sequence data; the current window is updated by a historical window;

[0207] Processing module 12, based on the current window, obtains a plurality of second data from the sequence data; the current window is centered on the first data;

[0208] and obtaining a first characteristic difference value of each second data;

[0209] And, if the first feature difference value meets a first preset condition, the current window is updated; if the first feature difference value meets a second preset condition, the attention weight of the first data is obtained based on each second data.

[0210] As a specific embodiment of the present application, the reading module 11 is further used to obtain third data and fourth data based on each second data; the third data and the fourth data are any two adjacent data in each second data;

[0211] The processing module 12 is further configured to obtain a first characteristic value based on the third data;

[0212] And, based on the fourth data, obtaining a second characteristic value;

[0213] And, based on the first feature value and the second feature value, a first feature difference value is obtained.

[0214] As a specific embodiment of the present application, the processing module 12 is further used to obtain fifth data based on each second data; the fifth data is any data in each second data;

[0215] And, based on the first data, obtaining a third characteristic value;

[0216] and, based on the fifth data, obtaining a fourth characteristic value;

[0217] And, based on the third feature value and the fourth feature value, a first feature difference value is obtained.

[0218] As a specific embodiment of the present application, the processing module 12 is further configured to, if the absolute value of the first feature difference value is greater than or equal to a first preset value, enlarge the current window;

[0219] and, if the absolute value of the first characteristic difference value is less than or equal to a second preset value, reducing the current window;

[0220] The second preset condition includes: an absolute value of the first characteristic difference value is greater than a second preset value and less than a first preset value.

[0221] As a specific embodiment of the present application, the processing module 12 is further used to obtain an absolute value of a difference between the first eigenvalue and the second eigenvalue;

[0222] and, based on the absolute values ​​of the respective differences, obtaining an average value;

[0223] And, based on the average value, obtaining a first characteristic difference value;

[0224] The processing module 12 is further configured to obtain an amplification ratio based on the first characteristic difference value and the first preset value;

[0225] And, based on the magnification ratio, the current window is magnified.

[0226] As a specific embodiment of the present application, the processing module 12 is further used to obtain an input matrix based on each second data;

[0227] And, based on the input matrix, obtaining a dimension reduction decomposition matrix;

[0228] And, obtaining an attention score based on the input matrix and the dimensionality reduction decomposition matrix;

[0229] And, based on the attention score, obtaining the attention weight of the first data.

[0230] As a specific embodiment of the present application, the processing module 12 is further used to obtain an initial weight matrix based on the input matrix;

[0231] And, based on the initial weight matrix, obtaining sixth data; the sixth data is obtained by multiplying the first matrix and the second matrix;

[0232] and, based on the initial weight matrix and the sixth data, obtaining a second characteristic difference value;

[0233] And, if the second characteristic difference value is greater than or equal to a third preset value, the first matrix and the second matrix are updated; otherwise, the sixth data is used as a dimensionality reduction decomposition matrix.

[0234] It should be clear that the embodiment of the self-attention calculation device proposed in this application, by setting a dynamic window (i.e., the current window) centered on the target data (i.e., the first data for which attention weight needs to be calculated in the above text), only selects part of the input data to participate in the attention weight calculation of the subsequent target data. This can effectively reduce the complexity of attention weight calculation and ensure that the self-attention mechanism can capture local information of the input data.

[0235] After introducing the embodiment of the self-attention computing device proposed in this application, the following introduces an embodiment of a self-attention computing system proposed in this application. Figure 3 As shown, the self-attention calculation system 20 includes:

[0236] The reader 21 is used to obtain first data and a current window based on sequence data; the sequence data is obtained in advance; the first data is any data in the sequence data; the current window is updated by the historical window;

[0237] The processor 22 acquires a plurality of second data from the sequence data based on the current window; the current window is centered on the first data;

[0238] and obtaining a first characteristic difference value of each second data;

[0239] And, if the first feature difference value meets a first preset condition, the current window is updated; if the first feature difference value meets a second preset condition, the attention weight of the first data is obtained based on each second data.

[0240] As a specific embodiment of the present application, the reader 21 is further used to obtain third data and fourth data based on each second data; the third data and the fourth data are any two adjacent data in each second data;

[0241] The processor 22 is further configured to obtain a first characteristic value based on the third data;

[0242] And, based on the fourth data, obtaining a second characteristic value;

[0243] And, based on the first feature value and the second feature value, a first feature difference value is obtained.

[0244] As a specific embodiment of the present application, the processor 22 is further configured to obtain fifth data based on each second data; the fifth data is any data in each second data;

[0245] And, based on the first data, obtaining a third characteristic value;

[0246] and, based on the fifth data, obtaining a fourth characteristic value;

[0247] And, based on the third feature value and the fourth feature value, a first feature difference value is obtained.

[0248] As a specific embodiment of the present application, the processor 22 is further configured to, if the absolute value of the first feature difference value is greater than or equal to a first preset value, enlarge the current window;

[0249] and, if the absolute value of the first characteristic difference value is less than or equal to a second preset value, reducing the current window;

[0250] The second preset condition includes: an absolute value of the first characteristic difference value is greater than a second preset value and less than a first preset value.

[0251] As a specific embodiment of the present application, the processor 22 is further configured to obtain an absolute value of a difference between the first eigenvalue and the second eigenvalue;

[0252] and, based on the absolute values ​​of the respective differences, obtaining an average value;

[0253] And, based on the average value, obtaining a first characteristic difference value;

[0254] The processor 22 is further configured to obtain a magnification ratio based on the first characteristic difference value and the first preset value;

[0255] And, based on the magnification ratio, the current window is magnified.

[0256] As a specific embodiment of the present application, the processor 22 is further configured to obtain an input matrix based on each second data;

[0257] And, based on the input matrix, obtaining a dimension reduction decomposition matrix;

[0258] And, obtaining an attention score based on the input matrix and the dimensionality reduction decomposition matrix;

[0259] And, based on the attention score, obtaining the attention weight of the first data.

[0260] As a specific embodiment of the present application, the processor 22 is further configured to obtain an initial weight matrix based on the input matrix;

[0261] And, based on the initial weight matrix, obtaining sixth data; the sixth data is obtained by multiplying the first matrix and the second matrix;

[0262] and, based on the initial weight matrix and the sixth data, obtaining a second characteristic difference value;

[0263] And, if the second characteristic difference value is greater than or equal to a third preset value, the first matrix and the second matrix are updated; otherwise, the sixth data is used as a dimensionality reduction decomposition matrix.

[0264] It should be clear that the embodiment of the self-attention calculation system proposed in this application, by setting a dynamic window (i.e., the current window) centered on the target data (i.e., the first data for which attention weight needs to be calculated in the above text), only selects part of the input data to participate in the attention weight calculation of the subsequent target data. This can effectively reduce the complexity of attention weight calculation and ensure that the self-attention mechanism can capture local information of the input data.

[0265] After introducing an embodiment of a self-attention calculation system proposed in the present application, the following introduces a computer-readable storage medium proposed in the present application, on which a computer program is stored. When the computer program is executed by a processor, a self-attention calculation method as described in any of the above embodiments is implemented.

[0266] It should be clear that the computer-readable storage media in this application include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0267] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0268] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the methods, devices and equipment described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0269] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0270] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0271] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0272] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0273] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center to another website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0274] Although the embodiments of the present application have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles of the present application.

Claims

1. A self-attention calculation method, characterized in that: include: Based on the sequence data, obtain the first data and the current window; The sequence data is acquired in advance, and the sequence data is converted from video data or image data; The first data is any data in the sequence data; The current window is updated by the historical window; Acquire a plurality of second data from the sequence data based on the current window; the current window is centered on the first data, and the plurality of second data are all located within the current window; Obtaining a first characteristic difference value of each second data; If the first feature difference value satisfies a first preset condition, updating the current window; if the first feature difference value satisfies a second preset condition, obtaining the attention weight of the first data based on each second data; The obtaining of the first characteristic difference value of each second data includes: Based on each second data, third data and fourth data are acquired; the third data and the fourth data are any two adjacent data in each second data; Based on the third data, obtaining a first characteristic value; Based on the fourth data, obtaining a second characteristic value; Based on the first feature value and the second feature value, a first feature difference value is obtained. Alternatively, the obtaining of the first characteristic difference value of each second data includes: Based on each second data, fifth data is obtained; the fifth data is any data in each second data; Based on the first data, obtaining a third characteristic value; Based on the fifth data, obtaining a fourth characteristic value; Based on the third eigenvalue and the fourth eigenvalue, obtaining a first eigenvalue difference value; If the first feature difference value satisfies a first preset condition, updating the current window includes: If the absolute value of the first characteristic difference value is greater than or equal to a first preset value, enlarging the current window; If the absolute value of the first characteristic difference value is less than or equal to a second preset value, shrinking the current window; The second preset condition includes: the absolute value of the first characteristic difference value is greater than the second preset value and less than the first preset value; The obtaining a first feature difference value based on the first feature value and the second feature value includes: Obtaining an absolute value of a difference between the first eigenvalue and the second eigenvalue; Based on the absolute values ​​of each difference, the average value is obtained; Based on the average value, obtaining a first characteristic difference value; If the absolute value of the first characteristic difference value is greater than or equal to a first preset value, enlarging the current window includes: Based on the first characteristic difference value and the first preset value, obtaining a magnification ratio; The current window is enlarged based on the enlargement ratio.

2. The self-attention calculation method according to claim 1, characterized in that: The acquiring the attention weight of the first data based on each second data includes: Based on each second data, obtaining an input matrix; Based on the input matrix, obtaining a dimensionality reduction decomposition matrix; Obtaining an attention score based on the input matrix and the dimensionality reduction decomposition matrix; Based on the attention score, an attention weight of the first data is obtained.

3. The self-attention calculation method according to claim 2, characterized in that: The step of obtaining a dimensionality reduction decomposition matrix based on the input matrix includes: Based on the input matrix, obtaining an initial weight matrix; Based on the initial weight matrix, obtaining sixth data; the sixth data is obtained by multiplying the first matrix and the second matrix; Based on the initial weight matrix and the sixth data, obtaining a second characteristic difference value; If the second characteristic difference value is greater than or equal to a third preset value, the first matrix and the second matrix are updated; otherwise, the sixth data is used as a dimensionality reduction decomposition matrix.

4. A self-attention computing device, characterized in that: include: A reading module, used for acquiring first data and a current window based on sequence data; The sequence data is acquired in advance, and the sequence data is converted from video data or image data; the first data is any data in the sequence data; The current window is updated by the historical window; A processing module, based on the current window, obtains a plurality of second data from the sequence data; the current window is centered on the first data; and obtaining a first characteristic difference value of each second data; And, if the first feature difference value satisfies a first preset condition, updating the current window; if the first feature difference value satisfies a second preset condition, acquiring the attention weight of the first data based on each second data; The reading module is further used to obtain third data and fourth data based on each second data; the third data and the fourth data are any two adjacent data in each second data; The processing module is further used to obtain a first characteristic value based on the third data; And, based on the fourth data, obtaining a second characteristic value; And, based on the first feature value and the second feature value, obtaining a first feature difference value; Alternatively, the processing module is further used to obtain fifth data based on each second data; the fifth data is any data in each second data; And, based on the first data, obtaining a third characteristic value; and, based on the fifth data, obtaining a fourth characteristic value; And, based on the third eigenvalue and the fourth eigenvalue, obtaining a first eigenvalue difference value; The processing module is further configured to, if the absolute value of the first characteristic difference value is greater than or equal to a first preset value, enlarge the current window; and, if the absolute value of the first characteristic difference value is less than or equal to a second preset value, reducing the current window; The second preset condition includes: the absolute value of the first characteristic difference value is greater than the second preset value and less than the first preset value; The processing module is further used to obtain an absolute value of a difference between the first eigenvalue and the second eigenvalue; and, based on the absolute values ​​of the respective differences, obtaining an average value; And, based on the average value, obtaining a first characteristic difference value; The processing module is further configured to obtain an amplification ratio based on the first characteristic difference value and the first preset value; And, based on the magnification ratio, the current window is magnified.

5. The self-attention computing device according to claim 4, characterized in that: The processing module is further used to obtain an input matrix based on each second data; And, based on the input matrix, obtaining a dimension reduction decomposition matrix; And, obtaining an attention score based on the input matrix and the dimensionality reduction decomposition matrix; And, based on the attention score, obtaining the attention weight of the first data.

6. The self-attention computing device according to claim 5, characterized in that: The processing module is further used to obtain an initial weight matrix based on the input matrix; And, based on the initial weight matrix, obtaining sixth data; the sixth data is obtained by multiplying the first matrix and the second matrix; and, based on the initial weight matrix and the sixth data, obtaining a second characteristic difference value; And, if the second characteristic difference value is greater than or equal to a third preset value, the first matrix and the second matrix are updated; otherwise, the sixth data is used as a dimensionality reduction decomposition matrix.

7. A self-attention computing system, characterized in that: include: A reader, configured to obtain first data and a current window based on sequence data; the sequence data is obtained in advance and is converted from video data or image data; the first data is any data in the sequence data; The current window is updated by the historical window; The processor acquires a plurality of second data from the sequence data based on the current window; the current window is centered on the first data; and obtaining a first characteristic difference value of each second data; And, if the first feature difference value satisfies a first preset condition, updating the current window; if the first feature difference value satisfies a second preset condition, acquiring the attention weight of the first data based on each second data; The reader is further used to obtain third data and fourth data based on each second data; the third data and the fourth data are any two adjacent data in each second data; The processor is further configured to obtain a first characteristic value based on the third data; And, based on the fourth data, obtaining a second characteristic value; And, based on the first feature value and the second feature value, obtaining a first feature difference value; Alternatively, the processor is further configured to obtain fifth data based on each second data; the fifth data is any data in each second data; And, based on the first data, obtaining a third characteristic value; and, based on the fifth data, obtaining a fourth characteristic value; And, based on the third eigenvalue and the fourth eigenvalue, obtaining a first eigenvalue difference value; The processor is further configured to, if the absolute value of the first feature difference value is greater than or equal to a first preset value, enlarge the current window; and, if the absolute value of the first characteristic difference value is less than or equal to a second preset value, reducing the current window; The second preset condition includes: the absolute value of the first characteristic difference value is greater than the second preset value and less than the first preset value The processor is further configured to obtain an absolute value of a difference between the first eigenvalue and the second eigenvalue; and, based on the absolute values ​​of the respective differences, obtaining an average value; And, based on the average value, obtaining a first characteristic difference value; The processor is further configured to obtain a magnification ratio based on the first characteristic difference value and the first preset value; And, based on the magnification ratio, the current window is magnified.

8. The self-attention computing system according to claim 7, characterized in that: The processor is further configured to obtain an input matrix based on each second data; And, based on the input matrix, obtaining a dimension reduction decomposition matrix; And, obtaining an attention score based on the input matrix and the dimensionality reduction decomposition matrix; And, based on the attention score, obtaining the attention weight of the first data.

9. The self-attention computing system according to claim 8, characterized in that The processor is further configured to obtain an initial weight matrix based on the input matrix; And, based on the initial weight matrix, obtaining sixth data; the sixth data is obtained by multiplying the first matrix and the second matrix; and, based on the initial weight matrix and the sixth data, obtaining a second characteristic difference value; And, if the second characteristic difference value is greater than or equal to a third preset value, the first matrix and the second matrix are updated; otherwise, the sixth data is used as a dimensionality reduction decomposition matrix.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, a self-attention calculation method as described in any one of claims 1 to 3 is implemented.

Images