Task execution method and device, electronic equipment and storage medium

Abstract

The invention provides a task execution method, and relates to the technical field of artificial intelligence, in particular to the technical fields of large language models, image processing, text processing, audio processing and the like. According to the specific implementation scheme, a first multiplication task of a target attention task queue is executed according to a first query feature and a first key feature, and a first multiplication result feature is obtained; according to the first multiplication result feature and a target threshold feature, a to-be-processed feature is determined through the calculation unit, the target threshold feature is obtained according to a second query feature and a second key feature, and the second query feature is obtained by conducting data type conversion on the first query feature; the second key feature is obtained by performing data type conversion on the first key feature; and according to the to-be-processed feature and the first value feature, executing a plurality of target tasks of the target attention task queue by using the computing unit to obtain an attention feature. The invention further provides a task execution device, electronic equipment and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the fields of large language model (LLM), image processing, text processing, audio processing, etc. More specifically, the present disclosure provides a task execution method, device, electronic device, and storage medium. Background Art

[0002] With the development of artificial intelligence technology, the number of parameters in deep learning models is increasing. The computing resource overhead and storage resource overhead required for deep learning model training or reasoning are also increasing. Summary of the invention

[0003] The present disclosure provides a task execution method, apparatus, device and storage medium.

[0004] According to one aspect of the present disclosure, a task execution method is provided, the method comprising: according to a first query feature and a first key feature, using a computing unit to execute a first multiplication task of a target attention task queue to obtain a first multiplication result feature; according to the first multiplication result feature and a target threshold feature, using a computing unit to determine a feature to be processed, wherein the target threshold feature is obtained according to a second query feature and a second key feature, the second query feature is obtained by performing a data type conversion on the first query feature, and the second key feature is obtained by performing a data type conversion on the first key feature; and according to the feature to be processed and the first value feature, using a computing unit to execute multiple target tasks of the target attention task queue to obtain an attention feature.

[0005] According to another aspect of the present disclosure, a task execution device is provided, which includes: a first execution module, which is used to use a computing unit to execute a first multiplication task of a target attention task queue according to a first query feature and a first key feature to obtain a first multiplication result feature; a determination module, which is used to use a computing unit to determine a feature to be processed according to the first multiplication result feature and a target threshold feature, wherein the target threshold feature is obtained according to a second query feature and a second key feature, the second query feature is obtained by performing a data type conversion on the first query feature, and the second key feature is obtained by performing a data type conversion on the first key feature; and a second execution module, which is used to use a computing unit to execute multiple target tasks of the target attention task queue according to the feature to be processed and the first value feature to obtain an attention feature.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method provided according to the present disclosure is implemented.

[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is a flowchart of a task execution method according to an embodiment of the present disclosure;

[0012] Figure 2A is a schematic diagram of data distribution of a first query feature according to an embodiment of the present disclosure;

[0013] Figure 2B is a schematic diagram of data distribution of a first key feature according to an embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram of a task execution method according to an embodiment of the present disclosure;

[0015] Figure 4 is a block diagram of a task execution device according to an embodiment of the present disclosure; and

[0016] Figure 5 is a block diagram of an electronic device to which a task execution method can be applied according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0018] Large models can include one or more Transformer networks. Transformer networks can process data based on the attention mechanism. The implementation process of the attention mechanism includes the generation of query features, key features, and value feature matrices, and processing using the softmax function to obtain attention weights. With the increase of model parameters and data volume, the computational complexity of the attention mechanism and the memory usage during the calculation process gradually become bottlenecks for model reasoning or training. When processing data based on the attention mechanism, 32-bit floating point numbers (FP32) can be used for precision calculations to ensure numerical stability. In the case of calculations based on 32-bit floating point numbers, the computing resource overhead is large, resulting in low efficiency in model training or reasoning.

[0019] Calculation based on 16-bit floating point numbers (FP16) can increase the speed of training or inference while reducing the computing resource overhead. As a result, 16-bit floating point data is widely used in accelerated calculations for deep learning. However, 16-bit floating point numbers have low precision. If the type of data used in the above-mentioned soft maximum function processing is converted to 16-bit floating point numbers, problems such as data overflow and gradient vanishing may occur, which may lead to numerical instability. This numerical instability phenomenon is particularly significant during the training phase of large models. For example, during training, when processing based on the attention mechanism, the activation values ​​obtained are likely to have data overflow problems. If the overflow value is replaced with 0, the activation value will fluctuate. In addition, in large text models and large image models, the channel dimension of the key feature matrix is ​​likely to have a large number of outliers.

[0020] Therefore, in order to reduce the hardware resource overhead of model training or reasoning, the present disclosure provides a task execution method, which will be described below.

[0021] Figure 1 is a flowchart of a task execution method according to an embodiment of the present disclosure.

[0022] like Figure 1 As shown, the method 200 may include operations S110 to S130.

[0023] In operation S110, a first multiplication task of a target attention task queue is executed by a computing unit according to the first query feature and the first key feature to obtain a first multiplication result feature.

[0024] In the embodiment of the present disclosure, the target attention task queue may be a task queue for processing data based on an attention mechanism. The attention mechanism may be a multi-head self-attention mechanism.

[0025] In the embodiment of the present disclosure, the first query feature may be obtained by multiplying the input feature by the query weight matrix. The first key feature may be obtained by multiplying the input feature by the key weight matrix.

[0026] In the embodiment of the present disclosure, the operation indicated by the first multiplication task is: multiplying the first query matrix and the first key transposed feature. The first key transposed feature is obtained by transposing the first key feature.

[0027] In the embodiments of the present disclosure, the computing unit may be an artificial intelligence computing unit, which may be a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), a neural network processing unit (NPU), a Kunlun core, or other hardware computing units.

[0028] In operation S120, a feature to be processed is determined by a computing unit according to the first multiplication result feature and the target threshold feature.

[0029] In the disclosed embodiment, the second query feature is obtained by converting the data type of the first query feature, and the second key feature is obtained by converting the data type of the first key feature. For example, the precision of the feature value of the first query feature may be higher than the precision of the feature value of the second query feature. The precision of the feature value of the first key feature may be higher than the precision of the feature value of the second key feature. Taking the data type of the first query feature as an example of a 32-bit floating point number, the data type of the second query feature may be a 16-bit floating point number. Taking the data type of the first key feature as an example of a 32-bit floating point number, the data type of the second key feature may be a 16-bit floating point number. For another example, the data type may be various types such as a 32-bit floating point number, a 16-bit floating point number, a 32-bit integer (INT32), and a 16-bit integer (INT16).

[0030] In the disclosed embodiment, the target threshold feature is obtained based on the second query feature and the second key feature. For example, the target threshold feature can be obtained by multiplying the second query feature and the second key transposed feature. The second key transposed feature can be obtained by transposing the second key feature.

[0031] In the disclosed embodiment, the first product result matrix feature can be adjusted using the target threshold feature to obtain the feature to be processed. Multiple feature values ​​in the feature to be processed can be within a preset numerical range to reduce the risk of data overflow. For example, if the data type of the second key feature is a 16-bit floating point number, the preset numerical range can be -65504~65504.

[0032] In operation S130, a plurality of target tasks of a target attention task queue are executed by a computing unit according to the feature to be processed and the first value feature to obtain an attention feature.

[0033] In an embodiment of the present disclosure, the multiple target tasks of the target attention task queue may include a scaling task, a preset function processing task, and a second multiplication task. The multiple target tasks of the target attention task queue may also include a mask processing task. It can be understood that based on the multi-head self-attention mechanism, the mask processing task may be optional.

[0034] Through the disclosed embodiment, the features to be processed are determined according to the target threshold feature and the first multiplication result feature, which reduces the risk of precision overflow when processing data based on the attention mechanism, and can effectively improve the accuracy of the chip when processing data based on the attention mechanism. The target threshold feature is obtained based on the query feature and key feature after the data type conversion, which can enable the chip to stably perform calculations based on the attention mechanism under lower precision data types.

[0035] It can be understood that the method of the present disclosure is described above, and the first query feature and the first key feature of the present disclosure will be described below.

[0036] Figure 2A It is a schematic diagram of data distribution of the first query feature according to an embodiment of the present disclosure.

[0037] In the embodiment of the present disclosure, the first query feature may be a feature of multiple dimensions. Taking the query feature of an image as an example, the multiple dimensions of the first query feature may include a channel dimension (c), a width dimension (w), and a height dimension (h). Figure 2A As shown, in the channel dimension, there are almost no abnormal feature values ​​in the first query feature Q211. The data distribution of the first query feature Q211 is close to a normal distribution.

[0038] Figure 2B It is a schematic diagram of data distribution of the first key feature according to an embodiment of the present disclosure.

[0039] In the embodiment of the present disclosure, the first key feature may be a feature of multiple dimensions. Taking the key feature of an image as an example, the multiple dimensions of the first key feature may include a channel dimension, a width dimension, and a height dimension. Figure 2B As shown, in the channel dimension, the first key feature K211 may include more abnormal values. For example, the frequency corresponding to the eigenvalue 0 of the first key feature K211 is significantly higher than the frequencies corresponding to other eigenvalues.

[0040] In an embodiment of the present disclosure, the data type of the first query feature may be the first data type. The data type of the first key feature may be the first data type. The data type of the first value feature may be the first data type. The data type of the second query feature may be the second data type, and the data type of the second key feature may be the second data type. The amount of data of the first query feature may be greater than the amount of data of the second query feature. The amount of data of the first key feature may be greater than the amount of data of the second key feature. The first data type may be, for example, a 32-bit floating point number. The second data type may be a 16-bit floating point number.

[0041] It can be understood that the first query feature and the first key feature of the present disclosure are described above, and some methods of performing the first multiplication operation task will be described below.

[0042] In some embodiments, in some embodiments of the above operation S110, according to the first query feature and the first key feature, the computing unit can be used to execute the first multiplication task of the target attention task queue to obtain the first multiplication result feature. For example, the data type of the feature value of the first query feature can be a 32-bit floating point number. The data type of the feature value of the first key feature can be a 32-bit floating point number. The first multiplication task can be a matrix multiplication task. The data type of the feature value of the first multiplication result feature can also be a 32-bit floating point number.

[0043] In the embodiment of the present disclosure, the calculation unit may perform the following operations: multiply the first query feature by the first key transposition feature to obtain a first multiplication result feature. The first key transposition feature may be obtained by transposing the first key feature. For example, the first multiplication result feature score may be determined using the following formula:

[0044] (Formula 1)

[0045] Q can be the first query feature. K T Features can be transposed for the first key.

[0046] It can be understood that the first multiplication task of the present disclosure is described above, and some methods of obtaining the second query feature and the second key feature will be described below.

[0047] In some embodiments, according to the first query feature and the first key feature, the second query feature and the second key feature can be obtained in various ways.

[0048] In an embodiment of the present disclosure, a type conversion function (cast) may be used to process the first query feature and the first key feature respectively to obtain the second query feature and the second key feature.

[0049] In another embodiment of the present disclosure, the second query feature may be obtained by multiplying the first query feature by a scaling parameter. The second key feature may be obtained by multiplying the first key feature by a scaling parameter. For example, multiple feature values ​​of the first query feature may be multiplied by the scaling parameter to obtain the second query feature. Multiple feature values ​​of the first key feature may be multiplied by the scaling parameter to obtain the second key feature. The scaling parameter may be determined in various ways, and some ways of determining the scaling parameter will be described below.

[0050] In an embodiment of the present disclosure, the scaling parameter is obtained based on a first maximum value and a second maximum value, and the first maximum value may be a maximum value in the execution result of the feedforward task queue before the target attention task queue. The second maximum value may be a maximum value representable by the second data type. For example, in the case where a large model includes multiple cascaded Transformer networks, the target attention task queue may include multiple tasks to be executed by the attention encoding module (Transformer encoder) of the target Transformer network. The previous Transformer network before the target Transformer network may include an attention encoding module and a feedforward (FFN) module. The feedforward module of the previous Transformer network includes multiple fully connected layers. Among the multiple values ​​output by the multiple fully connected layers, the maximum value may be used as the first maximum value. For another example, if the second data type is a 16-bit floating point number, the second maximum value may be the maximum value representable by the second data type. That is, the second maximum value may be 65504.

[0051] In the disclosed embodiment, the scaling parameter may be obtained by dividing the processed second maximum value by the first maximum value, the processed second maximum value is obtained by taking the square root of the adjusted second maximum value, and the adjusted second maximum value is obtained by dividing the second maximum value by the attention head dimension of the target attention task queue. For example, the second maximum value may be used as the dividend and the attention head dimension (head_dim) as the divisor to perform a division operation to obtain the adjusted second maximum value. The adjusted second maximum value may be subjected to a square root operation to determine the square root of the adjusted second maximum value as the processed second maximum value. Next, the processed second maximum value may be used as the dividend and the first maximum value as the divisor to perform a division operation to obtain the scaling parameter. For another example, the scaling parameter scale may be determined by the following formula:

[0052] (Formula 2)

[0053] history_max can be the first maximum value.

[0054] The second query feature Q' and the second key feature K' can also be determined by the following formula:

[0055] (Formula 3)

[0056] (Formula 4)

[0057] Q may be the first query feature, and K may be the first key feature.

[0058] Through the disclosed embodiment, the scaling parameter is determined by utilizing the maximum value of the previous feedforward task queue, and a dynamic adjustment mechanism is implemented, which can effectively reduce the risk of precision overflow and improve the stability of the chip.

[0059] It can be understood that the above describes some methods for determining the second query feature and the second key feature, and the following describes some methods for obtaining the target threshold feature.

[0060] In some embodiments, the target threshold feature may be obtained based on the second multiplication result feature. The second multiplication result feature may be obtained based on the second query feature and the second key feature, as will be described below.

[0061] In the disclosed embodiment, the second multiplication result feature is obtained by multiplying the second query feature with the processed second key transposed feature, the processed second key transposed feature is obtained by transposing the processed second key feature, the processed second key feature is obtained by subtracting the second key feature from the first median feature, and the first median feature is obtained by determining the median of the second key feature along the channel dimension.

[0062] For example, the second key feature may be a feature of multiple dimensions. Taking the key feature of an image as an example, the multiple dimensions of the second key feature may include a channel dimension, a width dimension, and a height dimension. The median of the second key feature may be determined along the channel dimension to obtain a first median feature. The second key feature may be used as a minuend and the first median feature as a subtrahend to perform a subtraction operation to obtain a processed second key feature. For another example, the processed second key feature may be determined by the following formula: :

[0063] (Formula 5)

[0064] A function that finds the median along the channel dimension.

[0065] Through the embodiment of the present disclosure, the second key feature is processed by using the first median feature, which can effectively reduce the impact of outliers in the second key feature on training or reasoning, improve the stability of chip operation, and also help reduce the impact of precision overflow on calculation accuracy.

[0066] Next, a second multiplication result feature is determined based on the processed second key feature and the second query feature. For example, the processed second key feature may be transposed to obtain a processed second key transposed feature. The processed second key transposed feature is multiplied by the second query feature to obtain a second multiplication result feature.

[0067] It can be understood that some methods of determining the characteristics of the second multiplication result are described above, and some methods of determining the characteristics of the target threshold value will be described below.

[0068] In the embodiment of the present disclosure, the second multiplication result can be processed by the maximum value function to obtain the first function processing result feature. For example, the first function processing result feature F_1 can be obtained by the following formula:

[0069] (Formula 6)

[0070] Features may be transposed for the processed second key.

[0071] In the disclosed embodiment, the second multiplication result feature can also be processed by a minimum function to obtain a second function processing result feature. The difference between the first function processing result feature and the second function processing result feature can be processed by an exponential function to obtain an exponential feature. The sum of the exponential feature and the preset feature can be processed by a logarithmic function to obtain a logarithmic feature. The first function processing feature and the logarithmic feature are added to obtain an addition result feature. For example, the preset feature can include a preset value, and the preset value can be 1. The addition result feature F_add can be obtained by the following formula:

[0072] (Formula 7)

[0073] max() can be a maximum value function. min() can be a minimum value function. log() can be a logarithmic function. exp() can be an exponential function.

[0074] In the disclosed embodiment, the above-mentioned addition result feature can be used as the target threshold feature, thereby effectively reducing the risk of data overflow and data underflow and improving the stability of the chip.

[0075] In the disclosed embodiment, the first function processing result feature may also be used as the target threshold feature, thereby effectively reducing the risk of data overflow and improving chip computing efficiency.

[0076] It can be understood that some methods for determining target threshold features are described above, and some methods for determining features to be processed will be described below.

[0077] In some embodiments, in some implementations of the above operation S120, determining the feature to be processed using a computing unit according to the first multiplication result feature and the target threshold feature may include: determining an intermediate feature using a computing unit according to the first multiplication result feature and the target threshold feature. For example, the computing unit may be used to subtract the first multiplication result feature from the target threshold feature to obtain an intermediate feature. For another example, the computing unit may be used to perform an operation corresponding to the following formula:

[0078] (Formula 8)

[0079] score' may be an intermediate feature. score_threshold may be a target threshold feature. It can be understood that the target threshold feature score_threshold may be the above-mentioned addition result feature or the first function processing result feature.

[0080] In some embodiments, in some implementations of the above operation S120, determining the feature to be processed using a computing unit according to the first multiplication result feature and the target threshold feature may include: adjusting the intermediate feature using the computing unit according to a scaling parameter to obtain the feature to be processed.

[0081] In the embodiment of the present disclosure, the calculation unit may be used to perform the following operations: the intermediate feature is divided by the processed scaling parameter to obtain the feature to be processed. The processed scaling parameter may be obtained by squaring the scaling parameter. For example, the intermediate feature may be used as the dividend and the processed scaling parameter may be used as the divisor to perform a division operation to obtain the feature to be processed. The feature to be processed P may be obtained using the following formula:

[0082] (Formula 9)

[0083] scale 2 It can be a processed scaling parameter. Through the embodiment of the present disclosure, the intermediate features can be converted into probability distribution to obtain the features to be processed, so as to continue processing based on the attention mechanism.

[0084] It can be understood that the above describes some methods for determining features to be processed, and the following describes some methods for executing the target attention task queue.

[0085] In some embodiments, the multiple target tasks of the target attention task queue may also include a scaling task, a preset function processing task, and a second multiplication task. Figure 3 Provide further explanation.

[0086] Figure 3 is a schematic diagram of a task execution method according to an embodiment of the present disclosure.

[0087] like Figure 3 As shown, by multiplying the first query feature Q311 by the scaling parameter, the second query feature Q321 can be obtained. By multiplying the first key feature K311 by the scaling parameter, the second key feature K321 can be obtained. The data type of the first query feature Q311 can be, for example, a 32-bit floating point number. The data type of the second query feature Q321 can be, for example, a 16-bit floating point number. The data type of the first key feature K311 can be, for example, a 32-bit floating point number. The data type of the second key feature K321 can be, for example, a 16-bit floating point number.

[0088] like Figure 3 As shown, the above operation S110 can be performed to obtain the first multiplication result feature score31 according to the first query feature Q311 and the first key feature K311. It can be understood that the above description of operation S110 is also applicable to this embodiment, and the disclosure will not repeat it here.

[0089] like Figure 3 As shown, the processed second key transposed feature Kt322 can be obtained according to the second key feature K321. For example, as mentioned above, the processed second key transposed feature Kt322 is obtained by transposing the processed second key feature, and the processed second key feature is obtained by subtracting the second key feature from the first median feature, and the first median feature is obtained by determining the median of the second key feature along the channel dimension. Next, the second query feature Q321 can be multiplied by the processed second key transposed feature Kt322 to obtain a second multiplication result feature. According to the second multiplication result feature, the target threshold feature score_th32 can be obtained. It can be understood that the above description of the target threshold feature is also applicable to the target threshold feature score_th32, and the present disclosure will not repeat it here.

[0090] Next, the above operation S120 may be performed to obtain the feature to be processed P30 according to the first multiplication result feature score31 and the target threshold feature score_th32. The data type of the feature to be processed P30 may be a 16-bit floating point number.

[0091] In some embodiments, in some implementations of the above operation S130, according to the feature to be processed and the first value feature, using the computing unit to execute multiple target tasks of the target attention task queue to obtain the attention feature may include: according to the dimension of the feature to be processed and the first key feature, using the computing unit to execute a scaling task to obtain the first processed feature. Figure 3 As shown, the calculation unit may perform the following operations: divide the feature to be processed P30 by the square root of the dimension of the first key feature K311 to obtain the first processed feature F31.

[0092] In some embodiments, in some implementations of the above operation S130, according to the feature to be processed and the first value feature, using the computing unit to execute multiple target tasks of the target attention task queue to obtain the attention feature may include: using the computing unit to execute a preset function processing task to process the first processed feature using the preset function to obtain the second processed feature. For example, the preset function may be the above-mentioned soft maximum function. The first processed feature F31 may be processed using the preset function to obtain the second processed feature F32.

[0093] In some embodiments, in some implementations of the above operation S130, according to the feature to be processed and the first value feature, using the computing unit to execute multiple target tasks of the target attention task queue, obtaining the attention feature may include: according to the third processed feature and the first value feature, using the computing unit to execute the second multiplication task to obtain the attention feature. The third processed feature is obtained by converting the data type of the second processed feature. For example, the data type of the second processed feature F32 can be a 16-bit floating point number. The data type of the first value feature can be a 32-bit floating point number. The second multiplication task can be a matrix multiplication task. The second processed feature can be converted into a data type to obtain a third processed feature of a 32-bit floating point number. The third processed feature is multiplied by the first value feature to obtain the attention feature att30.

[0094] In some embodiments, in some implementations of the above operation S130, according to the feature to be processed and the first value feature, using the computing unit to execute multiple target tasks of the target attention task queue to obtain the attention feature may include: using the computing unit to implement the following formula:

[0095] (Formula 10)

[0096] Att can be the attention feature. Softmax() is the softmax function, which can be used as a preset function. V can be the first value feature. d_k can be the dimension of the first value feature. T() can be a data type conversion function, for example, a 16-bit floating point number can be converted to a 32-bit floating point number. It can be understood that The above-mentioned first processed feature can be represented. A second processed feature may be represented. The above-mentioned third processed feature can be represented.

[0097] Through the disclosed embodiments, the chip can implement preset function processing at a relatively low precision, which can effectively improve computing efficiency and reduce computing resource overhead. In the process of preset function processing, data overflow can be effectively avoided, and the stability of the chip's attention mechanism is improved. In particular, in the model training process, efficient computing can be achieved, and the model training efficiency can be effectively improved, effectively reducing hardware resource overhead.

[0098] It can be understood that the above description uses the case where an outlier exists in the first key feature as an example to illustrate the present disclosure, but the present disclosure is not limited thereto, and the method of the present disclosure can also be used in the case where an outlier exists in the first query feature, which will be described below.

[0099] In another embodiment of the present disclosure, the target threshold feature is obtained based on the third multiplication result feature, the third multiplication result feature is obtained by multiplying the processed second query feature with the second key transposition feature, the processed second query feature is obtained by subtracting the second query feature from the second median feature, the second median feature is obtained by determining the median of the second query feature along the channel dimension, and the second key transposition feature is obtained by transposing the second key feature. For example, the second query feature can be used as the minuend and the second median feature as the subtrahend to perform a subtraction operation to obtain the processed second query feature. It can be understood that the method for determining the processed second query feature is the same or similar to the method for determining the above-mentioned processed second key feature, and the present disclosure will not repeat it here. It can also be understood that the above method for obtaining the target threshold feature based on the second multiplication result feature is also applicable to obtaining the target threshold feature based on the third multiplication result feature, and the present disclosure will not repeat it here.

[0100] It is understood that after executing multiple tasks of the target attention task queue, one or more tasks of the target feedforward task queue can be executed. The maximum value of the results output by the target feedforward task queue can be used as the first maximum value of the subsequent attention task queue of the target attention task queue.

[0101] It can be understood that the method of the present disclosure is described above, and the device of the present disclosure will be described below.

[0102] Figure 4 is a block diagram of a task execution device according to an embodiment of the present disclosure.

[0103] like Figure 4 As shown, the device 400 may include a first execution module 410 , a determination module 420 , and a second execution module 430 .

[0104] The first execution module 410 is used to use the computing unit to execute the first multiplication task of the target attention task queue according to the first query feature and the first key feature to obtain the first multiplication result feature.

[0105] The determination module 420 is used to determine the feature to be processed using a calculation unit according to the first multiplication result feature and the target threshold feature. The target threshold feature is obtained according to the second query feature and the second key feature. The second query feature is obtained by performing data type conversion on the first query feature, and the second key feature is obtained by performing data type conversion on the first key feature.

[0106] The second execution module 430 is used to use the computing unit to execute multiple target tasks in the target attention task queue according to the feature to be processed and the first value feature to obtain the attention feature.

[0107] In some embodiments, the determination module includes: a first determination submodule, configured to determine the intermediate feature using a computing unit according to the first multiplication result feature and the target threshold feature; and an adjustment submodule, configured to adjust the intermediate feature using a computing unit according to a scaling parameter to obtain a feature to be processed.

[0108] In some embodiments, the target threshold feature is obtained based on the second multiplication result feature, the second multiplication result feature is obtained by multiplying the second query feature with the processed second key transposed feature, the processed second key transposed feature is obtained by transposing the processed second key feature, the processed second key feature is obtained by subtracting the second key feature from the first median feature, and the first median feature is obtained by determining the median of the second key feature along the channel dimension.

[0109] In some embodiments, the target threshold feature is obtained by performing relevant operations based on the second multiplication result feature using the following sub-modules: a first processing sub-module, used to process the second multiplication result feature using a maximum value function and a minimum value function respectively, to obtain the first function processing result feature and the second function processing result feature; a second processing sub-module, used to process the difference between the first function processing result feature and the second function processing result feature using an exponential function, to obtain an exponential feature; a third processing sub-module, used to process the sum of the exponential feature and the preset feature using a logarithmic function, to obtain a logarithmic feature; an addition sub-module, used to add the first function processing feature and the logarithmic feature, to obtain the addition result feature as the target threshold feature.

[0110] In some embodiments, the target threshold feature is obtained by performing relevant operations based on the second multiplication result feature using the following sub-modules: a fourth processing sub-module is used to process the second multiplication result feature using a maximum value function to obtain the first function processing result feature as the target threshold feature.

[0111] In some embodiments, the first determination submodule is further used to use a computing unit to subtract the first multiplication result feature from the target threshold feature to obtain an intermediate feature.

[0112] In some embodiments, the adjustment submodule is further used to use the calculation unit to perform the following operations: divide the intermediate feature by the processed scaling parameter to obtain the feature to be processed, wherein the processed scaling parameter is obtained by squaring the scaling parameter.

[0113] In some embodiments, the data type of each of the first query feature, the first key feature, and the first value feature is the first data type, the data type of each of the second query feature and the second key feature is the second data type, the data volume of the first query feature is greater than the data volume of the first query feature, and the data volume of the first key feature is greater than the data volume of the second key feature. The scaling parameter is obtained based on the first maximum value and the second maximum value, the first maximum value is the maximum value in the execution result of the feedforward task queue before the target attention task queue, and the second maximum value is the maximum value that can be represented by the second data type.

[0114] In some embodiments, the scaling parameter is obtained by dividing the processed second maximum value by the first maximum value, where the processed second maximum value is obtained by taking the square root of the adjusted second maximum value, and the adjusted second maximum value is obtained by dividing the second maximum value by the attention head dimension of the target attention task queue.

[0115] In some embodiments, the second query feature is obtained by multiplying the first query feature by a scaling parameter, and the second key feature is obtained by multiplying the first key feature by the scaling parameter.

[0116] In some embodiments, the multiple target tasks of the target attention task queue include a scaling task, a preset function processing task, and a second multiplication task. The second execution module includes: a first execution submodule, which is used to use a calculation unit to perform a scaling task according to the dimensions of the feature to be processed and the first key feature to obtain a first processed feature. A second execution submodule is used to use a calculation unit to perform a preset function processing task to process the first processed feature using a preset function to obtain a second processed feature. A third execution submodule is used to use a calculation unit to perform a second multiplication task according to the third processed feature and the first value feature to obtain an attention feature. The third processed feature is obtained by converting the data type of the second processed feature.

[0117] In some embodiments, the target threshold feature is obtained based on the second multiplication result feature, the second multiplication result feature is obtained by multiplying the processed second query feature with the second key transposed feature, the processed second query feature is obtained by subtracting the second query feature from the second median feature, the second median feature is obtained by determining the median of the second query feature along the channel dimension, and the second key transposed feature is obtained by transposing the second key feature.

[0118] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0119] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0120] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0121] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 to a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0122] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0123] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the task execution method. For example, in some embodiments, the task execution method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the task execution method described above may be executed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the task execution method in any other appropriate manner (eg, by means of firmware).

[0124] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0126] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories (EPROM) or flash memories, optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) display or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0128] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0129] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0131] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A task execution method, comprising: According to the first query feature and the first key feature, using the computing unit to execute the first multiplication task of the target attention task queue, to obtain a first multiplication result feature; determining a feature to be processed by using the calculation unit according to the first multiplication result feature and the target threshold feature, wherein the target threshold feature is obtained according to a second query feature and a second key feature, the second query feature is obtained by performing data type conversion on the first query feature, and the second key feature is obtained by performing data type conversion on the first key feature; and According to the feature to be processed and the first value feature, the computing unit is used to execute multiple target tasks of the target attention task queue to obtain the attention feature.

2. The method according to claim 1, wherein: The determining the feature to be processed by the calculation unit according to the first multiplication result feature and the target threshold feature comprises: Determining an intermediate feature using the calculation unit according to the first multiplication result feature and the target threshold feature; According to the scaling parameter, the intermediate feature is adjusted using the calculation unit to obtain the feature to be processed.

3. The method according to claim 1, wherein: The target threshold feature is obtained based on the second multiplication result feature, which is obtained by multiplying the second query feature with the processed second key transposed feature, and the processed second key transposed feature is obtained by transposing the processed second key feature, and the processed second key feature is obtained by subtracting the second key feature from the first median feature, and the first median feature is obtained by determining the median of the second key feature along the channel dimension.

4. The method according to claim 3, wherein: The target threshold feature is obtained according to the second multiplication result feature by the following operation: Processing the second multiplication result feature using a maximum value finding function and a minimum value finding function respectively to obtain a first function processing result feature and a second function processing result feature; Using an exponential function to process the difference between the feature of the result of the first function processing and the feature of the result of the second function processing, to obtain an exponential feature; The logarithmic function is used to process the sum of the exponential feature and the preset feature to obtain the logarithmic feature; The first function processing feature and the logarithmic feature are added to obtain an addition result feature as the target threshold feature.

5. The method according to claim 3, wherein: The target threshold feature is obtained according to the second multiplication result feature by the following operation: The second multiplication result feature is processed by using a maximum value function to obtain a first function processing result feature as the target threshold feature.

6. The method according to claim 2, wherein: The determining the intermediate feature by using the calculation unit according to the first multiplication result feature and the target threshold feature comprises: The calculation unit is used to subtract the first multiplication result feature from the target threshold feature to obtain the intermediate feature.

7. The method according to claim 2, wherein: The step of adjusting the intermediate feature by using the calculation unit according to the scaling parameter to obtain the feature to be processed comprises: The calculation unit is used to perform the following operations: the intermediate feature is divided by a processed scaling parameter to obtain the feature to be processed, wherein the processed scaling parameter is obtained by squaring the scaling parameter.

8. The method according to claim 2, wherein: The data types of the first query feature, the first key feature, and the first value feature are each the first data type, the data types of the second query feature and the second key feature are each the second data type, the data volume of the first query feature is greater than the data volume of the first query feature, and the data volume of the first key feature is greater than the data volume of the second key feature, The scaling parameter is obtained based on a first maximum value and a second maximum value, wherein the first maximum value is a maximum value in the execution results of the feedforward task queue before the target attention task queue, and the second maximum value is a maximum value representable by the second data type.

9. The method according to claim 8, wherein: The scaling parameter is obtained by dividing the processed second maximum value by the first maximum value, wherein the processed second maximum value is obtained by taking the square root of the adjusted second maximum value, and the adjusted second maximum value is obtained by dividing the second maximum value by the attention head dimension of the target attention task queue.

10. The method according to claim 2, wherein: The second query feature is obtained by multiplying the first query feature by the scaling parameter, and the second key feature is obtained by multiplying the first key feature by the scaling parameter.

11. The method according to claim 1, wherein: The multiple target tasks of the target attention task queue include a scaling task, a preset function processing task, and a second multiplication task. The step of using the computing unit to execute multiple target tasks of the target attention task queue according to the feature to be processed and the first value feature to obtain the attention feature includes: According to the dimensions of the feature to be processed and the first key feature, using the computing unit to perform the scaling task to obtain a first processed feature; Using the computing unit to execute the preset function processing task, so as to process the first processed feature using the preset function to obtain a second processed feature; The attention feature is obtained by performing the second multiplication task using the computing unit according to the third processed feature and the first value feature, wherein the third processed feature is obtained by performing data type conversion on the second processed feature.

12. The method according to claim 1, wherein: The target threshold feature is obtained based on the third multiplication result feature, the third multiplication result feature is obtained by multiplying the processed second query feature and the second key transposition feature, the processed second query feature is obtained by subtracting the second query feature from the second median feature, the second median feature is obtained by determining the median of the second query feature along the channel dimension, and the second key transposition feature is obtained by transposing the second key feature.

13. A task execution device, comprising: A first execution module, configured to execute a first multiplication operation task of the target attention task queue using a computing unit according to the first query feature and the first key feature, and obtain a first multiplication operation result feature; a determination module, configured to determine a feature to be processed by using the calculation unit according to the first multiplication result feature and the target threshold feature, wherein the target threshold feature is obtained according to a second query feature and a second key feature, the second query feature is obtained by performing data type conversion on the first query feature, and the second key feature is obtained by performing data type conversion on the first key feature; as well as The second execution module is used to use the calculation unit to execute multiple target tasks in the target attention task queue according to the feature to be processed and the first value feature to obtain the attention feature.

14. The device according to claim 13, wherein: The determination module comprises: a determination submodule, configured to determine an intermediate feature using the calculation unit according to the first multiplication result feature and the target threshold feature; The adjustment submodule is used to adjust the intermediate feature using the calculation unit according to the scaling parameter to obtain the feature to be processed.

15. The device according to claim 13, wherein: The target threshold feature is obtained based on the second multiplication result feature, which is obtained by multiplying the second query feature with the processed second key transposed feature, which is obtained by transposing the processed second key feature, which is obtained by subtracting the second key feature from the first median feature, and which is obtained by determining the median of the second key feature along the channel dimension.

16. The device according to claim 15, wherein: The target threshold feature is obtained by performing relevant operations using the following submodules according to the second multiplication result feature: A first processing submodule, used to process the second multiplication result feature by using a maximum value finding function and a minimum value finding function respectively, to obtain a first function processing result feature and a second function processing result feature; A second processing submodule, configured to use an exponential function to process the difference between the first function processing result feature and the second function processing result feature to obtain an exponential feature; The third processing submodule is used to process the sum of the exponential feature and the preset feature using a logarithmic function to obtain a logarithmic feature; An addition submodule is used to add the first function processing feature and the logarithmic feature to obtain an addition result feature as the target threshold feature.

17. The device according to claim 15, wherein: The target threshold feature is obtained by performing relevant operations using the following submodules according to the second multiplication result feature: The fourth processing submodule is used to process the second multiplication result feature by using the maximum value function to obtain the first function processing result feature as the target threshold feature.

18. The device according to claim 14, wherein: The data types of the first query feature, the first key feature, and the first value feature are each the first data type, the data types of the second query feature and the second key feature are each the second data type, the data volume of the first query feature is greater than the data volume of the first query feature, and the data volume of the first key feature is greater than the data volume of the second key feature, The scaling parameter is obtained based on a first maximum value and a second maximum value, wherein the first maximum value is a maximum value in the execution results of the feedforward task queue before the target attention task queue, and the second maximum value is a maximum value representable by the second data type.

19. The device according to claim 18, wherein: The scaling parameter is obtained by dividing the processed second maximum value by the first maximum value, wherein the processed second maximum value is obtained by taking the square root of the adjusted second maximum value, and the adjusted second maximum value is obtained by dividing the second maximum value by the attention head dimension of the target attention task queue.

20. The device according to claim 13, wherein: The multiple target tasks of the target attention task queue include a scaling task, a preset function processing task, and a second multiplication task. The second execution module includes: A first execution submodule, configured to perform the scaling task using the computing unit according to the dimensions of the feature to be processed and the first key feature, to obtain a first processed feature; a second execution submodule, configured to use the calculation unit to execute the preset function processing task, so as to use the preset function to process the first processed feature to obtain a second processed feature; The third execution submodule is used to use the computing unit to perform the second multiplication task according to the third processed feature and the first value feature to obtain the attention feature, wherein the third processed feature is obtained by converting the data type of the second processed feature.

21. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 12.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 12.

23. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images