Data processing method, device, equipment and storage medium for large language model

Abstract

The present disclosure provides a data processing method for a large language model, which can be applied to the field of deep learning technology, comprising: inputting text data into the large language model, and outputting a text sequence obtained by translating the text data, wherein the large language model includes an intermediate layer; in the intermediate layer, processing the to-be-activated data outputted by the upper layer of the intermediate layer by using a gated projection of a gated linear unit to obtain first activation data, wherein the first activation data is integer type data; shifting the first activation data by using a shifter to obtain shifted activation data; processing the sum of the shifted activation data and a first activation parameter by using a linear rectification function to obtain first intermediate data; filtering out second intermediate data from the first intermediate data and the second activation parameter; multiplying the first activation data, the second intermediate data, and the third activation parameter by using a multiplier to obtain data to be shifted; and shifting the data to be shifted by using the shifter to obtain second activation data.

Description

Technical Field

[0001] The present disclosure relates to the field of deep learning technology, specifically to the field of quantization technology of large language models, and more specifically to a data processing method, apparatus, device, and storage medium for a large language model. Background Art

[0002] With the rapid development of artificial intelligence technology, we have been impressed by the performance of large language models such as LLaMA (Large Language Model Meta AI) in natural language processing tasks, such as text translation and intelligent question-answering. Large language models can have billions or even hundreds of billions of parameters, and the volume of input text they need to analyze during task execution is difficult to predict. This makes them highly dependent on platforms with high computing power. Related technologies use various model compression techniques, including weight pruning, knowledge distillation, and quantization, to reduce the computational complexity of large language models and their reliance on platform computing power.

[0003] In the process of realizing the concepts disclosed herein, at least the following problems exist in the related art: when processing text input to a large language model using quantization technology, the data that should have been input into the function must first be dequantized into floating-point data before calculation. This still hinders the large language model from improving the calculation speed or inference speed, and limits the large language model's choice of platform. Summary of the Invention

[0004] In view of the above problems, the present disclosure provides a data processing method, apparatus, device and storage medium for a large language model that improves the hardware computing rate.

[0005] According to a first aspect of the present disclosure, a data processing method for a large language model is provided, comprising: inputting text data into a large language model, and outputting a text sequence obtained by translating the text data, wherein the large language model includes an intermediate layer; in the intermediate layer, processing the data to be activated outputted by the upper layer of the intermediate layer using a gated projection of a gated linear unit to obtain first activation data, wherein the first activation data is integer type data; shifting the first activation data using a shifter to obtain shifted activation data; processing the sum of the shifted activation data and a first activation parameter using a linear rectification function to obtain first intermediate data, wherein the first activation parameter is pre-stored in a register used by the large language model, and the first activation parameter is pre-stored in a register used by the large language model. The first activation parameter is an integer type parameter; the second intermediate data is filtered out from the above-mentioned first intermediate data and the second activation parameter, wherein the above-mentioned second activation parameter is pre-stored in the above-mentioned register, and the above-mentioned second activation parameter is an integer type parameter; the first activation data, the above-mentioned second intermediate data and the third activation parameter are multiplied by a multiplier to obtain the data to be shifted, wherein the above-mentioned third activation parameter is pre-stored in the above-mentioned register, and the above-mentioned third activation parameter is an integer type parameter; the above-mentioned data to be shifted is shifted by the above-mentioned shifter to obtain the second activation data, wherein the above-mentioned second activation data is data for making a Hadamard product with the third activation data, and the above-mentioned third activation data is data output by the ascending projection of the above-mentioned gated linear unit.

[0006] According to an embodiment of the present disclosure, the first activation parameter, the second activation parameter, and the third activation parameter are calculated based on an amplified quantization factor, wherein the amplified quantization factor is floating-point data obtained by amplifying the quantization factor by an integer multiple.

[0007] According to an embodiment of the present disclosure, the first activation parameter is obtained by integerizing a first initial parameter of a floating-point type, wherein the first initial parameter is a quotient of a first fixed parameter and the amplified quantization factor.

[0008] According to an embodiment of the present disclosure, the second activation parameter is obtained by integerizing a second initial parameter of a floating-point type, wherein the second initial parameter is a quotient of a second fixed parameter and the amplified quantization factor.

[0009] According to an embodiment of the present disclosure, the third activation parameter is obtained by integerizing the third initial parameter, wherein the third initial parameter is obtained by integer-multiplying the quotient of the amplified quantization factor and the second fixed parameter.

[0010] According to an embodiment of the present disclosure, the shifting of the first activation data by using a shifter includes: shifting the first activation data to the right by 2 bits.

[0011] According to an embodiment of the present disclosure, the shifting of the data to be shifted by using the shifter includes: shifting the data to be shifted rightward by 16 bits.

[0012] A second aspect of the present disclosure provides a data processing device for a large language model, comprising: a text generation module, configured to input text data into a large language model and output a text sequence obtained by translating the text data, wherein the large language model comprises an intermediate layer, wherein the intermediate layer is configured to perform nonlinear transformation and feature extraction on the data to be activated outputted by the upper layer to generate a new feature matrix; wherein the text generation module comprises: a first determination unit, configured to process the data to be activated in the intermediate layer using a gated projection of a gated linear unit to obtain first activation data, wherein the first activation data is integer type data; a first shift unit, configured to shift the first activation data using a shifter to obtain shifted activation data; a variable determination unit, configured to perform a linear rectification function on the shifted activation data and the first activation data; The sum of the activation parameters is processed to obtain first intermediate data, wherein the above-mentioned first activation parameter is pre-stored in the register used by the above-mentioned large language model; a variable screening unit is used to screen out second intermediate data from the above-mentioned first intermediate data and the second activation parameter, wherein the above-mentioned second activation parameter is pre-stored in the above-mentioned register; a multiplication unit is used to use a multiplier to multiply the above-mentioned first activation data, the above-mentioned second intermediate data and the third activation parameter to obtain data to be shifted, wherein the above-mentioned third activation parameter is pre-stored in the above-mentioned register; a second shifting unit is used to shift the above-mentioned data to be shifted by using the above-mentioned shifter to obtain second activation data, wherein the above-mentioned second activation data is data for making a Hadamard product with the third activation data, and the above-mentioned third activation data is data output by the ascending projection of the above-mentioned gated linear unit.

[0013] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0014] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the computer program or instructions are executed by a processor.

[0015] According to an embodiment of the present disclosure, in the process of translating text data using a large language model, in the middle layer of the large language model, the data to be activated entering the gated linear unit is shifted to obtain shifted activation data, and the first activation parameter, second activation parameter, and third activation parameter obtained in advance are used to process the data to be activated, so that the gated linear unit of the middle layer can use the quantized data to be activated as input data without making any changes, without the need to perform an inverse quantization operation, thereby reducing the amount of calculation generated. In addition, the first activation parameter, the second activation parameter, and the third activation parameter are all integer type data, so the amount of calculation generated by floating-point type data in the operation process can be reduced, thereby reducing the dependence of the large language model on the computing power of the hardware platform, that is, the operations in this method can be directly executed efficiently using the basic instructions in the processor, thereby utilizing the capabilities of the existing underlying hardware to achieve faster computing speeds. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0017] Figure 1 The following schematically illustrates an application scenario diagram of the data processing method for a large language model according to an embodiment of the present disclosure.

[0018] Figure 2 The flowchart of the data processing method of the large language model according to the embodiment of the present disclosure is schematically shown.

[0019] Figure 3 The figure schematically shows a middle layer diagram of a large language model according to an embodiment of the present disclosure.

[0020] Figure 4 The structure of the data processing device of the large language model according to the embodiment of the present disclosure is schematically shown.

[0021] Figure 5 The figure schematically shows a block diagram of an electronic device suitable for implementing a data processing method for a large language model according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0023] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0024] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0025] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0026] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned. They should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0027] With the rapid development of artificial intelligence (AI), large language models such as LLaMA have achieved remarkable performance in natural language processing. However, these models contain billions, or even hundreds of billions, of model parameters, placing high demands on the computing performance of hardware platforms, thus limiting the platforms on which they can be deployed. To address this reliance on platform computing power, researchers have developed a variety of model compression techniques, including weight pruning, knowledge distillation, and quantization.

[0028] Floating-point data can represent a wider range of data than integer data. Therefore, floating-point data requires more storage space and computational complexity than integer data. Even if floating-point data and integer data occupy the same storage space, the representation and computation of floating-point data are more complex, requiring additional computational steps. This results in a much higher computational complexity for floating-point data than for integer data.

[0029] Methods based on quantization technology involve using lower-precision data types to store model weights and perform calculations. Quantization technology has attracted widespread attention due to its potential to reduce the amount of computation required by the model and accelerate the inference process. However, traditional large language models that use quantization technology, after quantizing the weights of the large language model and the calculation results of the model layer, although operations such as convolution and matrix multiplication can already achieve full integer calculations, whenever faced with nonlinear functions, especially large language models with nonlinear activation functions, in order to reduce the probability of errors in the large language model, it is necessary to first dequantize the calculation results of the model layer back to floating type data, and then perform subsequent calculations based on the function. This still limits the choice of model deployment platform and the improvement of model inference speed.

[0030] The disclosed embodiments are targeted at a dynamic random access memory (DRAM)-based integrated memory and computing architecture, specifically designed for deep learning applications. The coarse-grained reconfigurable array (CGRA) designed within this architecture utilizes a full-integer instruction set architecture. While full-integer computational complexity and power consumption are significantly lower than floating-point operations, this also presents a series of challenges for LLaMA2 model inference deployment, particularly with regard to the unique nonlinear activation functions within the LLaMA2 model, such as the Swish function and the Root Mean Squared Error Normalized (RMS Norm).

[0031] An embodiment of the present disclosure provides a data processing method for a large language model, which inputs text data into the large language model and outputs a text sequence obtained by translating the text data, wherein the large language model includes an intermediate layer, wherein the intermediate layer is used to perform nonlinear transformation and feature extraction on the data to be activated output by the upper layer to generate a new feature matrix; in the intermediate layer, the data to be activated is processed using a gated projection of a gated linear unit to obtain first activation data, wherein the first activation data is integer type data; the first activation data is shifted using a shifter to obtain shifted activation data; the sum of the shifted activation data and the first activation parameter is processed using a linear rectification function to obtain first intermediate data, wherein the first activation parameter The first activation parameter is pre-stored in a register used by a large language model, and is an integer type parameter; the second intermediate data is screened out from the first intermediate data and the second activation parameter, wherein the second activation parameter is pre-stored in the register, and is an integer type parameter; the first activation data, the second intermediate data, and the third activation parameter are multiplied by a multiplier to obtain data to be shifted, wherein the third activation parameter is pre-stored in the register, and is an integer type parameter; the data to be shifted is shifted by a shifter to obtain second activation data, wherein the second activation data is data for performing a Hadamard product with the third activation data, and the third activation data is data output by an ascending projection of a gated linear unit.

[0032] Figure 1 The following schematically illustrates an application scenario diagram of the data processing method for a large language model according to an embodiment of the present disclosure.

[0033] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a DRAM storage-computing integrated architecture 105. The network 104 is used as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the DRAM storage-computing integrated architecture 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0034] A user can use a first terminal device 101, a second terminal device 102, or a third terminal device 103 to interact with a server 105 via a network 104 to receive or send data to the DRAM storage and computing architecture 105. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, or the third terminal device 103, such as web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0035] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers. The first terminal device 101, the second terminal device 102, and the third terminal device 103 are devices that send text data to the server 105.

[0036] The server 105 may include a DRAM storage and computing integrated architecture, which may be composed of multiple cores. The DRAM storage and computing integrated architecture is used to provide hardware support for the calculation or operation of a large language model. The parameters used in the large language model may be stored in the cores, and multiple devices including registers, control units, computing units, caches, and interconnection networks may be integrated on the cores.

[0037] For example, server 105 can process text data sent by users using first terminal device 101, second terminal device 102, and third terminal device 103 using the DRAM storage-computing architecture to obtain a translated text sequence (for example only). The DRAM storage-computing architecture can then return the processed "text sequence" stored in binary form to the terminal device.

[0038] It should be noted that the data processing method for a large language model provided in the embodiments of the present disclosure can generally be executed on the server 105. Accordingly, the data processing device for a large language model provided in the embodiments of the present disclosure can generally be provided in the server 105, wherein the text generation module can generally be provided in a DRAM storage and computing integrated architecture.

[0039] It should be understood that Figure 1 The number of terminal devices and servers in the embodiment is merely illustrative. Any number of terminal devices and servers may be used as required.

[0040] The following will be based on Figure 1 The scene described by Figure 2~Figure 3 The data processing method of the large language model of the disclosed embodiment is described in detail.

[0041] Figure 2 The flowchart of the data processing method of the large language model according to the embodiment of the present disclosure is schematically shown.

[0042] like Figure 2 As shown, the data processing method of the large language model of this embodiment includes operations S210 to S270.

[0043] In operation S210 , text data is input into a large language model, and a text sequence translated according to the text data is output, wherein the large language model includes an intermediate layer.

[0044] The middle layer is used to perform nonlinear transformation and feature extraction on the activated data output by the previous layer to generate a new feature matrix

[0045] In operation S220 , in the middle layer, the data to be activated is processed using a gated projection of a gated linear unit to obtain first activation data, where the first activation data is integer type data.

[0046] In operation S230, the first activation data is shifted by a shifter to obtain shifted activation data.

[0047] In operation S240, a linear rectification function is used to process the sum of the shifted activation data and the first activation parameter to obtain first intermediate data, wherein the first activation parameter is pre-stored in a register used by the large language model and is an integer type parameter.

[0048] In operation S250 , second intermediate data is filtered out from the first intermediate data and a second activation parameter, wherein the second activation parameter is pre-stored in a register and is an integer type parameter.

[0049] In operation S260, a multiplier is used to perform a multiplication operation on the first activation data, the second intermediate data, and the third activation parameter to obtain data to be shifted, wherein the third activation parameter is pre-stored in a register and is an integer type parameter.

[0050] In operation S270, the data to be shifted is shifted by a shifter to obtain second activation data, wherein the second activation data is data for performing a Hadamard product with the third activation data, and the third activation data is data output by an ascending projection of a gated linear unit.

[0051] According to embodiments of the present disclosure, text data is translation text input into a large language model. The text data can be one or more paragraphs of text, for example, multiple paragraphs of text in English or other languages. The large language model can be a trained model that can be used for translation between multiple languages. The text data to be translated into another language is input into the large language model, which then translates the input text data to produce the text sequence required by the user.

[0052] According to an embodiment of the present disclosure, a large language model may include an encoding layer, a plurality of data processing layer structures stacked by a self-attention mechanism and an intermediate layer, a normalization layer, etc. The intermediate layer includes a gated linear unit, which can be used to perform subsequent processing on the vector obtained by the encoder, and extract the semantic features of the context through the self-attention mechanism and the gating mechanism, wherein the encoder is used to perform word segmentation, text minimum unit (token) mapping, and embedding operations on the input text data in sequence to convert the input text data into a corresponding matrix representation, wherein the token can be a word, a subword or a character, etc.

[0053] According to an embodiment of the present disclosure, the layer above the intermediate layer may further include an RMS Norm function, which is used to normalize the input feature matrix to obtain the data to be activated by the input gated linear unit. The gated linear unit performs nonlinear transformation and feature extraction on the data to be activated to generate a new feature matrix. After being processed by multiple self-attention mechanisms and gated linear units, a feature representation for predicting text sequences is generated.

[0054] According to an embodiment of the present disclosure, in subsequent operations, the large language model will calculate the probability distribution of each translation token in the translation result of the text data, that is, in the text sequence, based on the feature representation of the predicted text sequence, select the translation token with the highest probability, or introduce parameters for improving the randomness of the output to obtain multiple translation tokens that constitute the text sequence.

[0055] According to an embodiment of the present disclosure, the large language model includes an intermediate layer, which is used to improve the expressive power of the output result of the large language model, that is, to improve the accuracy of the text sequence. The intermediate layer includes a gated linear unit, and the gated linear unit may include a gated projection, an ascending projection, and a descending projection to realize multi-level data processing. In the intermediate layer, the gated projection of the gated linear unit is used to process the data to be activated outputted by the upper layer of the intermediate layer to obtain the first activation data, wherein the first activation data is integer type data, and the data to be activated may be the data obtained after encoding, normalization, self-attention and other operations of the text data. The gated projection is used to perform a linear transformation on the data to be activated to provide the first activation data after the linear transformation for subsequent operations.

[0056] According to an embodiment of the present disclosure, a large language model is implemented based on a DRAM storage-computing integrated architecture, wherein the DRAM storage-computing integrated architecture may include multiple core particles, wherein each core particle may be a pre-fabricated chip with a specific function that can be combined and integrated. The core particles may include shifters, multipliers, registers, etc. By calling the shifter on the core particle, the first activation data is shifted to obtain a shift activation function.

[0057] According to an embodiment of the present disclosure, the first activation parameter may be obtained through pre-training, and the first activation parameter is stored in a register, wherein the register may be pre-prepared in the above-mentioned core particle, and the first activation parameter is integer type data. The sum of the shifted activation data and the first activation parameter is processed using a linear rectification function to obtain first intermediate data, wherein the first intermediate data is 0, or the sum of the shifted activation data and the first activation parameter. The linear rectification function is used to select the sum of the shifted activation data and the first activation parameter as the first intermediate data when the sum of the shifted activation data and the first activation parameter is greater than 0; and to select 0 as the first intermediate data when the sum of the shifted activation data and the first activation parameter is not greater than 0, so that the large language model can better learn the nonlinear relationship in the processed data, thereby improving the expressive power and generalization ability of the large language model.

[0058] According to an embodiment of the present disclosure, the second activation parameter can be pre-trained and stored in a register, and the second activation parameter is integer type data. In the first intermediate data and the second activation parameter, data that meets the accuracy requirements of the large language model is screened out as the second intermediate data. Preferably, when the first intermediate data and the second activation parameter are both certificate-type matrices, the elements of corresponding positions in the two matrices can be compared element by element in the first intermediate data and the second activation parameter, and the smaller value of the two elements is returned to form a new matrix, i.e., the second intermediate data.

[0059] According to an embodiment of the present disclosure, the third activation parameter may be pre-trained and stored in a register, and the third activation parameter is integer data. The multiplier is called to perform a multiplication operation on the first activation data, the second intermediate data, and the third activation parameter to obtain the data to be shifted.

[0060] According to an embodiment of the present disclosure, the second activation data is obtained by calling a shifter to shift the data to be shifted, wherein the second activation data is data for performing a Hadamard product with the third activation data, and the third activation data is data output by the ascending projection of the gated linear unit.

[0061] According to an embodiment of the present disclosure, in the process of translating text data using a large language model, in the middle layer of the large language model, the data to be activated entering the gated linear unit is shifted to obtain shifted activation data, and the first activation parameter, second activation parameter, and third activation parameter obtained in advance are used to process the data to be activated, so that the gated linear unit of the middle layer can use the quantized data to be activated as input data without making any changes, without the need to perform an inverse quantization operation, thereby reducing the amount of calculation generated. In addition, the first activation parameter, the second activation parameter, and the third activation parameter are all integer type data, so the amount of calculation generated by floating-point type data in the operation process can be reduced, thereby reducing the dependence of the large language model on the computing power of the hardware platform, that is, the operations in this method can be directly executed efficiently using the basic instructions in the processor, thereby utilizing the capabilities of the existing underlying hardware to achieve faster computing speeds.

[0062] According to an embodiment of the present disclosure, depending on the training purpose of the large language model, when its purpose is to generate text in a specific format based on the input text, wherein the text in a specific format may include a seven-character quatrain, a five-character regulated verse, etc., the text data is input into the large language model, and a text sequence in a specific format is output. Specifically, the encoder of the large language model performs word segmentation, token mapping, and embedding operations on the input text data in sequence to convert the input text data into a corresponding matrix representation. The gated linear unit is used to calculate the relationship between the current token and other tokens to obtain the corresponding feature matrix.

[0063] According to the embodiments of the present disclosure, the input feature matrix is ​​normalized by the normalization layer in the middle layer to obtain the data to be activated by the input gated linear unit. The gated linear unit performs nonlinear transformation and feature extraction on the data to generate a new feature matrix. After multiple layers of self-attention and gated linear unit processing, a feature representation is generated for predicting text sequences in a specific format.

[0064] According to an embodiment of the present disclosure, in subsequent operations, the large language model calculates the probability distribution of each token in a text sequence of a specific format based on the feature representation to obtain a text sequence of the specific format.

[0065] Figure 3 The figure schematically shows a middle layer diagram of a large language model according to an embodiment of the present disclosure.

[0066] like Figure 3As shown, the middle layer may include an RMS Norm function 310 and a gated linear unit 320 for normalizing the input data of the middle layer. The gated linear unit may include an up projection 321 (up_proj), a gated projection 322 (gate_proj), an activation layer 323, and a down projection 324 (down_proj).

[0067] The ascending projection 321 is used to convert the dimension of the input data to be activated from one size to another, preferably, to convert the dimension of the data to be activated from a small size to a larger size. The gated projection 322 is used to linearly transform the data to be activated to provide a set of transformed features, which are the first activation data. The activation layer 323 is used to implement the operations S230 to S270 described above. The descending projection 324 is the last linear transformation of the gated linear unit and is used to compress the increased dimension size back to the original size, which is the dimension size of the dimension of the data to be activated.

[0068] According to the embodiment of the present disclosure, by Figure 3 The gated linear unit of the structure shown performs complex feature mapping while maintaining consistent input and output dimensions, thereby enhancing the expressive power of the large language model. Because the data processing in activation layer 323 is performed using non-floating-point data, the computational complexity of the model is reduced. Furthermore, because the parameters in activation layer 323 can change with the quantization factor, and the quantization factor can be gradually optimized during the training of the large language model, the large language model can still ensure the accuracy of the output results with low computational complexity.

[0069] According to an embodiment of the present disclosure, the first activation parameter, the second activation parameter, and the third activation parameter are calculated based on the amplified quantization factor, wherein the amplified quantization factor is floating-point data obtained by amplifying the quantization factor by an integer multiple.

[0070] According to an embodiment of the present disclosure, the quantization factor may be floating-point data, and the quantization factor may be obtained by following the training of a large language model. The quantization factor is amplified by an integer multiple to obtain an amplified quantization factor. Preferably, the quantization factor is amplified by 4 times to obtain the amplified quantization factor. The relationship between the amplified quantization factor and the quantization factor can be expressed by the following formula (1):

[0071] (1);

[0072] in, Characterize the amplification quantization factor, Characterize the quantization factor.

[0073] According to an embodiment of the present disclosure, after amplifying the quantization factor by a positive integer multiple, the amplified quantization factor is obtained, which can effectively reduce the probability of overflow of the value range of parameters or data such as the first activation parameter, the second activation parameter, and the third activation parameter.

[0074] According to an embodiment of the present disclosure, the first activation parameter is obtained by integerizing a first initial parameter of a floating-point type, wherein the first initial parameter is a quotient of a first fixed parameter and an amplified quantization factor.

[0075] According to an embodiment of the present disclosure, integerization converts floating-point data into integer data. The first fixed parameter is integer data. Preferably, the first quantization parameter may be 3. The relationship between the first fixed parameter, the amplified quantization factor, and the first activation parameter may be expressed by the following formula (2):

[0076] (2);

[0077] in, Characterizes the first activation parameter, represents the function of integerizing the data, 3 represents the first fixed parameter, Characterize the amplification quantization factor, Characterize the first initial parameter.

[0078] According to an embodiment of the present disclosure, the second activation parameter is obtained by integerizing a second initial parameter of a floating-point type, wherein the second initial parameter is a quotient of the second fixed parameter and the amplified quantization factor.

[0079] According to an embodiment of the present disclosure, the second fixed parameter is integer type data. Preferably, the first quantization parameter may be 6. The relationship between the second fixed parameter, the amplified quantization factor, and the second activation parameter may be expressed by the following formula (3):

[0080] (3);

[0081] in, represents the second activation parameter, 6 represents the second fixed parameter, Characterize the second initial parameter.

[0082] According to an embodiment of the present disclosure, the third activation parameter is obtained by integerizing the third initial parameter, wherein the third initial parameter is obtained by integer-multiplying the quotient of the amplified quantization factor and the second fixed parameter.

[0083] According to an embodiment of the present disclosure, the quotient of the amplified quantization factor and the second fixed parameter is amplified by an integer multiple. Preferably, the quotient of the amplified quantization factor and the second fixed parameter can be amplified by a positive integer power of 2. Preferably, the quotient of the amplified quantization factor and the second fixed parameter can be amplified by the 16th power of 2. The relationship between the third activation parameter, the amplified quantization factor, and the second fixed parameter can be expressed by the following formula (4):

[0084] (4);

[0085] in, Characterizes the third activation parameter, Characterize the third initial parameter.

[0086] According to an embodiment of the present disclosure, the related art uses the Swish activation function to determine the data to be used in the Hadamard product with the third activation data. However, the original Swish activation function includes an e-exponential operation, which has high computational complexity. Even if the Swish activation function is replaced with an activation function that is similar to it but does not include an exponential operation, it is still impossible to use quantized integer type activation data as input, because the activation function that approximates the Swish function after replacement is still a nonlinear function, and the data obtained will obviously be erroneous.

[0087] According to an embodiment of the present disclosure, based on the Swish activation function, an approximate function thereof is used as a basis. Preferably, the approximate function thereof can be the Hard-Swish activation function. Based on the approximate function, a modification is performed, and multiple parameters of the activation function are modified to a form that can change the quantization factor, so that both the data to be activated and the processing of the data to be activated can be calculated using integer data. This reduces the computational complexity of the large language model when processing text data.

[0088] According to an embodiment of the present disclosure, the relationship between the data to be activated and the quantization factor, and the input data using Swish as the activation function can be expressed by the following formula (5):

[0089] (5);

[0090] in, Represents the data to be activated, Characterize the quantization factor, Characterizes the floating-point data obtained by inverse quantization of the input activation function in related technologies.

[0091] According to an embodiment of the present disclosure, shifting the first activation data by using a shifter includes: shifting the first activation data to the right by 2 bits.

[0092] According to an embodiment of the present disclosure, shifting the data to be shifted by using a shifter includes: shifting the data to be shifted right by 16 bits.

[0093] According to the embodiments of the present disclosure, when hardware is used to implement calculations, all calculations of powers of 2 can be implemented using shifting. When a large language model is used to translate text data, the amount of data that needs to be processed is very large. By leveraging the characteristics of shifting, which requires fewer machine cycles, shorter execution time, and higher efficiency than multiplication and, in particular, division, data shifting replaces the processing of the first activation data and the data to be shifted in related technologies. This can improve the processing efficiency of the hardware platform where the large language model resides, thereby reducing the computing power requirements of the large language model on the hardware platform.

[0094] Based on the above-mentioned data processing method of a large language model, the present disclosure also provides a data processing device of a large language model. Figure 4 The device is described in detail.

[0095] Figure 4 The structure of the data processing device of the large language model according to the embodiment of the present disclosure is schematically shown.

[0096] like Figure 4 As shown, the data processing device 400 for a large language model in this embodiment includes a text generation module 410 .

[0097] Text generation module 410 is configured to input text data into a large language model and output a text sequence translated from the text data. The large language model includes an intermediate layer configured to perform nonlinear transformation and feature extraction on the activated data output by the previous layer to generate a new feature matrix. In one embodiment, text generation module 410 may be configured to perform operation S210 described above and will not be further described herein.

[0098] The text generation module includes: a first determination unit 411 , a first shift unit 412 , a variable determination unit 413 , a variable screening unit 414 , a multiplication unit 415 , and a second shift unit 416 .

[0099] A first determining unit 411 is configured to process the to-be-activated data in the intermediate layer using a gated projection of a gated linear unit to obtain first activation data, where the first activation data is integer data. In one embodiment, the first determining unit may be configured to perform operation S220 described above, which will not be further described herein.

[0100] The first shift unit 412 is configured to shift the first activation data using a shifter to obtain shifted activation data. In one embodiment, the first shift unit may be configured to perform the operation S230 described above, which will not be described in detail herein.

[0101] The variable determination unit 413 is configured to process the sum of the shifted activation data and the first activation parameter using a linear rectification function to obtain first intermediate data, where the first activation parameter is pre-stored in a register used by the large language model. In one embodiment, the variable determination unit may be configured to perform operation S240 described above, and will not be further described here.

[0102] The variable screening unit 414 is configured to screen the second intermediate data from the first intermediate data and the second activation parameter, wherein the second activation parameter is pre-stored in the register. In one embodiment, the variable screening unit may be configured to perform the operation S250 described above, which will not be described in detail here.

[0103] The multiplication unit 415 is configured to use a multiplier to multiply the first activation data, the second intermediate data, and the third activation parameter to obtain the data to be shifted, where the third activation parameter is pre-stored in a register. In one embodiment, the multiplication unit can be configured to perform operation S260 described above, which will not be further described here.

[0104] The second shift unit 416 is configured to shift the data to be shifted using a shifter to obtain second activation data, where the second activation data is data used to perform a Hadamard product with the third activation data, and the third activation data is data output by the ascending projection of the gated linear unit. In one embodiment, the second shift unit can be configured to perform operation S270 described above, which will not be further described here.

[0105] According to an embodiment of the present disclosure, the first activation parameter, the second activation parameter, and the third activation parameter are calculated based on the amplified quantization factor, wherein the amplified quantization factor is floating-point data obtained by amplifying the quantization factor by an integer multiple.

[0106] According to an embodiment of the present disclosure, the first activation parameter is obtained by integerizing a first initial parameter of a floating-point type, wherein the first initial parameter is a quotient of a first fixed parameter and an amplified quantization factor.

[0107] According to an embodiment of the present disclosure, the second activation parameter is obtained by integerizing a second initial parameter of a floating-point type, wherein the second initial parameter is a quotient of the second fixed parameter and the amplified quantization factor.

[0108] According to an embodiment of the present disclosure, the third activation parameter is obtained by integerizing the third initial parameter, wherein the third initial parameter is obtained by integer-multiplying the quotient of the amplified quantization factor and the second fixed parameter.

[0109] According to an embodiment of the present disclosure, shifting the first activation data by using a shifter includes: shifting the first activation data to the right by 2 bits.

[0110] According to an embodiment of the present disclosure, shifting the data to be shifted by using a shifter includes: shifting the data to be shifted right by 16 bits.

[0111] According to embodiments of the present disclosure, any multiple units in the generation module 410 may be combined into a single module or unit, or any one of the units may be split into multiple units. Alternatively, at least part of the functionality of one or more of these units may be combined with at least part of the functionality of other units and implemented in a single unit. According to embodiments of the present disclosure, at least one of the first determination unit 411, the first shift unit 412, the variable determination unit 413, the variable screening unit 414, the multiplication unit 415, and the second shift unit 416 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented in hardware or firmware by any other reasonable means of integrating or packaging circuits, or may be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of these. Alternatively, at least one of the first determination unit 411, the first shift unit 412, the variable determination unit 413, the variable screening unit 414, the multiplication unit 415, and the second shift unit 416 can be at least partially implemented as a computer program unit, which can perform corresponding functions when the computer program unit is run.

[0112] Figure 5 The figure schematically shows a block diagram of an electronic device suitable for implementing a data processing method for a large language model according to an embodiment of the present disclosure.

[0113] like Figure 5 As shown, the electronic device 500 according to an embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage unit 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0114] Various programs and data required for the operation of the electronic device 500 are stored in the RAM 503. The processor 501, ROM 502, and RAM 503 are connected to each other via a bus 504. The processor 501 executes the various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 502 and / or RAM 503. It should be noted that the programs may also be stored in one or more memories other than the ROM 502 and RAM 503. The processor 501 may also execute the various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in one or more memories.

[0115] According to an embodiment of the present disclosure, electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to bus 504. Electronic device 500 may also include one or more of the following components connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a LAN card or modem. Communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 510 as needed, so that computer programs read from the removable media can be installed into storage section 508 as needed.

[0116] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0117] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 502 and / or RAM 503 described above, and / or one or more memories other than ROM 502 and RAM 503.

[0118] Embodiments of the present disclosure also include a computer program product, comprising a computer program containing program code for executing the method shown in the flowchart. When the computer program product is executed in a computer system, the program code is used to cause the computer system to implement the data processing method for a large language model provided by embodiments of the present disclosure.

[0119] The computer program executes the above functions defined in the system / device of the embodiment of the present disclosure when the computer program is executed by the processor 501. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0120] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 509, and / or installed from a removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0121] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 509, and / or installed from a removable medium 511. When the computer program is executed by the processor 501, the above-described functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0122] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] Those skilled in the art will appreciate that the features described in the various embodiments of the present disclosure may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present disclosure. In particular, the features described in the various embodiments of the present disclosure may be combined and / or coupled in various ways without departing from the spirit and teachings of the present disclosure. All such combinations and / or couplings fall within the scope of the present disclosure.

[0125] The above describes the embodiments of the present disclosure. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A data processing method for a large language model, characterized in that: The method comprises: Inputting text data into a large language model and outputting a text sequence translated from the text data, wherein the large language model includes an intermediate layer, wherein the intermediate layer is used to perform nonlinear transformation and feature extraction on the to-be-activated data output by the previous layer to generate a new feature matrix; In the middle layer, the data to be activated is processed using a gated projection of a gated linear unit to obtain first activation data, wherein the first activation data is integer type data; Shifting the first activation data using a shifter to obtain shifted activation data; Processing the sum of the shifted activation data and a first activation parameter using a linear rectification function to obtain first intermediate data, wherein the first activation parameter is pre-stored in a register used by the large language model and is an integer type parameter; Filtering out second intermediate data from the first intermediate data and a second activation parameter, wherein the second activation parameter is pre-stored in the register and is an integer type parameter; performing a multiplication operation on the first activation data, the second intermediate data, and a third activation parameter by a multiplier to obtain data to be shifted, wherein the third activation parameter is pre-stored in the register and is an integer type parameter; and Shifting the to-be-shifted data using the shifter to obtain second activation data, where the second activation data is data for performing a Hadamard product with third activation data, where the third activation data is data output by an ascending projection of the gated linear unit; The first activation parameter, the second activation parameter, and the third activation parameter are calculated based on an amplified quantization factor, wherein the amplified quantization factor is floating-point data obtained by amplifying the quantization factor by an integer multiple.

2. The method according to claim 1, characterized in that The first activation parameter is obtained by integerizing a first initial parameter of a floating-point type, wherein the first initial parameter is a quotient of a first fixed parameter and the amplified quantization factor.

3. The method according to claim 1, characterized in that The second activation parameter is obtained by integerizing a second initial parameter of a floating-point type, wherein the second initial parameter is a quotient of a second fixed parameter and the amplified quantization factor.

4. The method according to claim 3, characterized in that The third activation parameter is obtained by integerizing the third initial parameter, wherein the third initial parameter is obtained by integer-multiplying the quotient of the amplified quantization factor and the second fixed parameter.

5. The method according to claim 1, wherein The shifting the first activation data by using a shifter includes: The first activation data is shifted right by 2 bits.

6. The method according to claim 1, characterized in that The shifting the data to be shifted by using the shifter includes: Shift the data to be shifted right by 16 bits.

7. A data processing device for a large language model, characterized in that: The device comprises: a text generation module, configured to input text data into a large language model and output a text sequence translated from the text data, wherein the large language model includes an intermediate layer, wherein the intermediate layer is configured to perform nonlinear transformation and feature extraction on the to-be-activated data output by the previous layer to generate a new feature matrix; Wherein, the text generation module includes: a first determining unit, configured to process the data to be activated in the intermediate layer by using a gated projection of a gated linear unit to obtain first activation data, wherein the first activation data is integer type data; a first shift unit, configured to shift the first activation data using a shifter to obtain shifted activation data; a variable determination unit, configured to process the sum of the shifted activation data and a first activation parameter using a linear rectification function to obtain first intermediate data, wherein the first activation parameter is pre-stored in a register used by the large language model; a variable screening unit, configured to screen out second intermediate data from the first intermediate data and a second activation parameter, wherein the second activation parameter is pre-stored in the register; a multiplication unit, configured to perform a multiplication operation on the first activation data, the second intermediate data, and a third activation parameter using a multiplier to obtain data to be shifted, wherein the third activation parameter is pre-stored in the register; and a second shifting unit, configured to shift the data to be shifted using the shifter to obtain second activation data, wherein the second activation data is data for performing a Hadamard product with third activation data, and the third activation data is data output by an ascending projection of the gated linear unit; The first activation parameter, the second activation parameter, and the third activation parameter are calculated based on an amplified quantization factor, wherein the amplified quantization factor is floating-point data obtained by amplifying the quantization factor by an integer multiple.

8. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images