Data processing method and device for large language model, equipment and storage medium

Abstract

The invention provides a data processing method for a large language model, which can be applied to the technical field of deep learning, and comprises the following steps: inputting text data into the large language model, and outputting a text sequence obtained by translating the text data, the large language model comprising a middle layer; in the middle layer, the gating projection of the gating linear unit is used for processing the to-be-activated data output by the upper layer of the middle layer to obtain first activated data, and the first activated data is data of an integer type; shifting the first activation data by using a shifter to obtain shifted activation data; processing the sum of the shift activation data and the first activation parameter by using a linear rectification function to obtain first intermediate data; screening out second intermediate data from the first intermediate data and the second activation parameter; performing multiplication operation on the first activation data, the second intermediate data and the third activation parameter by using a multiplier to obtain to-be-shifted data; and shifting the to-be-shifted data by using the shifter to obtain second activated data.

Description

Technical Field

[0001] The present disclosure relates to the field of deep learning technology, specifically to the field of quantization technology of large language models, and more specifically to a data processing method, device, equipment and storage medium for a large language model. Background Art

[0002] With the rapid development of artificial intelligence technology, we are impressed by the performance of large language models such as LLaMA (Large Language Model Meta AI) in natural language processing tasks, such as text translation tasks and intelligent question-answering tasks. Large language models have up to one billion or even hundreds of billions of parameters, and the amount of input text that needs to be analyzed during task execution is difficult to estimate, making large language models very dependent on platforms with high computing power. In related technologies, a variety of model compression technologies, including weight pruning, knowledge distillation, and quantization technologies, are used to reduce the computational complexity of large language models and thus reduce their dependence on platform computing power.

[0003] In the process of realizing the concept disclosed herein, there are at least the following problems in the related art: when processing text input to a large language model using quantization technology, it is necessary to first dequantize the data that should have been input into the function into floating-point data before performing calculations. This makes it difficult for the large language model to improve the calculation speed or reasoning speed, and limits the choice of platform for the large language model. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a data processing method, apparatus, device and storage medium for a large language model that improves the hardware computing rate.

[0005] According to a first aspect of the present disclosure, a data processing method for a large language model is provided, comprising: inputting text data into a large language model, and outputting a text sequence obtained by translating the text data, wherein the large language model comprises an intermediate layer; in the intermediate layer, processing the data to be activated outputted by the upper layer of the intermediate layer by using a gated projection of a gated linear unit to obtain first activation data, wherein the first activation data is integer type data; shifting the first activation data by using a shifter to obtain shifted activation data; processing the sum of the shifted activation data and a first activation parameter by using a linear rectification function to obtain first intermediate data, wherein the first activation parameter is pre-stored in a register used by the large language model, and the first activation parameter is stored in a register used by the large language model. The first activation parameter is an integer type parameter; the second intermediate data is selected from the first intermediate data and the second activation parameter, wherein the second activation parameter is pre-stored in the register, and the second activation parameter is an integer type parameter; the first activation data, the second intermediate data and the third activation parameter are multiplied by a multiplier to obtain data to be shifted, wherein the third activation parameter is pre-stored in the register, and the third activation parameter is an integer type parameter; the data to be shifted is shifted by the shifter to obtain second activation data, wherein the second activation data is data for making a Hadamard product with the third activation data, and the third activation data is data output by the ascending projection of the gated linear unit.

[0006] According to an embodiment of the present disclosure, the first activation parameter, the second activation parameter and the third activation parameter are calculated based on an amplified quantization factor, wherein the amplified quantization factor is floating-point data obtained by amplifying the quantization factor by an integer multiple.

[0007] According to an embodiment of the present disclosure, the first activation parameter is obtained by integerizing a first initial parameter of a floating point type, wherein the first initial parameter is a quotient of a first fixed parameter and the amplified quantization factor.

[0008] According to an embodiment of the present disclosure, the second activation parameter is obtained by integerizing a second initial parameter of a floating point type, wherein the second initial parameter is a quotient of a second fixed parameter and the amplified quantization factor.

[0009] According to an embodiment of the present disclosure, the third activation parameter is obtained by integerizing the third initial parameter, wherein the third initial parameter is obtained by integer-multiplying the quotient of the amplified quantization factor and the second fixed parameter.

[0010] According to an embodiment of the present disclosure, the shifting of the first activation data by using a shifter includes: shifting the first activation data to the right by 2 bits.

[0011] According to an embodiment of the present disclosure, the shifting of the data to be shifted by using the shifter includes: shifting the data to be shifted rightward by 16 bits.

[0012] A second aspect of the present disclosure provides a data processing device for a large language model, comprising: a text generation module, used to input text data into a large language model, and output a text sequence obtained by translating the text data, wherein the large language model comprises an intermediate layer, wherein the intermediate layer is used to perform nonlinear transformation and feature extraction on the data to be activated output by the upper layer to generate a new feature matrix; wherein the text generation module comprises: a first determination unit, used to process the data to be activated in the intermediate layer using a gated projection of a gated linear unit to obtain first activation data, wherein the first activation data is integer type data; a first shift unit, used to shift the first activation data using a shifter to obtain shifted activation data; a variable determination unit, used to perform a linear rectification function on the shifted activation data and the first activation data; The first activation parameter is processed to obtain the first intermediate data, wherein the first activation parameter is pre-stored in the register used by the large language model; a variable screening unit is used to screen out the second intermediate data from the first intermediate data and the second activation parameter, wherein the second activation parameter is pre-stored in the register; a multiplication unit is used to use a multiplier to multiply the first activation data, the second intermediate data and the third activation parameter to obtain the data to be shifted, wherein the third activation parameter is pre-stored in the register; a second shift unit is used to shift the data to be shifted by using the shifter to obtain the second activation data, wherein the second activation data is the data used to make a Hadamard product with the third activation data, and the third activation data is the data output by the ascending projection of the gated linear unit.

[0013] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0014] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the above computer program or instructions are executed by a processor.

[0015] According to an embodiment of the present disclosure, in the process of translating text data using a large language model, in the middle layer of the large language model, the data to be activated entering the gated linear unit is shifted to obtain shifted activation data, and the first activation parameter, the second activation parameter, and the third activation parameter obtained by pre-training are used to process the data to be activated, so that the gated linear unit of the middle layer can use the quantized data to be activated as input data without making any changes, without performing an inverse quantization operation, thereby reducing the amount of calculation generated thereby. In addition, the first activation parameter, the second activation parameter, and the third activation parameter are all integer type data, so the amount of calculation generated additionally by floating-point type data in the operation process can be reduced, thereby reducing the dependence of the large language model on the computing power of the hardware platform, that is, the operations in this method can be directly executed efficiently using the basic instructions in the processor, thereby utilizing the capabilities of the existing underlying hardware to achieve faster computing speeds. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0017] Figure 1 The application scenario diagram of the data processing method of a large language model according to an embodiment of the present disclosure is schematically shown.

[0018] Figure 2 The flowchart of the data processing method of the large language model according to the embodiment of the present disclosure is schematically shown.

[0019] Figure 3 The figure schematically shows a middle layer diagram of a large language model according to an embodiment of the present disclosure.

[0020] Figure 4 The structural block diagram of the data processing device of the large language model according to the embodiment of the present disclosure is schematically shown.

[0021] Figure 5 A block diagram of an electronic device suitable for implementing a data processing method for a large language model according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0022] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0023] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0024] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0025] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0026] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0027] With the rapid development of artificial intelligence technology, large language models such as LLaMA have performed very well in natural language processing tasks. However, large language models contain billions or even hundreds of billions of model parameters, which makes large language models have high computing performance requirements on hardware platforms, thus limiting the deployable platforms of large language models. To solve the dependence of large language models on platform computing power, researchers have developed a variety of model compression technologies, including weight pruning, knowledge distillation, and quantization technology.

[0028] The range of data that can be represented by floating-point data is larger than that of integer data. Therefore, the storage space and data calculation required by floating-point data are larger than those required by integer data. Even if floating-point data and integer data occupy the same storage space, the representation and calculation of floating-point data are more complicated, which adds additional calculation steps for floating-point data, resulting in the calculation amount of floating-point data being much higher than that of integer data.

[0029] Methods based on quantization technology involve using lower precision data types to store model weights and perform calculations. Quantization technology has received widespread attention for its potential to reduce the amount of computation in the model and accelerate the inference process. However, for traditional large language models that use quantization technology, after quantizing the weights of the large language model and the calculation results of the model level, although operations such as convolution and matrix multiplication can already achieve full integer calculations, whenever faced with nonlinear functions, especially large language models with nonlinear activation functions, in order to reduce the probability of errors in the large language model, it is necessary to first dequantize the calculation results of the model level back to floating type data, and then perform subsequent calculations according to the function, which still limits the choice of model deployment platform and the improvement of model inference speed.

[0030] The embodiments of the present disclosure are oriented to a storage-computing integrated architecture based on dynamic random access memory (DRAM), which is an architecture that serves deep learning applications. The coarse grained reconfigurable array (CGRA) array designed in this architecture adopts a full integer instruction set architecture. Although the full integer computing complexity and power consumption are significantly less than floating point computing operations, it also poses a series of challenges to the deployment of LLaMA2 model inference, especially for the unique Swish function and root mean square error normalization function (Root Mean Squared Error Normalized, RMS Norm) in the LLaMA2 model. Nonlinear activation functions.

[0031] The embodiment of the present disclosure provides a data processing method for a large language model, wherein text data is input into the large language model, and a text sequence obtained by translating the text data is output, wherein the large language model includes an intermediate layer, wherein the intermediate layer is used to perform nonlinear transformation and feature extraction on the data to be activated output by the previous layer to generate a new feature matrix; in the intermediate layer, the data to be activated is processed by gated projection of a gated linear unit to obtain first activation data, wherein the first activation data is integer type data; the first activation data is shifted by a shifter to obtain shifted activation data; the sum of the shifted activation data and the first activation parameter is processed by a linear rectification function to obtain first intermediate data, wherein the first activation parameter The first activation parameter is pre-stored in a register used by a large language model, and is a parameter of an integer type; the second intermediate data is selected from the first intermediate data and the second activation parameter, wherein the second activation parameter is pre-stored in the register, and is a parameter of an integer type; the first activation data, the second intermediate data, and the third activation parameter are multiplied by a multiplier to obtain data to be shifted, wherein the third activation parameter is pre-stored in the register, and is a parameter of an integer type; the data to be shifted is shifted by a shifter to obtain second activation data, wherein the second activation data is data for making a Hadamard product with the third activation data, and the third activation data is data output by an ascending projection of a gated linear unit.

[0032] Figure 1 The application scenario diagram of the data processing method of a large language model according to an embodiment of the present disclosure is schematically shown.

[0033] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a DRAM storage-computing integrated architecture 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the DRAM storage-computing integrated architecture 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0034] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send data to the DRAM storage and computing integrated architecture 105. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0035] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc. The first terminal device 101, the second terminal device 102, and the third terminal device 103 are devices that send text data to the server 105.

[0036] The server 105 may include a DRAM storage-computing integrated architecture, which may be composed of multiple chiplets. The DRAM storage-computing integrated architecture is used to provide hardware support for the operation or running of a large language model. The parameters used in the large language model may be stored in the chiplets, and multiple devices including registers, control units, computing units, caches, and interconnection networks may be integrated on the chiplets.

[0037] For example, the server 105 can process the text data sent by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 through the DRAM storage and computing integrated architecture to obtain a translated text sequence (only as an example). The DRAM storage and computing integrated architecture can feed back the processed "text sequence" stored in binary form to the terminal device.

[0038] It should be noted that the data processing method of the large language model provided in the embodiment of the present disclosure can generally be executed on the server 105. Accordingly, the data processing device of the large language model provided in the embodiment of the present disclosure can generally be set in the server 105, wherein the text generation module can generally be set in the storage and computing integrated architecture of DRAM.

[0039] It should be understood that Figure 1 The number of terminal devices and servers in the embodiment is only for illustration. Any number of terminal devices and servers may be provided according to implementation requirements.

[0040] The following will be based on Figure 1 The scene described by Figure 2~Figure 3 The data processing method of the large language model of the disclosed embodiment is described in detail.

[0041] Figure 2 The flowchart of the data processing method of the large language model according to the embodiment of the present disclosure is schematically shown.

[0042] like Figure 2 As shown, the data processing method of the large language model of this embodiment includes operations S210 to S270.

[0043] In operation S210, text data is input into a large language model, and a text sequence translated from the text data is output, wherein the large language model includes an intermediate layer.

[0044] The intermediate layer is used to perform nonlinear transformation and feature extraction on the data to be activated output by the previous layer to generate a new feature matrix

[0045] In operation S220, in the middle layer, the data to be activated is processed by using a gated projection of a gated linear unit to obtain first activation data, wherein the first activation data is integer type data.

[0046] In operation S230, the first activation data is shifted by using a shifter to obtain shifted activation data.

[0047] In operation S240, a linear rectification function is used to process the sum of the shifted activation data and the first activation parameter to obtain first intermediate data, wherein the first activation parameter is pre-stored in a register used by the large language model, and the first activation parameter is an integer type parameter.

[0048] In operation S250, the second intermediate data is screened out from the first intermediate data and the second activation parameter, wherein the second activation parameter is pre-stored in a register and the second activation parameter is an integer type parameter.

[0049] In operation S260, a multiplier is used to perform a multiplication operation on the first activation data, the second intermediate data, and the third activation parameter to obtain data to be shifted, wherein the third activation parameter is pre-stored in a register and is an integer type parameter.

[0050] In operation S270, the data to be shifted is shifted by a shifter to obtain second activation data, wherein the second activation data is data for performing a Hadamard product with the third activation data, and the third activation data is data output by a rising projection of a gated linear unit.

[0051] According to an embodiment of the present disclosure, the text data is a translation text input into a large language model, and the text data may be one or more texts, for example, multiple texts in English or other languages, wherein the large language model may be a trained model that can be used for translation into multiple languages. The text data to be translated into another language is input into the large language model, and the large language model translates the input text data to obtain the text sequence required by the user.

[0052] According to an embodiment of the present disclosure, a large language model may include an encoding layer, a plurality of data processing layer structures stacked by a self-attention mechanism and an intermediate layer, a normalization layer, and the like. The intermediate layer includes a gated linear unit, which can be used to perform subsequent processing on the vector obtained by the encoder, and extract the semantic features of the context through the self-attention mechanism and the gating mechanism, wherein the encoder is used to perform word segmentation, text minimum unit (token) mapping, and embedding operations on the input text data in sequence to convert the input text data into a corresponding matrix representation, wherein a token can be a word, a subword, a character, and the like.

[0053] According to an embodiment of the present disclosure, the upper layer of the middle layer may further include an RMS Norm function, which is used to normalize the input feature matrix to obtain the data to be activated for the input gated linear unit. The gated linear unit performs nonlinear transformation and feature extraction on the data to be activated to generate a new feature matrix, which is processed by multiple self-attention mechanisms and gated linear units to generate a feature representation for predicting a text sequence.

[0054] According to an embodiment of the present disclosure, in subsequent operations, the large language model will calculate the probability distribution of each translation token in the translation result of the text data, that is, in the text sequence, based on the feature representation of the predicted text sequence, and select the translation token with the highest probability, or introduce parameters for improving the randomness of the output, to obtain multiple translation tokens that constitute the text sequence.

[0055] According to an embodiment of the present disclosure, a large language model includes an intermediate layer, and the intermediate layer is used to improve the expressive power of the output result of the large language model, that is, to improve the accuracy of the text sequence. The intermediate layer includes a gated linear unit, and the gated linear unit may include a gated projection, an ascending projection, and a descending projection to realize multi-level data processing. In the intermediate layer, the data to be activated outputted by the upper layer of the intermediate layer is processed using the gated projection of the gated linear unit to obtain the first activation data, wherein the first activation data is integer type data, and the data to be activated may be the data obtained after encoding, normalization, self-attention and other operations of the text data. The gated projection is used to perform a linear transformation on the data to be activated, so as to provide the first activation data after the linear transformation for subsequent operations.

[0056] According to an embodiment of the present disclosure, a large language model is implemented based on the storage-computing integrated architecture of DRAM, wherein the storage-computing integrated architecture of DRAM may include multiple core particles, wherein each core particle may be a pre-fabricated chip with a specific function and may be combined and integrated. The core particles may include shifters, multipliers, registers, etc. By calling the shifter on the core particle, the first activation data is shifted to obtain a shift activation function.

[0057] According to an embodiment of the present disclosure, the first activation parameter may be pre-trained, and the first activation parameter is stored in a register, wherein the register may be pre-prepared in the above-mentioned core particle, and the first activation parameter is integer type data. The sum of the shifted activation data and the first activation parameter is processed using a linear rectification function to obtain first intermediate data, wherein the first intermediate data is 0, or the sum of the shifted activation data and the first activation parameter. The linear rectification function is used to select the sum of the shifted activation data and the first activation parameter as the first intermediate data when the sum of the shifted activation data and the first activation parameter is greater than 0; and select 0 as the first intermediate data when the sum of the shifted activation data and the first activation parameter is not greater than 0, so that the large language model can better learn the nonlinear relationship in the processed data, thereby improving the expression ability and generalization ability of the large language model.

[0058] According to an embodiment of the present disclosure, the second activation parameter may be obtained by pre-training, and the second activation parameter is stored in a register, and the second activation parameter is integer type data. In the first intermediate data and the second activation parameter, data that meets the accuracy requirements of the large language model is screened out as the second intermediate data. Preferably, in the case where the first intermediate data and the second activation parameter are both certificate type matrices, the elements of the corresponding positions in the two matrices can be compared element by element in the first intermediate data and the second activation parameter, and the smaller value of the two elements is returned to form a new matrix, i.e., the second intermediate data.

[0059] According to an embodiment of the present disclosure, the third activation parameter may be pre-trained, and the third activation parameter is stored in a register, and the third activation parameter is integer type data. The first activation data, the second intermediate data, and the third activation parameter are multiplied by calling a multiplier to obtain the data to be shifted.

[0060] According to an embodiment of the present disclosure, a shifter is called to shift the data to be shifted to obtain second activation data, wherein the second activation data is data for making a Hadamard product with third activation data, and the third activation data is data output by a rising projection of a gated linear unit.

[0061] According to an embodiment of the present disclosure, in the process of translating text data using a large language model, in the middle layer of the large language model, the data to be activated entering the gated linear unit is shifted to obtain shifted activation data, and the first activation parameter, the second activation parameter, and the third activation parameter obtained by pre-training are used to process the data to be activated, so that the gated linear unit of the middle layer can use the quantized data to be activated as input data without making any changes, without performing an inverse quantization operation, thereby reducing the amount of calculation generated thereby. In addition, the first activation parameter, the second activation parameter, and the third activation parameter are all integer type data, so the amount of calculation generated additionally by floating-point type data in the operation process can be reduced, thereby reducing the dependence of the large language model on the computing power of the hardware platform, that is, the operations in this method can be directly executed efficiently using the basic instructions in the processor, thereby utilizing the capabilities of the existing underlying hardware to achieve faster computing speeds.

[0062] According to an embodiment of the present disclosure, depending on the training purpose of the large language model, when its purpose is to generate text in a specific format based on the input text, where the text in a specific format may include seven-character quatrains, five-character regulated verses, etc., the text data is input into the large language model, and a text sequence in a specific format is output. Specifically, the encoder of the large language model sequentially performs word segmentation, token mapping, and embedding operations on the input text data to convert the input text data into a corresponding matrix representation. The gated linear unit is used to calculate the relationship between the current token and other tokens to obtain the corresponding feature matrix.

[0063] According to an embodiment of the present disclosure, the input feature matrix is ​​normalized by the normalization layer of the middle layer to obtain the data to be activated for the input gated linear unit. The gated linear unit performs nonlinear transformation and feature extraction on the data to be activated to generate a new feature matrix, and after being processed by multiple layers of self-attention and gated linear units, a feature representation for predicting a text sequence in a specific format is generated.

[0064] According to an embodiment of the present disclosure, the large language model will calculate the probability distribution of each token in the text sequence of the specific format according to the feature representation in subsequent operations to obtain the text sequence of the specific format.

[0065] Figure 3 The figure schematically shows a middle layer diagram of a large language model according to an embodiment of the present disclosure.

[0066] like Figure 3As shown, the middle layer may include an RMS Norm function 310 and a gated linear unit 320 for normalizing the input data of the middle layer. The gated linear unit may include an up projection 321 (up_proj), a gated projection 322 (gate_proj), an activation layer 323, and a down projection 324 (down_proj).

[0067] The ascending projection 321 is used to convert the dimension of the input data to be activated from one size to another size, preferably, to convert the dimension of the data to be activated from a small size to a larger size. The gated projection 322 is used to linearly transform the data to be activated to provide a set of transformed features, and the transformed features are the first activation data. The activation layer 323 is used to implement the above-mentioned operations S230 to S270. The descending projection 324 is the last linear transformation of the gated linear unit, which is used to compress the increased dimension size back to the original size, which is the dimension size of the dimension of the data to be activated.

[0068] According to the embodiments of the present disclosure, by Figure 3 The gated linear unit of the structure shown performs complex feature mapping while maintaining the consistency of input and output dimensions, thereby enhancing the expressive power of the large language model. Since the data processing process of the activation layer 323 is completed with the participation of non-floating point data, the computational complexity of the model is reduced. Also, because the parameters in the activation layer 323 can change with the quantization factor, and the quantization factor can be gradually optimized with the training of the large language model, the large language model can still ensure the accuracy of the output results with low computational complexity.

[0069] According to an embodiment of the present disclosure, the first activation parameter, the second activation parameter and the third activation parameter are calculated based on the enlarged quantization factor, wherein the enlarged quantization factor is floating point data obtained by enlarging the quantization factor by an integer multiple.

[0070] According to an embodiment of the present disclosure, the quantization factor may be floating point data, and the quantization factor may be obtained by following the training of the large language model. The quantization factor is amplified by an integer multiple to obtain the amplified quantization factor. Preferably, the quantization factor is amplified by 4 times to obtain the amplified quantization factor. The relationship between the amplified quantization factor and the quantization factor may be expressed by the following formula (1):

[0071] (1);

[0072] in, Characterize the amplification quantization factor, Characterize the quantitative factor.

[0073] According to an embodiment of the present disclosure, after the quantization factor is amplified by a positive integer multiple, the amplified quantization factor is obtained, which can effectively reduce the probability of overflow of the value range of parameters or data such as the first activation parameter, the second activation parameter and the third activation parameter.

[0074] According to an embodiment of the present disclosure, the first activation parameter is obtained by integerizing a first initial parameter of a floating point type, wherein the first initial parameter is a quotient of a first fixed parameter and an amplified quantization factor.

[0075] According to an embodiment of the present disclosure, integerization converts floating-point data into integer data. The first fixed parameter is integer data, and preferably, the first quantization parameter may be 3. The relationship between the first fixed parameter, the enlarged quantization factor, and the first activation parameter may be expressed by the following formula (2):

[0076] (2);

[0077] in, Characterizes the first activation parameter, represents the function of integerizing the data, 3 represents the first fixed parameter, Characterize the amplification quantization factor, Characterize the first initial parameter.

[0078] According to an embodiment of the present disclosure, the second activation parameter is obtained by integerizing a second initial parameter of a floating point type, wherein the second initial parameter is a quotient of the second fixed parameter and an amplified quantization factor.

[0079] According to an embodiment of the present disclosure, the second fixed parameter is integer type data, and preferably, the first quantization parameter may be 6. The relationship between the second fixed parameter, the enlarged quantization factor and the second activation parameter may be expressed by the following formula (3):

[0080] (3);

[0081] in, represents the second activation parameter, 6 represents the second fixed parameter, Characterize the second initial parameter.

[0082] According to an embodiment of the present disclosure, the third activation parameter is obtained by integerizing the third initial parameter, wherein the third initial parameter is obtained by integer-multiplying the quotient of the amplified quantization factor and the second fixed parameter.

[0083] According to an embodiment of the present disclosure, the quotient of the amplified quantization factor and the second fixed parameter is amplified by an integer multiple. Preferably, the quotient of the amplified quantization factor and the second fixed parameter can be amplified by a positive integer power of 2. Preferably, the quotient of the amplified quantization factor and the second fixed parameter can be amplified by the 16th power of 2. The relationship between the third activation parameter, the amplified quantization factor, and the second fixed parameter can be expressed by the following formula (4):

[0084] (4);

[0085] in, Characterizes the third activation parameter, Characterize the third initial parameter.

[0086] According to an embodiment of the present disclosure, the Swish activation function is used in the related art to determine the data for making a Hadamard product with the third activation data. However, the original Swish activation function contains an e-exponential operation, and the computational complexity is high. Even if the Swish activation function is replaced with an activation function that is similar to it but does not contain an exponential operation, the quantized integer type of activation data cannot be used as input, because the activation function of the replaced approximate Swish function is still a nonlinear function, and the data obtained will obviously be wrong.

[0087] According to an embodiment of the present disclosure, based on the Swish activation function, its approximate function is used as a basis, preferably, its approximate function can be a Hard-Swish activation function. Based on its approximate function, a transformation is performed, and multiple parameters of the activation function are transformed into a form that can change the quantization factor, so that both the data to be activated and the processing process of the activated data can be calculated with integer type data. Thereby reducing the amount of calculation of the large language model in the process of processing text data.

[0088] According to an embodiment of the present disclosure, the relationship between the data to be activated and the quantization factor, and the input data using Swish as the activation function can be expressed by the following formula (5):

[0089] (5);

[0090] in, Represents the data to be activated, Characterize the quantitative factor, Characterizes the floating-point data obtained by inverse quantization of the input activation function in the related technology.

[0091] According to an embodiment of the present disclosure, shifting the first activation data by using a shifter includes: shifting the first activation data to the right by 2 bits.

[0092] According to an embodiment of the present disclosure, shifting the data to be shifted by using a shifter includes: shifting the data to be shifted rightward by 16 bits.

[0093] According to the embodiments of the present disclosure, in the process of realizing the calculation by hardware, the calculation of integer powers of 2 can be realized by means of shifting. In the process of translating text data by a large language model, the amount of data to be processed is very large. Compared with multiplication operations, especially division operations, shifting requires fewer machine cycles, has a shorter execution time, and is more efficient. Replacing the processing of the first activated data and the data to be shifted in the related art with data shifting can improve the processing efficiency of the hardware platform where the large language model is located, thereby reducing the requirements of the large language model for the computing power of the hardware platform.

[0094] Based on the above-mentioned data processing method of a large language model, the present disclosure also provides a data processing device of a large language model. Figure 4 The device is described in detail.

[0095] Figure 4 The structural block diagram of the data processing device of the large language model according to the embodiment of the present disclosure is schematically shown.

[0096] like Figure 4 As shown, the data processing device 400 of the large language model of this embodiment includes a text generation module 410.

[0097] The text generation module 410 is used to input text data into a large language model and output a text sequence obtained by translating the text data, wherein the large language model includes an intermediate layer, wherein the intermediate layer is used to perform nonlinear transformation and feature extraction on the data to be activated output by the upper layer to generate a new feature matrix. In one embodiment, the text generation module 410 can be used to perform the operation S210 described above, which will not be repeated here.

[0098] The text generation module includes: a first determination unit 411 , a first shift unit 412 , a variable determination unit 413 , a variable screening unit 414 , a multiplication unit 415 , and a second shift unit 416 .

[0099] The first determination unit 411 is used to process the data to be activated in the middle layer using the gated projection of the gated linear unit to obtain first activation data, wherein the first activation data is integer type data. In one embodiment, the first determination unit can be used to perform the operation S220 described above, which will not be repeated here.

[0100] The first shift unit 412 is used to shift the first activation data using a shifter to obtain shifted activation data. In one embodiment, the first shift unit can be used to perform the operation S230 described above, which will not be described in detail here.

[0101] The variable determination unit 413 is used to process the sum of the shifted activation data and the first activation parameter using a linear rectification function to obtain first intermediate data, wherein the first activation parameter is pre-stored in a register used by the large language model. In one embodiment, the variable determination unit can be used to perform the operation S240 described above, which will not be repeated here.

[0102] The variable screening unit 414 is used to screen out the second intermediate data from the first intermediate data and the second activation parameter, wherein the second activation parameter is pre-stored in the register. In one embodiment, the variable screening unit can be used to perform the operation S250 described above, which will not be described in detail here.

[0103] The multiplication unit 415 is used to use a multiplier to perform a multiplication operation on the first activation data, the second intermediate data and the third activation parameter to obtain the data to be shifted, wherein the third activation parameter is pre-stored in the register. In one embodiment, the multiplication unit can be used to perform the operation S260 described above, which will not be repeated here.

[0104] The second shift unit 416 is used to shift the data to be shifted using a shifter to obtain second activation data, wherein the second activation data is data for performing a Hadamard product with the third activation data, and the third activation data is data output by the ascending projection of the gated linear unit. In one embodiment, the second shift unit can be used to perform the operation S270 described above, which will not be described in detail here.

[0105] According to an embodiment of the present disclosure, the first activation parameter, the second activation parameter and the third activation parameter are calculated based on the enlarged quantization factor, wherein the enlarged quantization factor is floating point data obtained by enlarging the quantization factor by an integer multiple.

[0106] According to an embodiment of the present disclosure, the first activation parameter is obtained by integerizing a first initial parameter of a floating point type, wherein the first initial parameter is a quotient of a first fixed parameter and an amplified quantization factor.

[0107] According to an embodiment of the present disclosure, the second activation parameter is obtained by integerizing a second initial parameter of a floating point type, wherein the second initial parameter is a quotient of the second fixed parameter and an amplified quantization factor.

[0108] According to an embodiment of the present disclosure, the third activation parameter is obtained by integerizing the third initial parameter, wherein the third initial parameter is obtained by integer-multiplying the quotient of the amplified quantization factor and the second fixed parameter.

[0109] According to an embodiment of the present disclosure, shifting the first activation data by using a shifter includes: shifting the first activation data to the right by 2 bits.

[0110] According to an embodiment of the present disclosure, shifting the data to be shifted by using a shifter includes: shifting the data to be shifted rightward by 16 bits.

[0111] According to an embodiment of the present disclosure, any multiple units in the generation module 410 can be combined into one module or unit for implementation, or any one of the units can be split into multiple units. Alternatively, at least part of the functions of one or more of these units can be combined with at least part of the functions of other units and implemented in one unit. According to an embodiment of the present disclosure, at least one of the first determination unit 411, the first shift unit 412, the variable determination unit 413, the variable screening unit 414, the multiplication unit 415, and the second shift unit 416 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the first determination unit 411, the first shift unit 412, the variable determination unit 413, the variable screening unit 414, the multiplication unit 415, and the second shift unit 416 can be at least partially implemented as a computer program unit, and when the computer program unit is executed, the corresponding function can be performed.

[0112] Figure 5 A block diagram of an electronic device suitable for implementing a data processing method for a large language model according to an embodiment of the present disclosure is schematically shown.

[0113] like Figure 5 As shown, the electronic device 500 according to an embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 to a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0114] In RAM 503, various programs and data required for the operation of electronic device 500 are stored. Processor 501, ROM 502 and RAM 503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 502 and / or RAM 503. It should be noted that the program can also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0115] According to an embodiment of the present disclosure, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the input / output (I / O) interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage portion 508 as needed.

[0116] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0117] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 502 and / or RAM 503 described above and / or one or more memories other than ROM 502 and RAM 503.

[0118] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the data processing method of the large language model provided by the embodiment of the present disclosure.

[0119] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 501. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0120] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0121] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0122] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0123] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0125] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A data processing method for a large language model, characterized in that: The method comprises: Inputting text data into a large language model, and outputting a text sequence obtained by translating the text data, wherein the large language model includes an intermediate layer, wherein the intermediate layer is used to perform nonlinear transformation and feature extraction on the data to be activated output by the previous layer to generate a new feature matrix; In the middle layer, the data to be activated is processed by using a gated projection of a gated linear unit to obtain first activated data, wherein the first activated data is integer type data; Shifting the first activation data using a shifter to obtain shifted activation data; Processing the sum of the shifted activation data and a first activation parameter using a linear rectification function to obtain first intermediate data, wherein the first activation parameter is pre-stored in a register used by the large language model, and the first activation parameter is an integer type parameter; Filtering out the second intermediate data from the first intermediate data and the second activation parameter, wherein the second activation parameter is pre-stored in the register and the second activation parameter is an integer type parameter; Using a multiplier to perform a multiplication operation on the first activation data, the second intermediate data, and a third activation parameter to obtain data to be shifted, wherein the third activation parameter is pre-stored in the register and is an integer type parameter; and The shifter is used to shift the data to be shifted to obtain second activation data, where the second activation data is data used to perform a Hadamard product with third activation data, and the third activation data is data output by a rising projection of the gated linear unit.

2. The method according to claim 1, characterized in that The first activation parameter, the second activation parameter, and the third activation parameter are calculated based on an amplified quantization factor, wherein the amplified quantization factor is floating-point data obtained by amplifying the quantization factor by an integer multiple.

3. The method according to claim 2, characterized in that The first activation parameter is obtained by integerizing a first initial parameter of a floating point type, wherein the first initial parameter is a quotient of a first fixed parameter and the enlarged quantization factor.

4. The method according to claim 2, characterized in that: The second activation parameter is obtained by integerizing a second initial parameter of a floating point type, wherein the second initial parameter is a quotient of a second fixed parameter and the enlarged quantization factor.

5. The method according to claim 4, characterized in that The third activation parameter is obtained by integerizing the third initial parameter, wherein the third initial parameter is obtained by integer-multiplying the quotient of the enlarged quantization factor and the second fixed parameter.

6. The method according to claim 1, characterized in that The shifting the first activation data by using a shifter includes: The first activation data is shifted right by 2 bits.

7. The method according to claim 1, characterized in that The step of shifting the data to be shifted by using the shifter comprises: The data to be shifted is shifted rightward by 16 bits.

8. A data processing device for a large language model, characterized in that: The device comprises: A text generation module, used for inputting text data into a large language model, and outputting a text sequence obtained by translating the text data, wherein the large language model includes an intermediate layer, wherein the intermediate layer is used for performing nonlinear transformation and feature extraction on the data to be activated outputted by the previous layer, so as to generate a new feature matrix; Wherein, the text generation module includes: A first determining unit is used to process the data to be activated in the intermediate layer by using a gated projection of a gated linear unit to obtain first activation data, wherein the first activation data is integer type data; A first shift unit, used for shifting the first activation data using a shifter to obtain shifted activation data; a variable determination unit, configured to process the sum of the shifted activation data and a first activation parameter using a linear rectification function to obtain first intermediate data, wherein the first activation parameter is pre-stored in a register used by the large language model; a variable screening unit, configured to screen out second intermediate data from the first intermediate data and a second activation parameter, wherein the second activation parameter is pre-stored in the register; a multiplication unit, configured to use a multiplier to perform a multiplication operation on the first activation data, the second intermediate data, and a third activation parameter to obtain data to be shifted, wherein the third activation parameter is pre-stored in the register; and The second shift unit is used to shift the data to be shifted by using the shifter to obtain second activation data, wherein the second activation data is data used to make a Hadamard product with third activation data, and the third activation data is data output by the ascending projection of the gated linear unit.

9. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images