Processing method and device for introducing linear attention mechanism into large language model

Abstract

The embodiment of the invention relates to a processing method and device for introducing a linear attention mechanism into a large language model. The method comprises the steps that a conventional large language model achieved based on a Transform model architecture serves as a corresponding base model; the receiving model improvement scheme is recorded as a corresponding first scheme; resetting an attention function of the base model based on a first attention function configuration set of the first scheme, and recording the reset model as a corresponding first large language model; training and evaluating the first large language model based on a model training data set corresponding to the first data set identifier of the first scheme to obtain a corresponding first evaluation value; and when the first evaluation value exceeds the first evaluation index of the first scheme, taking the first large language model as an improved version model of the base version model. Through the method, the computing resource occupation amount of the large language model can be reduced, and the computing efficiency of the large language model is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a processing method and device for introducing a linear attention mechanism into a large language model. Background Art

[0002] Large Language Models (LLMs) based on the Transformer model architecture include the BERT series of models and the GPT series of models. These large language models use the Transformer's self-attention mechanism to perform language modeling and representation learning, which can capture contextual information and generate high-quality text. However, we also found that these large language models usually require a large amount of computing resources to process natural language tasks, and the computing efficiency will be significantly reduced when processing long text tasks. One of the main reasons for this problem is that the self-attention processing mechanism used in these large language models is usually based on the softmax attention mechanism. From the well-known softmax attention algorithm, it can be seen that the computing resources consumed by the algorithm during operation increase in a quadratic increment. Therefore, when processing long text tasks, large language models using the softmax attention mechanism will inevitably consume too much computing resources and have a significant decrease in computing efficiency. Summary of the invention

[0003] The purpose of the present invention is to provide a processing method, device, electronic device and computer-readable storage medium for introducing a linear attention mechanism into a large language model in view of the defects of the prior art. The present invention takes a conventional large language model implemented based on the Transformer model architecture as the corresponding base model, and provides a processing mechanism for dynamically configuring the attention mechanism of each coding unit for the base model, and provides at least four types of linear attention functions (Diag-Attention function, Norm-Attention function, Linear-Attention function, Focused-Linear-Attention function) for selection; and the base model introduced with the linear attention mechanism based on the configuration scheme is recorded as the first large language model, and the first large language model is trained and evaluated based on the model training data set of the base model, and the first large language model is launched as an improved version of the base model when the evaluation value exceeds the evaluation index. On the one hand, the present invention reduces the computing resource usage of the large language model and improves the computing efficiency of the large language model by replacing the softmax attention mechanism with a linear attention mechanism; on the other hand, it also provides users with a dynamic configuration method that can introduce multiple linear attention mechanisms.

[0004] To achieve the above object, a first aspect of an embodiment of the present invention provides a processing method for introducing a linear attention mechanism into a large language model, the method comprising:

[0005] A conventional large language model implemented based on the Transformer model architecture is used as the corresponding base model; the first encoder of the conventional large language model is composed of N first encoding units connected in sequence, and the total number of encoding units N is a positive integer; each of the first encoding units includes a corresponding first multi-head self-attention unit; each of the first multi-head self-attention units includes multiple parallel single-head self-attention units; each of the single-head self-attention units performs self-attention calculation based on the Softmax attention function;

[0006] The received model improvement plan is recorded as the corresponding first plan; the first plan includes a first attention function configuration set, a first data set identifier and a first evaluation index; the first attention function configuration set includes N first attention function configuration types, and the first attention function configuration type corresponds to the first encoding unit one by one; the first attention function configuration type includes at least a Softmax attention function and four types of linear attention functions, and the four types of linear attention functions include a Diag-Attention function, a Norm-Attention function, a Linear-Attention function, and a Focused-Linear-Attention function; the first data set identifier is a data set identifier of a model training data set corresponding to the conventional large language model; the first evaluation index is a model evaluation threshold;

[0007] Based on the first attention function configuration set of the first scheme, the attention function of the base model is reset and the reset model is recorded as the corresponding first large language model; and based on the model training data set corresponding to the first data set identifier of the first scheme, the first large language model is trained and evaluated to obtain a corresponding first evaluation value; and when the first evaluation value exceeds the first evaluation indicator of the first scheme, the first large language model is used as an improved model of the base model.

[0008] Preferably, the conventional large language model includes at least a GPT series model and a BERT series model; the conventional large language model has completed model pre-training and model fine-tuning, and can handle at least one type of natural language processing task;

[0009] The conventional large language model performs corresponding natural language text generation processing based on the natural language understanding of the first instruction text input to the model when processing the natural language processing task and outputs the corresponding first generated text; the model input end of the conventional large language model is used to receive the first instruction text, and the model output end is used to output the corresponding first generated text;

[0010] The model components of the conventional large language model include at least a first preprocessing layer, a first embedding layer, the first encoder, a first pooling layer, a first fully connected network, and a first output layer;

[0011] The input end of the first preprocessing layer is connected to the input end of the model, and the output end is connected to the input end of the first embedding layer; the output end of the first embedding layer is connected to the input end of the first encoder; the output end of the first encoder is connected to the input end of the first pooling layer; the output end of the first pooling layer is connected to the input end of the first fully connected network; the output end of the first fully connected network is connected to the input end of the first output layer;

[0012] The first preprocessing layer is used to perform word segmentation, stop word removal and part-of-speech tagging on the first instruction text to obtain a corresponding first word segmentation sequence and send it to the first embedding layer;

[0013] The first embedding layer is used to perform word embedding coding and word position coding on the first word segmentation sequence to obtain a corresponding first embedded coding vector and send it to the first encoder;

[0014] The first encoder is used to perform deep learning on the associated features of each word segmentation text and other word segmentation texts in the first instruction text according to the first embedded coding vector to obtain the corresponding first feature vector and send it to the first pooling layer; the first encoder is implemented based on the Transformer model Encoder component structure, and is composed of N first encoding units connected in sequence; each of the first encoding units is composed of the first multi-head self-attention unit, the first residual connection and normalization unit, the first feedforward neural network, and the second residual connection and normalization unit connected in sequence; each of the first multi-head self-attention units is composed of a group of multiple parallel single-head self-attention units, a single-head splicing unit and a linear layer connected in sequence; each of the first multi-head self-attention units corresponds to a class of attention functions, and the attention function corresponding to each of the first multi-head self-attention units is configurable, and the attention function used by all the single-head self-attention units of each of the first multi-head self-attention units is the attention function corresponding to the current multi-head self-attention unit; under normal circumstances, the attention function corresponding to all the first multi-head self-attention units of the conventional large language model is set to the Softmax attention function;

[0015] The first pooling layer is used to downsample the first feature vector through a pooling operation to obtain a corresponding second feature vector and send it to the first fully connected network;

[0016] The first fully connected network is composed of one fully connected layer or multiple fully connected layers connected in sequence; the first fully connected network is used to perform a fully connected transformation on the second feature vector to obtain a corresponding third feature vector and send it to the first output layer;

[0017] The first output layer corresponds to a text generator model of a type of natural language processing task; the first output layer is used to perform text generation processing of the corresponding task according to the third feature vector and output the corresponding first generated text.

[0018] Preferably, the first attention function configuration set based on the first solution resets the attention function of the base model and records the reset model as the corresponding first language model, specifically including:

[0019] A round of traversal is performed on all the first attention function configuration types of the first attention function configuration set; and in this round of traversal, the first attention function configuration type currently traversed is used as the corresponding current attention function configuration type; and the first encoding unit corresponding to the current attention function configuration type in the base model is used as the corresponding current encoding unit; and it is identified whether the attention function currently corresponding to the current encoding unit matches the current attention function configuration type, and if not, the attention function corresponding to the current encoding unit is reset based on the current attention function configuration type; and at the end of this round of traversal, the reset model is recorded as the corresponding first large language model.

[0020] A second aspect of an embodiment of the present invention provides a device for implementing the processing method of introducing a linear attention mechanism into a large language model as described in the first aspect, the device comprising: a base model determination module, a configuration scheme receiving module and an attention mechanism introduction module;

[0021] The base model determination module is used to use a conventional large language model implemented based on the Transformer model architecture as the corresponding base model; the first encoder of the conventional large language model is composed of N first encoding units connected in sequence, and the total number of encoding units N is a positive integer; each of the first encoding units includes a corresponding first multi-head self-attention unit; each of the first multi-head self-attention units includes multiple parallel single-head self-attention units; each of the single-head self-attention units performs self-attention calculation based on the Softmax attention function;

[0022] The configuration scheme receiving module is used to receive a model improvement scheme recorded as a corresponding first scheme; the first scheme includes a first attention function configuration set, a first data set identifier and a first evaluation index; the first attention function configuration set includes N first attention function configuration types, and the first attention function configuration type corresponds to the first encoding unit one by one; the first attention function configuration type includes at least a Softmax attention function and four types of linear attention functions, and the four types of linear attention functions include a Diag-Attention function, a Norm-Attention function, a Linear-Attention function, and a Focused-Linear-Attention function; the first data set identifier is a data set identifier of a model training data set corresponding to the conventional large language model; the first evaluation index is a model evaluation threshold;

[0023] The attention mechanism introduction module is used to reset the attention function of the base model based on the first attention function configuration set of the first scheme and record the reset model as the corresponding first large language model; and train and evaluate the first large language model based on the model training data set corresponding to the first data set identifier of the first scheme to obtain a corresponding first evaluation value; and when the first evaluation value exceeds the first evaluation indicator of the first scheme, use the first large language model as an improved model of the base model.

[0024] A third aspect of an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0025] The processor is used to be coupled to the memory, read and execute instructions in the memory, so as to implement the method steps described in the first aspect above;

[0026] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0027] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed by a computer, the computer executes the instructions of the method described in the first aspect above.

[0028] The embodiment of the present invention provides a processing method, device, electronic device and computer-readable storage medium for introducing a linear attention mechanism into a large language model. As can be seen from the above content, the embodiment of the present invention uses a conventional large language model implemented based on the Transformer model architecture as the corresponding base model, and provides a processing mechanism for dynamically configuring the attention mechanism of each coding unit for the base model, and provides at least four types of linear attention functions (Diag-Attention function, Norm-Attention function, Linear-Attention function, Focused-Linear-Attention function) for selection; and the base model introduced with the linear attention mechanism based on the configuration scheme is recorded as the first large language model, and the first large language model is trained and evaluated based on the model training data set of the base model, and the first large language model is launched as an improved version of the base model when the evaluation value exceeds the evaluation index. On the one hand, the embodiment of the present invention reduces the computing resource usage of the large language model and improves the computing efficiency of the large language model by using the linear attention mechanism to replace the softmax attention mechanism; on the other hand, it also provides users with a dynamic configuration method that can introduce multiple linear attention mechanisms, thereby achieving the purpose of improving the flexibility of model configuration. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a processing method for introducing a linear attention mechanism into a large language model provided in Embodiment 1 of the present invention;

[0030] Figure 2 A module structure diagram of a conventional large language model provided in the first embodiment of the present invention;

[0031] Figure 3 A module structure diagram of a processing device for introducing a linear attention mechanism into a large language model provided in Embodiment 2 of the present invention;

[0032] Figure 4 A schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] Embodiment 1 of the present invention provides a processing method for introducing a linear attention mechanism into a large language model, such as Figure 1 As shown in the schematic diagram of a processing method for introducing a linear attention mechanism into a large language model provided in the first embodiment of the present invention, the method mainly includes the following steps:

[0035] Step 1: Use a conventional large language model based on the Transformer model architecture as the corresponding base model.

[0036] Here, the conventional large language model of the embodiment of the present invention includes at least a GPT series model and a BERT series model; and the conventional large language model has completed model pre-training and model fine-tuning, and can handle at least one type of natural language processing task.

[0037] When processing a natural language processing task, a conventional large language model performs corresponding natural language text generation processing based on the natural language understanding of the first instruction text input to the model and outputs the corresponding first generated text. Figure 2 As shown in the module structure diagram of the conventional large language model provided in the first embodiment of the present invention, the model input end of the conventional large language model is used to receive the first instruction text, and the model output end is used to output the corresponding first generated text.

[0038] like Figure 2 As shown, the model components of the conventional large language model of the embodiment of the present invention include at least a first preprocessing layer, a first embedding layer, a first encoder, a first pooling layer, a first fully connected network and a first output layer.

[0039] The connection relationship between the model components is: the input end of the first preprocessing layer is connected to the model input end, and the output end is connected to the input end of the first embedding layer; the output end of the first embedding layer is connected to the input end of the first encoder; the output end of the first encoder is connected to the input end of the first pooling layer; the output end of the first pooling layer is connected to the input end of the first fully connected network; the output end of the first fully connected network is connected to the input end of the first output layer.

[0040] A brief functional description of each model component is given below.

[0041] 1) The first preprocessing layer is used to perform word segmentation, stop word removal and part-of-speech tagging on the first instruction text to obtain a corresponding first word segmentation sequence and send it to the first embedding layer;

[0042] 2) The first embedding layer is used to perform word embedding coding and word position coding on the first word segmentation sequence to obtain a corresponding first embedded coding vector and send it to the first encoder;

[0043] 3) The first encoder is used to perform deep learning on the associated features of each segmented text and other segmented texts in the first instruction text according to the first embedded coding vector to obtain a corresponding first feature vector and send it to the first pooling layer;

[0044] It should be noted that if Figure 2 As shown, the first encoder of the embodiment of the present invention is implemented based on the Transformer model Encoder component structure, and is composed of N first encoding units connected in sequence, where the total number of encoding units N is a positive integer; each first encoding unit is composed of a first multi-head self-attention unit, a first residual connection and normalization unit, a first feedforward neural network, and a second residual connection and normalization unit connected in sequence; each first multi-head self-attention unit is composed of a group of multiple parallel single-head self-attention units, a single-head splicing unit and a linear layer connected in sequence;

[0045] It should also be noted that each first multi-head self-attention unit corresponds to a type of attention function, and the attention function corresponding to each first multi-head self-attention unit is configurable, and the attention function used by all single-head self-attention units of each first multi-head self-attention unit is the attention function corresponding to the current multi-head self-attention unit; Under normal circumstances, the attention functions corresponding to all first multi-head self-attention units of the conventional large language model of the embodiment of the present invention are set to Softmax attention function;

[0046] 4) The first pooling layer is used to downsample the first feature vector through a pooling operation to obtain a corresponding second feature vector and send it to the first fully connected network;

[0047] 5) The first fully connected network is composed of one fully connected layer or multiple layers of sequentially connected fully connected layers; the first fully connected network is used to perform a fully connected transformation on the second eigenvector to obtain a corresponding third eigenvector and send it to the first output layer;

[0048] 6) The first output layer corresponds to a text generator model of a type of natural language processing task; the first output layer is used to perform text generation processing for the corresponding task according to the third feature vector and output the corresponding first generated text.

[0049] Step 2: The received model improvement plan is recorded as the corresponding first plan.

[0050] Here, the model improvement scheme of the embodiment of the present invention, that is, the first scheme, can dynamically configure the attention mechanism of the conventional large language model at the coding unit level, and can configure different attention functions for different coding units.

[0051] It should be noted that the first scheme of the embodiment of the present invention consists of a first attention function configuration set, a first data set identifier and a first evaluation indicator; wherein, the first attention function configuration set includes N first attention function configuration types, and the first attention function configuration types correspond one-to-one to the first encoding unit; the first attention function configuration types include at least a Softmax attention function and four types of linear attention functions, and the four types of linear attention functions include a Diag-Attention function, a Norm-Attention function, a Linear-Attention function, and a Focused-Linear-Attention function; the first data set identifier is a data set identifier of a model training data set corresponding to a conventional large language model; and the first evaluation indicator is a model evaluation threshold.

[0052] The four types of linear attention functions mentioned here are all public linear attention functions. The function descriptions of the Diag-Attention function and the Norm-Attention function can be obtained by querying the technical document "The Devil in Linear Transformer", and the function descriptions of the Linear-Attention function and the Focused-Linear-Attention function can be obtained by querying the technical document "FLatten Transformer: Vision Transformer usingFocused Linear Attention".

[0053] Step 3: reset the attention function of the base model based on the first attention function configuration set of the first solution and record the reset model as the corresponding first language model; train and evaluate the first language model based on the model training data set corresponding to the first data set identifier of the first solution to obtain the corresponding first evaluation value; and when the first evaluation value exceeds the first evaluation index of the first solution, use the first language model as an improved model of the base model;

[0054] Specifically, it includes: step 31, resetting the attention function of the base model based on the first attention function configuration set of the first solution and recording the reset model as the corresponding first language model;

[0055] Specifically, it includes: performing a round of traversal on all first attention function configuration types of the first attention function configuration set; and in this round of traversal, taking the currently traversed first attention function configuration type as the corresponding current attention function configuration type; and taking the first coding unit corresponding to the current attention function configuration type in the base model as the corresponding current coding unit; and identifying whether the attention function currently corresponding to the current coding unit matches the current attention function configuration type, and if not, resetting the attention function corresponding to the current coding unit based on the current attention function configuration type; and at the end of this round of traversal, recording the reset model as the corresponding first language model;

[0056] Step 32: training and evaluating the first language model based on the model training data set corresponding to the first data set identifier of the first solution to obtain a corresponding first evaluation value;

[0057] Specifically, the method includes: recording the model training data set corresponding to the first data set identifier as the corresponding first data set; performing training set and evaluation set segmentation on the first data set according to the training set and evaluation set segmentation method corresponding to the conventional large language model to obtain the corresponding first training set and first evaluation set; training the first large language model according to the first training set according to the model training method and training loss function corresponding to the conventional large language model; and performing model evaluation on the first large language model according to the first evaluation set according to the model evaluation method corresponding to the conventional large language model and using the obtained evaluation value as the corresponding first evaluation value;

[0058] In step 33, when the first evaluation value exceeds the first evaluation index of the first solution, the first large language model is used as an improved model of the base model.

[0059] Figure 3 The module structure diagram of a processing device for introducing a linear attention mechanism into a large language model provided in the second embodiment of the present invention, the device is a terminal device or server that implements the aforementioned method embodiment, and can also be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiment, for example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 3 As shown, the device includes: a base model determination module 201, a configuration scheme receiving module 202 and an attention mechanism introduction module 203.

[0060] The base model determination module 201 is used to take a class of conventional large language models implemented based on the Transformer model architecture as the corresponding base model; the first encoder of the conventional large language model is composed of N first encoding units connected in sequence, and the total number of encoding units N is a positive integer; each first encoding unit includes a corresponding first multi-head self-attention unit; each first multi-head self-attention unit includes multiple parallel single-head self-attention units; each single-head self-attention unit performs self-attention calculation based on the Softmax attention function.

[0061] The configuration scheme receiving module 202 is used to receive a model improvement scheme recorded as the corresponding first scheme; the first scheme includes a first attention function configuration set, a first data set identifier and a first evaluation indicator; the first attention function configuration set includes N first attention function configuration types, and the first attention function configuration types correspond one-to-one to the first encoding unit; the first attention function configuration types include at least a Softmax attention function and four types of linear attention functions, and the four types of linear attention functions include a Diag-Attention function, a Norm-Attention function, a Linear-Attention function, and a Focused-Linear-Attention function; the first data set identifier is a data set identifier of a model training data set corresponding to a conventional large language model; and the first evaluation indicator is a model evaluation threshold.

[0062] The attention mechanism introduction module 203 is used to reset the attention function of the base model based on the first attention function configuration set of the first scheme and record the reset model as the corresponding first language model; and train and evaluate the first language model based on the model training data set corresponding to the first data set identifier of the first scheme to obtain the corresponding first evaluation value; and when the first evaluation value exceeds the first evaluation index of the first scheme, use the first language model as an improved model of the base model.

[0063] An embodiment of the present invention provides a processing device for introducing a linear attention mechanism into a large language model, which can execute the method steps in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0064] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the base model determination module can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0065] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module above is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0066] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the above method embodiments are generated. The above computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above-mentioned computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, Bluetooth, microwave, etc.) methods. The above-mentioned computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The above-mentioned available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0067] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. The electronic device may be a terminal device or a server for implementing the method of the aforementioned embodiment, or may be a terminal device or a server for implementing the method of the aforementioned embodiment connected to the aforementioned terminal device or server. Figure 4 As shown, the electronic device may include: a processor 301 (such as a CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiver 303. Various instructions may be stored in the memory 302 to complete various processing functions and implement the processing steps described in the aforementioned embodiment method. Preferably, the electronic device involved in the embodiment of the present invention also includes: a power supply 304, a system bus 305 and a communication port 306. The system bus 305 is used to realize the communication connection between components. The above-mentioned communication port 306 is used for connecting and communicating between the electronic device and other peripherals.

[0068] exist Figure 4The system bus 305 mentioned in the figure can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize the communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM), and may also include non-volatile memory (Non-Volatile Memory), such as at least one disk storage.

[0069] The above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (Network Processor, NP), a graphics processing unit (Graphics Processing Unit, GPU), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0070] It should be noted that an embodiment of the present invention further provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the method and processing process provided in the above embodiments.

[0071] The embodiment of the present invention provides a processing method, device, electronic device and computer-readable storage medium for introducing a linear attention mechanism into a large language model. As can be seen from the above content, the embodiment of the present invention uses a conventional large language model implemented based on the Transformer model architecture as the corresponding base model, and provides a processing mechanism for dynamically configuring the attention mechanism of each coding unit for the base model, and provides at least four types of linear attention functions (Diag-Attention function, Norm-Attention function, Linear-Attention function, Focused-Linear-Attention function) for selection; and the base model introduced with the linear attention mechanism based on the configuration scheme is recorded as the first large language model, and the first large language model is trained and evaluated based on the model training data set of the base model, and the first large language model is launched as an improved version of the base model when the evaluation value exceeds the evaluation index. On the one hand, the embodiment of the present invention reduces the computing resource usage of the large language model and improves the computing efficiency of the large language model by using the linear attention mechanism to replace the softmax attention mechanism; on the other hand, it also provides users with a dynamic configuration method that can introduce multiple linear attention mechanisms, thereby achieving the purpose of improving the flexibility of model configuration.

[0072] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0073] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for introducing a linear attention mechanism into a large language model, characterized in that: The method comprises: A conventional large language model implemented based on the Transformer model architecture is used as the corresponding base model; the first encoder of the conventional large language model is composed of N first encoding units connected in sequence, and the total number of encoding units N is a positive integer; each of the first encoding units includes a corresponding first multi-head self-attention unit; each of the first multi-head self-attention units includes multiple parallel single-head self-attention units; each of the single-head self-attention units performs self-attention calculation based on the Softmax attention function; The received model improvement plan is recorded as the corresponding first plan; the first plan includes a first attention function configuration set, a first data set identifier and a first evaluation index; the first attention function configuration set includes N first attention function configuration types, and the first attention function configuration type corresponds to the first encoding unit one by one; the first attention function configuration type includes at least a Softmax attention function and four types of linear attention functions, and the four types of linear attention functions include a Diag-Attention function, a Norm-Attention function, a Linear-Attention function, and a Focused-Linear-Attention function; the first data set identifier is a data set identifier of a model training data set corresponding to the conventional large language model; the first evaluation index is a model evaluation threshold; Based on the first attention function configuration set of the first scheme, the attention function of the base model is reset and the reset model is recorded as the corresponding first large language model; and based on the model training data set corresponding to the first data set identifier of the first scheme, the first large language model is trained and evaluated to obtain a corresponding first evaluation value; and when the first evaluation value exceeds the first evaluation indicator of the first scheme, the first large language model is used as an improved model of the base model.

2. The method for introducing a linear attention mechanism into a large language model according to claim 1, characterized in that: The conventional large language model includes at least a GPT series model and a BERT series model; the conventional large language model has completed model pre-training and model fine-tuning, and can handle at least one type of natural language processing task; The conventional large language model performs corresponding natural language text generation processing based on the natural language understanding of the first instruction text input to the model when processing the natural language processing task and outputs the corresponding first generated text; The model input end of the conventional large language model is used to receive the first instruction text, and the model output end is used to output the corresponding first generated text; The model components of the conventional large language model include at least a first preprocessing layer, a first embedding layer, the first encoder, a first pooling layer, a first fully connected network, and a first output layer; The input end of the first preprocessing layer is connected to the input end of the model, and the output end is connected to the input end of the first embedding layer; the output end of the first embedding layer is connected to the input end of the first encoder; the output end of the first encoder is connected to the input end of the first pooling layer; the output end of the first pooling layer is connected to the input end of the first fully connected network; the output end of the first fully connected network is connected to the input end of the first output layer; The first preprocessing layer is used to perform word segmentation, stop word removal and part-of-speech tagging on the first instruction text to obtain a corresponding first word segmentation sequence and send it to the first embedding layer; The first embedding layer is used to perform word embedding coding and word position coding on the first word segmentation sequence to obtain a corresponding first embedded coding vector and send it to the first encoder; The first encoder is used to perform deep learning on the associated features of each word segmentation text and other word segmentation texts in the first instruction text according to the first embedded coding vector to obtain the corresponding first feature vector and send it to the first pooling layer; the first encoder is implemented based on the Transformer model Encoder component structure, and is composed of N first encoding units connected in sequence; each of the first encoding units is composed of the first multi-head self-attention unit, the first residual connection and normalization unit, the first feedforward neural network, and the second residual connection and normalization unit connected in sequence; each of the first multi-head self-attention units is composed of a group of multiple parallel single-head self-attention units, a single-head splicing unit and a linear layer connected in sequence; each of the first multi-head self-attention units corresponds to a class of attention functions, and the attention function corresponding to each of the first multi-head self-attention units is configurable, and the attention function used by all the single-head self-attention units of each of the first multi-head self-attention units is the attention function corresponding to the current multi-head self-attention unit; Under normal circumstances, the attention functions corresponding to all the first multi-head self-attention units of the conventional large language model are set to Softmax attention function; The first pooling layer is used to downsample the first feature vector through a pooling operation to obtain a corresponding second feature vector and send it to the first fully connected network; The first fully connected network is composed of one fully connected layer or multiple fully connected layers connected in sequence; the first fully connected network is used to perform a fully connected transformation on the second feature vector to obtain a corresponding third feature vector and send it to the first output layer; The first output layer corresponds to a text generator model of a type of natural language processing task; the first output layer is used to perform text generation processing of the corresponding task according to the third feature vector and output the corresponding first generated text.

3. The method for introducing a linear attention mechanism into a large language model according to claim 1, characterized in that: The first attention function configuration set based on the first solution resets the attention function of the base model and records the reset model as the corresponding first language model, specifically including: A round of traversal is performed on all the first attention function configuration types of the first attention function configuration set; and in this round of traversal, the first attention function configuration type currently traversed is used as the corresponding current attention function configuration type; and the first encoding unit corresponding to the current attention function configuration type in the base model is used as the corresponding current encoding unit; and it is identified whether the attention function currently corresponding to the current encoding unit matches the current attention function configuration type, and if not, the attention function corresponding to the current encoding unit is reset based on the current attention function configuration type; and at the end of this round of traversal, the reset model is recorded as the corresponding first large language model.

4. A device for executing the processing method of introducing a linear attention mechanism into a large language model according to any one of claims 1 to 3, characterized in that: The device comprises: a base model determination module, a configuration scheme receiving module and an attention mechanism introduction module; The base model determination module is used to use a conventional large language model implemented based on the Transformer model architecture as the corresponding base model; the first encoder of the conventional large language model is composed of N first encoding units connected in sequence, and the total number of encoding units N is a positive integer; each of the first encoding units includes a corresponding first multi-head self-attention unit; each of the first multi-head self-attention units includes multiple parallel single-head self-attention units; each of the single-head self-attention units performs self-attention calculation based on the Softmax attention function; The configuration scheme receiving module is used to receive a model improvement scheme recorded as a corresponding first scheme; the first scheme includes a first attention function configuration set, a first data set identifier and a first evaluation index; the first attention function configuration set includes N first attention function configuration types, and the first attention function configuration type corresponds to the first encoding unit one by one; the first attention function configuration type includes at least a Softmax attention function and four types of linear attention functions, and the four types of linear attention functions include a Diag-Attention function, a Norm-Attention function, a Linear-Attention function, and a Focused-Linear-Attention function; the first data set identifier is a data set identifier of a model training data set corresponding to the conventional large language model; the first evaluation index is a model evaluation threshold; The attention mechanism introduction module is used to reset the attention function of the base model based on the first attention function configuration set of the first scheme and record the reset model as the corresponding first large language model; and train and evaluate the first large language model based on the model training data set corresponding to the first data set identifier of the first scheme to obtain a corresponding first evaluation value; and when the first evaluation value exceeds the first evaluation indicator of the first scheme, use the first large language model as an improved model of the base model.

5. An electronic device, characterized in that: include: memory, processors, and transceivers; The processor is used to couple with the memory, read and execute instructions in the memory, so as to implement the method according to any one of claims 1 to 3; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a computer, the computer is enabled to execute the method according to any one of claims 1 to 3.

Images