Large language model optimization method, device, electronic device and storage medium
By embedding side branch networks in the backbone network of the large language model and training the loss function, the problems of LLMs inference delay and resource consumption are solved, and the semi-autoregressive decoding capability is improved and the output quality is maintained.
Patent Information
- Application Number
- CN202410796661.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-20
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-06-20
AI Technical Summary
Existing large language models (LLMs) have problems with low computational efficiency and time delay in inference delay, especially in scenarios with limited resources. The autoregressive model lacks semi-autoregressive decoding capabilities, resulting in large resource and time consumption. The output quality is degraded.
By combining the original LLM model and the large model SFT dataset, and embedding side branch networks in the backbone network of the original LLM model, the new LLM model is trained using loss function, so that it has semi-autoregressive decoding capabilities and execute candidate word elements in parallel Sequence generation and verification ensure output quality.
It realizes that without training semi-autoregressive models from scratch, the new LLM model has semi-autoregressive decoding capabilities, saving resources and time, while improving decoding efficiency and maintaining output quality.
Smart Images

Figure CN118364870B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and more specifically, to an optimization method, device, electronic device and storage medium for a large language model. Background Art
[0002] In recent years, LLMs based on the Transformer architecture have undergone rapid evolution. The parameters of LLMs have increased rapidly, ranging from billions to trillions, such as models such as Chat-GPT and LLaMA-2. Although LLMs have demonstrated diverse and powerful generation capabilities, they face challenges in inference latency due to the computational burden brought by a large number of parameters. Therefore, accelerating the inference process of LLMs has become an important concern, especially in resource-limited scenarios such as edge devices, real-time applications, chatbots, etc.
[0003] The commonly used decoder-only LLMs follow the autoregressive generation method for reasoning. Each generated word needs to perform an independent reasoning process, resulting in a large number of Transformer calls during the reasoning process. These calls are subject to limitations related to memory bandwidth, resulting in reduced computational efficiency and increased latency. Therefore, the key strategy to speed up LLMs is to minimize the number of model reasoning calls while outputting the same number of words.
[0004] Semi-autoregressive decoding technology is a feasible and commonly used method to accelerate large model inference, which reduces the number of model inference calls by generating multiple tokens in a single model inference step. However, most current LLMs are autoregressive models and lack the ability of semi-autoregressive decoding. Moreover, due to the inconsistency between the decoding target of semi-autoregressive and the decoding target of autoregressive, it is challenging to convert an autoregressive model into a semi-autoregressive model with a small amount of retraining. If a semi-autoregressive model is trained from scratch, it is almost impractical to consume a lot of resources and time. In addition, semi-autoregressive models usually suffer from quality degradation in the quality of the output results.
[0005] To this end, the present invention provides an optimization method, device, electronic device and storage medium for a large language model, which can enable the LLM model to have semi-autoregressive decoding capabilities without the need to train a semi-autoregressive model from scratch, and the quality of the output results is the same as that of the original LLM model; thereby saving resource consumption and time, enhancing the decoding capability of the LLM model, and ensuring the quality of the LLM model output results. Summary of the invention
[0006] In order to overcome the shortcomings of the prior art, the present invention provides an optimization method, device, electronic device and storage medium for a large language model, which can enable the LLM model to have semi-autoregressive decoding capability without the need to train a semi-autoregressive model from scratch, and the quality of the output result is the same as that of the original LLM model; thereby saving resource consumption and time, enhancing the decoding capability of the LLM model, and ensuring the quality of the output result of the LLM model.
[0007] The technical solution adopted by the present invention to solve the technical problem is: a method for optimizing a large language model, the improvement of which is that the method for optimizing a large language model specifically comprises the following steps:
[0008] S10, combining the original LLM model and the large model SFT data set to generate a training data set;
[0009] S20, embedding a side branch network into the backbone network of the original LLM model to obtain a new LLM model, and inputting a masked word sequence into the input end of the side branch network;
[0010] S30, using the loss function to train the new LLM model on the training data set. After the training is completed, the LLM model can predict multiple candidate word-meta sequences in one reasoning;
[0011] S40, executing the generation of the candidate word-gram sequence and the verification of the correctness of the candidate word-gram sequence in parallel.
[0012] Furthermore, in step S10, the training data includes a question word sequence and an answer word sequence, wherein the question word sequence is a sample collected from the large model SFT data set and serves as the input of the original LLM model; the answer word sequence is generated by the original LLM model in combination with the question word sequence through greedy sampling.
[0013] Furthermore, in step S20, the backbone network includes a first language head module, a plurality of Transformer modules and a word unit generator; the output end of the word unit generator is connected to the input end of the first Transformer module; the output end of the previous Transformer module between adjacent Transformer modules is connected to the input end of the next Transformer module, wherein the Transformer module close to the word unit generator is the previous Transformer module, and the Transformer module far from the word unit generator is the next Transformer module; the output end of the last Transformer module is connected to the input end of the first language head module.
[0014] Furthermore, the side branch network includes several dimensionality reduction modules, several LST modules, a splicing module, a second language head module and a dimensionality increase module; the input end of the splicing module is connected to the output end of the word unit generator through the dimensionality reduction module, and the output end of the splicing module is connected to the input end of the first LST module; the output end of the previous LST module between adjacent LST modules is connected to the input end of the next LST module, wherein the LST module close to the splicing module is the previous LST module, and the LST module far from the splicing module is the next LST module; the output end of the last LST module is connected to the input end of the second language head module through the dimensionality increase module; the number of the LST modules is the same as the number of Transformer modules, and the input end of the LST module at the same position is connected to the output end of the Transformer module through the dimensionality reduction module.
[0015] Furthermore, in step S30, the loss function is expressed as:
[0016] ;
[0017] In the formula, The number of masked word sequences to set; Indicates from the range A randomly selected index from Indicates the length of the answer word sequence; Represents that based on the training input The semi-autoregressive output is the label The posterior probability of Represents the logarithm operator.
[0018] Furthermore, in step S40, the specific process of executing the generation of the candidate word-unit sequence and the verification of the correctness of the candidate word-unit sequence in parallel is as follows:
[0019] S401, during the first inference, the backbone network inputs the test word sequence , the side branch network inputs the masked word sequence , and the masked word sequence The number of , the new LLM model generates an autoregressive word prediction result and Semi-autoregressive word prediction results, where the autoregressive word is the prediction result as the final output result of the first inference , and the new LLM model will evaluate the confidence score of each semi-autoregressive word prediction result and extract the Top-K candidate words according to the confidence score to construct a candidate word tree, forming a total number of candidate words to be verified. The candidate word sequence set ;
[0020] S402, at the nth reasoning, , the backbone network inputs the test word sequence ,forward The word sequence that is accepted after the inference verification is successful and the candidate word sequence to be verified in this reasoning , the number of masked tokens in the side branch network input is The masked word sequence , and the new LLM model generates The autoregressive word prediction results and semi-autoregressive word prediction results; and generate The autoregressive word prediction results and The semi-autoregressive word prediction results perform fast parallel verification of candidate words based on tree decoding. The successful candidate word sequence and the autoregressive prediction result at the end of the sequence are the final output of this reasoning. , where the number of successfully verified candidate words is , ;
[0021] S403, repeatedly executing S402 until the new LLM model outputs an end identifier EOS.
[0022] An optimization device for a large language model is applied to the optimization method for a large language model as described above, wherein the optimization device for a large language model comprises:
[0023] The data generation module is used to generate a training data set by combining the original LLM model and the large model SFT data set;
[0024] A model optimization module is used to embed a side branch network into the main network of the original LLM model to obtain a new LLM model, and input a masked word sequence at the input end of the side branch network;
[0025] A data processing module is used to train the new LLM model using a loss function on the training data set;
[0026] The reasoning verification module is used to execute the generation of candidate word-unit sequences and the verification of the correctness of the candidate word-unit sequences in parallel.
[0027] An electronic device, the improvement of which is that it comprises at least one processor and at least one memory, wherein:
[0028] The memory has computer-readable instructions stored thereon;
[0029] The computer-readable instructions are executed by one or more processors to enable the electronic device to implement the optimization method of the large language model as described above.
[0030] A storage medium having computer-readable instructions stored thereon, wherein the improvement lies in that the computer-readable instructions are executed by one or more processors to implement the optimization method of the large language model as described above.
[0031] The beneficial effects of the present invention are as follows: the present scheme generates a training data set by combining the original LLM model and the large model SFT data set; and embeds a side network in the backbone network of the original LLM model to obtain a new LLM model, and inputs a masked word-meta sequence at the input end of the side network; and uses a loss function to train the new LLM model on the training data set. After the training is completed, the LLM model can predict multiple candidate word-meta sequences in one reasoning; it can enable the LLM model to have semi-autoregressive decoding capabilities, and there is no need to train a semi-autoregressive model from scratch, and through the parallel execution of the generation of candidate word-meta sequences and the verification of the correctness of candidate word-meta sequences, to verify that the quality of the output results of the new LLM model is the same as that of the original LLM model; thereby saving resource consumption and time, enhancing the decoding capability of the LLM model, and ensuring the quality of the output results of the LLM model. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is an overall flow chart of a large language model optimization method of the present invention;
[0033] Figure 2 The overall structure diagram of the new LLM model is shown as an exemplary embodiment;
[0034] Figure 3 A block diagram of an optimization device for a large language model of the present invention;
[0035] Figure 4 A hardware structure diagram of an electronic device is shown in an exemplary embodiment;
[0036] Figure 5 The present invention is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION
[0037] The present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0038] The following will clearly and completely describe the concept, specific structure and technical effects of the present invention in combination with the embodiments and drawings, so as to fully understand the purpose, characteristics and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by technicians in this field without creative work are all within the scope of protection of the present invention. In addition, all the connection / connection relationships involved in the patent do not refer to the direct connection of components, but refer to the formation of a better connection structure by adding or reducing connection accessories according to the specific implementation situation. The various technical features in the invention can be combined interchangeably without conflicting with each other.
[0039] Reference Figure 1 As shown, the present invention discloses a method for optimizing a large language model, and the method for optimizing a large language model specifically comprises the following steps:
[0040] S10, generating a training data set by combining the original LLM model and the large model SFT data set; wherein the training data includes a question word sequence and an answer word sequence, wherein the question word sequence is a sample collected from the large model SFT data set and serves as the input of the original LLM model; the answer word sequence is generated by the original LLM model in combination with the question word sequence through greedy sampling; in this embodiment, in the process of generating the training data set, it is necessary to first sample a question word sequence from the large model SFT data set as the input of the original LLM model; then the original LLM model generates the question word sequence corresponding to the greedy sampling The answer word sequence, then, this question word sequence and its corresponding answer word sequence constitute a training sample; Specifically, the large model SFT dataset is a supervised fine-tuned training dataset, and the large model SFT dataset contains a large amount of labeled data, which is used to guide the performance improvement of the model on a specific task; The greedy sampling is a method of generating a sequence. When generating an answer at each step, the model selects the next word with the highest probability. This method is simple and intuitive, but may not always generate the best overall answer; As a preferred embodiment, first sample the following question from the large model SFT dataset: "What is the capital of France?"; After the question is segmented, it forms a word-gram sequence ["What", "is", "the", "capital", "of", "France", "?"], this process is achieved by random sampling from the large model SFT data set. Specifically, the random sampling of the question word-gram sequence can be achieved through the random function. This sampling method is a common technical means used by people in this field and will not be described here; then, the original LLM model takes the question word-gram sequence as input, and the original LLM model starts to generate the answer. Furthermore, in the process of generating the answer, the original LLM model selects the next word-gram with the highest probability at each step. For example, the model may first generate the word-gram "Paris"; if the answer is multi-word, such as "Paris is the capital of France", the model will continue to generate the next most likely word-gram after generating "Paris" until the answer is completed; in this way, the original LLM model can generate a reasonable answer word-gram sequence based on the input question word-gram sequence using a simple and efficient greedy sampling strategy;
[0041] S20, embedding a side branch network in the backbone network of the original LLM model to obtain a new LLM model, and inputting a masked word sequence at the input end of the side branch network; specifically, the backbone network includes a first language head module, a plurality of Transformer modules and a word unit generator; the output end of the word unit generator is connected to the input end of the first Transformer module; the output end of the previous Transformer module between adjacent Transformer modules is connected to the input end of the next Transformer module, wherein the Transformer module close to the word unit generator is the previous Transformer module, and the Transformer module far from the word unit generator is the next Transformer module; the output end of the last Transformer module is connected to the input end of the first Transformer module; The input end of the first language head module is connected; the side branch network includes several dimensionality reduction modules, several LST modules, a splicing module, a second language head module and a dimensionality increase module; the input end of the splicing module is connected to the output end of the word unit generator through the dimensionality reduction module, and the output end of the splicing module is connected to the input end of the first LST module; the output end of the previous LST module between adjacent LST modules is connected to the input end of the next LST module, wherein the LST module close to the splicing module is the previous LST module, and the LST module far from the splicing module is the next LST module; the output end of the last LST module is connected to the input end of the second language head module through the dimensionality increase module; the number of the LST modules is the same as the number of the Transformer modules, and the input end of the LST module at the same position is connected to the output end of the Transformer module through the dimensionality reduction module;
[0042] In this embodiment, the original LLM model is a large language model without any structural improvement. This solution adds a side branch network to the main network of the original LLM model to construct a new LLM model, so that the new LLM model not only retains the autoregressive word unit output capability of the original LLM model, but also obtains the semi-autoregressive word unit output capability that the original LLM model does not have. Specifically, since the new LLM model retains the autoregressive word unit output capability of the original LLM model, the parameters of the original LLM model in the new LLM model are not changed, and only the LST module is used to enhance the original LLM model; specifically, refer to Figure 2As shown, the backbone network includes a first language head module, two stacked Transformer modules and a word unit generator; wherein, a word unit generator is arranged at the head of the backbone network, and the word unit generator converts the input text into a word unit sequence; a Transformer module is arranged between the word unit generator and the first language head module, which is used to process and extract features of the word unit sequence after the input text is converted, and finally generate the prediction result of the next word unit through the first language head module; a first language head module is arranged at the end of the backbone network, which converts the feature sequence representing the hidden layer state of the word unit into an output word unit, which serves as the output layer of the backbone network; since the original LLM model is an autoregressive model, it outputs an autoregressive output word unit, that is, each inference can only obtain the prediction result of the next word unit; the side branch network includes 3 dimensionality reduction modules, a dimensionality increase module, two stacked LST modules, a splicing module and a second language head module; wherein, the number of LST modules is the same as the number of Transformer modules in the backbone network, and its structure is similar to that of the Transformer module in the backbone network, the only difference is The feature dimension is reduced to 1 / 8; the input of the side network is the masked word sequence and the word feature sequence from the backbone network; the masked word sequence is a number of learnable placeholder words, whose dimensions are the same as the backbone network input word after dimensionality reduction. Through the splicing module, the backbone network input word sequence and the masked word sequence after dimensionality reduction are spliced into an LST input word sequence; before each LST module is entered, the LST input word sequence is added to the output feature sequence of the Transformer module corresponding to the current LST module by position, for example, the backbone network The length of the input word sequence is A, the length of the mask word sequence is B, and the length of the LST input word sequence is A+B. When adding by position, the addition operation is only performed on the first A words of the LST input word sequence, and the last B words remain unchanged. After the LST input word sequence is operated by multiple LST modules, it is obtained by a dimension-raising module to obtain the same dimension as the backbone network input word. The second language head module is used to convert it into an output word sequence, that is, a semi-autoregressive output word, so that the new LLM model retains the autoregressive word output capability of the original LLM model.
[0043] S30, training the new LLM model using a loss function on the training data set. After the training is completed, the LLM model can predict multiple candidate word-unit sequences in one reasoning; specifically, the expression of the loss function is:
[0044] ;
[0045] In the formula, The number of masked word sequences to set; Indicates from the range A randomly selected index from Indicates the length of the answer word sequence; Represents that based on the training input The semi-autoregressive output is the label The posterior probability of represents the logarithm operator;
[0046] In the specific implementation of the present invention, the new LLM model adopts the loss function for training on the training data set, and after the training, the new LLM model has the ability to predict multiple candidate word-unit sequences in one reasoning; specifically, before training the new LLM model, it is assumed that a problem word-unit sequence is sampled from the large model SFT data set, and the original LLM generates a word-unit sequence of length 1 through greedy sampling. The answer word sequence of this question word sequence and the generated answer word sequence Y constitute a training sample; assuming that the number of mask word sequences is set to , then from the range Randomly pick an index from , and then the answer word sequence corresponding to the index is inserted into the question word sequence to form a new input word sequence. The new input word sequence is used as the training input, and the index The answer word sequence after the answer is used as a label. Through the above data generation process, a large number of ready-made SFT data sets can be easily used to create a large amount of training data. This way of constructing the training data set ensures that the distribution of the semi-autoregressive output word units is consistent with the distribution of the autoregressive output word units of the original LLM, which promotes the accuracy of candidate word unit prediction. When training the new LLM model, a loss function is used on the training data set. The loss function takes into account parameters such as the number of mask word unit sequences, the randomly extracted index, the length of the answer word unit sequence, and the posterior probability based on the semi-autoregressive output. After training, the new LLM model can predict multiple candidate word unit sequences in one reasoning, and can make the new LLM model predict candidate word units more accurately, thereby optimizing its performance.
[0047] S40, executing the generation of candidate word-unit sequences and the verification of the correctness of the candidate word-unit sequences in parallel; specifically, the specific process of executing the generation of candidate word-unit sequences and the verification of the correctness of the candidate word-unit sequences in parallel is:
[0048] S401, during the first inference, the backbone network inputs the test word sequence , the side branch network inputs the masked word sequence , and the masked word sequence The number of , the new LLM model generates an autoregressive word prediction result and Semi-autoregressive word prediction results, where the autoregressive word is the prediction result as the final output result of the first inference , and the new LLM model will evaluate the confidence score of each semi-autoregressive word prediction result and extract the Top-K candidate words according to the confidence score to construct a candidate word tree, forming a total number of candidate words to be verified. The candidate word sequence set ;
[0049] S402, at the nth reasoning, , the backbone network inputs the test word sequence ,forward The word sequence that is accepted after the inference verification is successful and the candidate word sequence to be verified in this reasoning , the number of masked tokens in the side branch network input is The masked word sequence , and the new LLM model generates The autoregressive word prediction results and semi-autoregressive word prediction results; and generate The autoregressive word prediction results and The semi-autoregressive word prediction results perform fast parallel verification of candidate words based on tree decoding. The successful candidate word sequence and the autoregressive prediction result at the end of the sequence are the final output of this reasoning. , where the number of successfully verified candidate words is , ;
[0050] S403, repeatedly executing S402 until the new LLM model outputs an end identifier EOS.
[0051] In the prior art, the specific process of the commonly used candidate word parallel verification method is as follows:
[0052] In the first inference process, suppose the first word sequence of the input , the LLM model outputs the input word sequence through a masked word sequence of length M The next word and the following M consecutive tokens ,because yes The autoregressive output word of is a reliable output result, so it is accepted without any doubt. It is used as the candidate word sequence to be verified in the next reasoning; in the i-th (i>1) reasoning, the input word sequence includes the original input word sequence , accepted word units, candidate word unit sequences to be verified and a masked word sequence of length M; for the word sequence to be verified For each word in the sentence, the current candidate word is verified according to the output result of its previous word. If the output result of the previous word is consistent with the content of the current candidate word, the current candidate word is accepted and added to the previously generated received word sequence. If the output result of the previous word is different from the content of the current candidate word, the current candidate word is rejected, and the word and all subsequent words are discarded and no verification is performed. At the same time, the mask word sequence of length M generates the candidate word sequence to be verified in the next reasoning. ; In an inference process, the verification of the current candidate word-unit sequence and the generation of the next candidate word-unit sequence occur simultaneously, and the verification and generation are parallel. However, the above-mentioned candidate word unit parallel verification method faces two thorny problems; on the one hand, the accuracy of the candidate word-unit cannot always be guaranteed; on the other hand, in some inference processes, the number of candidate word-units may be very small; for example, in one inference, 4 word-units are received after verification, and 4 candidate word-units to be verified are also generated. At this time, the newly generated 4 candidate word-units to be verified overlap with the 4 word-units received in this inference, resulting in the newly generated 4 candidate word-units to be verified losing their function. Therefore, the number of candidate word-units for the next inference becomes 0, and the acceleration effect is lost; for this reason, this embodiment provides a candidate word-unit fast parallel verification method based on tree decoding. On the one hand, in order to improve the accuracy of the candidate words, when predicting the candidate word-units, for the output result of each mask word-unit, the top k scores (Top K) prediction result is used as the candidate word set for that position, then the candidate word sets for all positions can be organized into a non-full candidate word tree, where each leaf node represents a possible output sequence for subsequent verification. It should be noted that the candidate word tree here is not a complete tree structure. Only the first candidate word with the highest score has a subtree at each layer, and other words are used as leaf nodes. This non-full sparse structure ensures high efficiency during verification, and some low-scoring word sequence combinations will not be used for subsequent verification. On the other hand, in order to ensure the number of available candidate word units, when verifying each sequence combination, this method will prepare a set of masked word units for it. Once the sequence combination is verified and accepted, they are used to generate the candidate word units to be verified for the sequence combination in the next reasoning. This operation ensures that the number of available candidate word units for each reasoning is always the same and will not change. As described in steps S401-S403, the new LLM model can output at most It can output at least one word per reasoning, which is more efficient than the original LLM which only outputs one word per reasoning. In addition, due to the existence of the verification step, the output of one reasoning is word units and the original LLM model through The word units obtained by the two inferences are exactly the same. Therefore, the reasoning process of the new LLM model can be regarded as a lossless acceleration of the reasoning process of the original LLM model.
[0053] Reference Figure 3 As shown, the present invention further discloses a large language model optimization device 600, which is applied to the large language model optimization method as described above, and the large language model optimization device includes:
[0054] A data generation module 601 is used to generate a training data set by combining the original LLM model and the large model SFT data set;
[0055] A model optimization module 602 is used to embed a side branch network in the main network of the original LLM model to obtain a new LLM model, and input a mask word sequence at the input end of the side branch network;
[0056] A data processing module 603 is used to train the new LLM model using a loss function on the training data set;
[0057] The reasoning verification module 604 is used to execute the generation of the candidate word-gram sequence and the verification of the correctness of the candidate word-gram sequence in parallel.
[0058] It should be noted that the optimization strategy of the large language model provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the optimization device 600 for the large language model will be divided into different functional modules to complete all or part of the functions described above.
[0059] In addition, the large language model optimization device 600 provided in the above embodiment and the embodiment of the large language model optimization method belong to the same concept, and the specific manner in which each module performs the operation has been described in detail in the method embodiment and will not be repeated here.
[0060] Figure 4 The present invention schematically illustrates the structure of an electronic device according to an exemplary embodiment.
[0061] It should be noted that the electronic device is only an example adapted to the present invention and cannot be considered to provide any limitation on the scope of use of the present invention. The electronic device cannot be interpreted as requiring dependence on or having Figure 4 One or more components of exemplary electronic device 2000 are shown.
[0062] The hardware structure of the electronic device 2000 may vary greatly due to differences in configuration or performance, such as Figure 4 As shown, the electronic device 2000 includes: a power supply 210 , an interface 230 , at least one memory 250 , and at least one central processing unit (CPU) 270 .
[0063] Specifically, the power supply 210 is used to provide operating voltage for each hardware device on the electronic device 2000 .
[0064] The interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. Of course, in other examples adapted by the present invention, the interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, etc. Figure 4 As shown, this is not intended to be a specific limitation.
[0065] The memory 250 is a carrier for storing resources, which may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 251, an application 253 and data 255, etc. The storage method may be temporary storage or permanent storage.
[0066] Among them, the operating system 251 is used to manage and control various hardware devices and application programs 253 on the electronic device 2000 to enable the central processor 270 to calculate and process the massive data 255 in the memory 250. It can be WindowsServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
[0067] The application program 253 is a computer-readable instruction that performs at least one specific task based on the operating system 251, and may include at least one module ( Figure 4 (not shown), each module may respectively include computer-readable instructions for the electronic device 2000. For example, the device for assessing task priority may be regarded as an application 253 deployed on the electronic device 2000.
[0068] The data 255 may be signal information, etc., stored in the memory 250 .
[0069] The central processor 270 may include one or more processors and is configured to communicate with the memory 250 through at least one communication bus to read the computer-readable instructions stored in the memory 250, thereby realizing the operation and processing of the mass data 255 in the memory 250. For example, the optimization method of the large language model is completed by the central processor 270 reading a series of computer-readable instructions stored in the memory 250.
[0070] In addition, the present invention can also be implemented by hardware circuits or hardware circuits combined with software. Therefore, the implementation of the present invention is not limited to any specific hardware circuits, software, or a combination of the two.
[0071] See also Figure 5 In an embodiment of the present invention, an electronic device 4000 is provided. The electronic device 400 may include: a desktop computer, a laptop computer, a server, etc. with sensor recognition capability.
[0072] exist Figure 5 In the embodiment, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.
[0073] The data interaction between the processor 4001 and the memory 4003 can be realized through at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0074] Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception, etc. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.
[0075] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0076] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program instructions or codes in the form of instructions or data structures and can be accessed by the electronic device 400, but is not limited to this.
[0077] Computer-readable instructions are stored in the memory 4003 , and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002 .
[0078] The computer-readable instructions are executed by one or more processors 4001 to implement the optimization method of the large language model in the above embodiments.
[0079] In addition, an embodiment of the present invention provides a storage medium on which computer-readable instructions are stored. The computer-readable instructions are executed by one or more processors to implement the optimization method of the large language model as described above.
[0080] Compared with the related art, the present invention has the following beneficial effects:
[0081] 1. The present invention embeds a side network in the backbone network of the original LLM model to obtain a new LLM model, and inputs a masked word sequence at the input end of the side network; and uses a loss function to train the new LLM model on a training data set, so that the new LLM model can retain the autoregressive decoding ability of the original LLM and have a semi-autoregressive decoding ability, and there is no need to train a semi-autoregressive model from scratch, thereby saving resource consumption and time and enhancing the decoding ability of the LLM model.
[0082] 2. The present invention provides an efficient tree-based decoding method for verifying candidate word units generated by semi-autoregression, and the generation and verification of candidate word units are carried out simultaneously to verify that the quality of the output results of the new LLM model is the same as that of the original LLM model, which not only accelerates the reasoning process, but also ensures the quality of the output results of the LLM model.
[0083] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.
Claims
1. A method for optimizing a large language model, characterized in that: The optimization method of the large language model specifically includes the following steps: S10, combining the original LLM model and the large model SFT dataset to generate a training dataset, wherein the large model SFT dataset is a supervised fine-tuned training dataset; S20, embedding a side branch network into the backbone network of the original LLM model to obtain a new LLM model, and inputting a masked word sequence into the input end of the side branch network; S30, using the loss function to train the new LLM model on the training data set. After the training is completed, the LLM model can predict multiple candidate word-meta sequences in one reasoning; S40, executing the generation of the candidate word-gram sequence and the verification of the correctness of the candidate word-gram sequence in parallel.
2. The optimization method of a large language model according to claim 1, characterized in that: In step S10, the training data includes a question word sequence and an answer word sequence, wherein the question word sequence is a sample collected from the large model SFT data set and serves as the input of the original LLM model; the answer word sequence is generated by the original LLM model in combination with the question word sequence through greedy sampling.
3. The optimization method of a large language model according to claim 2, characterized in that: In step S20, the backbone network includes a first language head module, a plurality of Transformer modules and a word unit generator; the output end of the word unit generator is connected to the input end of the first Transformer module; the output end of the previous Transformer module between adjacent Transformer modules is connected to the input end of the next Transformer module, wherein the Transformer module close to the word unit generator is the previous Transformer module, and the Transformer module far from the word unit generator is the next Transformer module; the output end of the last Transformer module is connected to the input end of the first language head module.
4. The method for optimizing a large language model according to claim 3, characterized in that: The side branch network includes several dimensionality reduction modules, several LST modules, a splicing module, a second language head module and a dimensionality increase module; the input end of the splicing module is connected to the output end of the word unit generator through the dimensionality reduction module, and the output end of the splicing module is connected to the input end of the first LST module; the output end of the previous LST module between adjacent LST modules is connected to the input end of the next LST module, wherein the LST module close to the splicing module is the previous LST module, and the LST module far from the splicing module is the next LST module; the output end of the last LST module is connected to the input end of the second language head module through the dimensionality increase module; the number of the LST modules is the same as the number of Transformer modules, and the input end of the LST module at the same position is connected to the output end of the Transformer module through the dimensionality reduction module.
5. The method for optimizing a large language model according to claim 4, characterized in that: In step S30, the loss function is expressed as: ; In the formula, The number of masked word sequences to set; Indicates from the range A randomly selected index from Indicates the length of the answer word sequence; Represents that based on the training input The semi-autoregressive output is the label The posterior probability of Represents the logarithm operator.
6. The method for optimizing a large language model according to claim 5, characterized in that: In step S40, the specific process of executing the generation of the candidate word-unit sequence and the verification of the correctness of the candidate word-unit sequence in parallel is as follows: S401, during the first inference, the backbone network inputs the test word sequence , the side branch network inputs the masked word sequence , and the masked word sequence The number of , the new LLM model generates an autoregressive word prediction result and Semi-autoregressive word prediction results, where the autoregressive word is the prediction result as the final output result of the first inference , and the new LLM model will evaluate the confidence score of each semi-autoregressive word prediction result and extract the Top-K candidate words according to the confidence score to construct a candidate word tree, forming a total number of candidate words to be verified. The candidate word sequence set ; S402, at the nth reasoning, , the backbone network inputs the test word sequence ,forward The word sequence that is accepted after the inference verification is successful and the candidate word sequence to be verified in this reasoning , the number of masked tokens in the side branch network input is The masked word sequence , and the new LLM model generates The autoregressive word prediction results and semi-autoregressive word prediction results; and generate The autoregressive word prediction results and The semi-autoregressive word prediction results perform fast parallel verification of candidate words based on tree decoding. The successful candidate word sequence and the autoregressive prediction result at the end of the sequence are the final output of this reasoning. , where the number of successfully verified candidate words is , ; S403, repeatedly executing S402 until the new LLM model outputs an end identifier EOS.
7. A large language model optimization device, applied to the large language model optimization method according to any one of claims 1 to 6, characterized in that: The optimization device of the large language model includes: The data generation module is used to generate a training data set by combining the original LLM model and the large model SFT data set; A model optimization module is used to embed a side branch network into the main network of the original LLM model to obtain a new LLM model, and input a masked word sequence into the input end of the side branch network; A data processing module, used to train the new LLM model using a loss function on the training data set; The reasoning verification module is used to perform the generation of candidate word-unit sequences and the verification of the correctness of the candidate word-unit sequences in parallel.
8. An electronic device, characterized in that: comprises at least one processor and at least one memory, wherein: The memory has computer-readable instructions stored thereon; The computer-readable instructions are executed by one or more processors, so that the electronic device implements the large language model optimization method as described in any one of claims 1-6.
9. A storage medium having computer-readable instructions stored thereon, characterized in that: The computer-readable instructions are executed by one or more processors to implement the large language model optimization method according to any one of claims 1 to 6.
