A large language model reasoning method and system based on behavior simulation

Abstract

The present application relates to a kind of big language model reasoning method and system based on behavior simulation, inference is carried out using big language model comprising multiple computing units, each computing unit is correspondingly provided with lightweight feature simulation network and prospective routing network, when inference, each token is dynamically routed to corresponding computing unit or corresponding lightweight feature simulation network by prospective routing network and inference is carried out, the training process of big language model includes: under the premise of freezing big language model parameter, with learning the input-output mapping of corresponding computing unit as the goal, each lightweight feature simulation network is trained respectively;Under the premise of freezing big language model parameter and lightweight feature simulation network parameter, with sparsity constraint and language modeling loss as optimization goal, all prospective routing networks are trained;Low-resource adapter is constructed and trained, and big language model is fine-tuned.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method and system for reasoning based on large language models using behavioral simulation. Background Technology

[0002] In recent years, large language models have made significant progress in natural language processing tasks, but their huge computational overhead severely limits their practical deployment efficiency.

[0003] To alleviate this problem, dynamic computation allocation methods have gradually become a research hotspot. These methods achieve adaptive allocation of computational resources by setting up a lightweight routing network at each computational unit, such as a self-attention or feedforward network module, and dynamically deciding whether to skip a unit based on the importance of the smallest semantic unit (token) of the input. However, existing methods generally employ a "greedy routing" mechanism, which has two significant drawbacks:

[0004] (1) Irreversible information loss. Once the routing network decides to skip a unit, the transformation of the current token by that unit will be completely discarded, resulting in the destruction of the semantic information of the token. Even if fine-tuning is used to repair it later, it is difficult to fully restore the original feature distribution, resulting in a decrease in the expressive power of the model.

[0005] (2) Short-sighted token selection mechanism. The training of the routing network is based solely on the immediate performance loss after skipping units, without considering the recoverability of tokens, i.e., whether their transformations are easily recovered by subsequent fine-tuning. This lack of forward-looking decision-making means that the model cannot accurately identify those simple and easily predictable tokens, resulting in unintelligent allocation of computing resources.

[0006] At its root, the above problems stem from the fact that existing routing mechanisms fail to assess the complexity and predictability of the transformations performed by the target computational unit when making skip decisions, thus failing to achieve an effective balance between preserving semantic integrity and improving reasoning efficiency.

[0007] Against this backdrop, there is currently a lack of a large language model reasoning method and system to solve or partially solve the aforementioned problems. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a large language model reasoning method and system based on behavior simulation, so as to solve or partially solve the problems of information loss and high computational overhead caused by directly skipping the calculation unit in the existing large language model reasoning.

[0009] The objective of this invention can be achieved through the following technical solutions:

[0010] One aspect of the present invention provides a large language model inference method based on behavior simulation. The method utilizes a large language model comprising multiple computational units for inference. Each computational unit is configured with a lightweight feature simulation network and a prospective routing network. During inference, the prospective routing network dynamically routes each token to the corresponding computational unit or the corresponding lightweight feature simulation network for inference. The training process of the large language model includes:

[0011] With the parameters of the large language model frozen, and with the goal of learning the input-output mapping of the corresponding computational unit, each lightweight feature simulation network is trained to achieve behavior simulation.

[0012] Under the premise of freezing the parameters of the large language model and the parameters of the lightweight feature simulation network, the entire prospective routing network is trained with sparsity constraints and language modeling loss as optimization objectives.

[0013] By freezing the parameters of the lightweight simulation network and the prospective routing network, a low-resource adapter is constructed and trained, and the parameters of the large language model are fine-tuned by introducing additional trainable parameters.

[0014] As a preferred technical solution, during the training of the lightweight feature simulation network, the loss function... for:

[0015]

[0016] in, , These are the outputs of the original large model and the simulated network, respectively.

[0017] As a preferred technical solution, sparsity constraint for:

[0018]

[0019] in, Given the preset desired sparsity, The proportion of tokens routed from the forward-looking routing network to the lightweight feature simulation network out of all tokens.

[0020] As a preferred technical solution, language modeling loss for:

[0021]

[0022] in, The length of the token sequence. Therefore The predicted probability of a large language model with parameters. Indicates the first A token.

[0023] As a preferred technical solution, during the training process of a prospective routing network, the loss function... for:

[0024]

[0025] in, , These are language modeling loss and sparsity constraint, respectively. Preset weights.

[0026] As a preferred technical solution, the lightweight feature simulation network is a feedforward network.

[0027] As a preferred technical solution, the forward-looking routing network is a binary classification network.

[0028] As a preferred technical solution, local feature reconstruction loss is used for training in the lightweight feature simulation network.

[0029] As a preferred technical solution, the computing unit includes a feedforward network and a self-attention mechanism.

[0030] Another aspect of the present invention provides a large language model reasoning system based on behavior simulation, for implementing the aforementioned large language model reasoning method based on behavior simulation, the system comprising:

[0031] The large language model module includes multiple computing units, each of which is equipped with a lightweight feature simulation network and a prospective routing network.

[0032] In the inference module, during the inference process, the forward routing network dynamically routes each token to the corresponding computing unit or the corresponding lightweight feature simulation network for inference.

[0033] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0034] (1) Mitigating information loss caused by directly skipping computation units: In the computation units of the large language model, the present invention sets up a lightweight feature simulation network and a forward-looking routing network. During the inference process, the forward-looking routing network dynamically routes each token to the corresponding computation unit or the corresponding lightweight feature simulation network for inference. By introducing the lightweight feature simulation network, an intermediate path between full computation and lightweight approximation is provided for each computation unit. This design changes the routing decision from the original 0-1 skip to a trade-off between accuracy and efficiency, which significantly reduces the feature distribution destruction caused by direct skipping and provides a better starting point for subsequent fine-tuning.

[0035] (2) The routing network is forward-looking: Under the premise of freezing the parameters of the large language model and the parameters of the lightweight feature simulation network, this invention trains the entire forward-looking routing network with sparsity constraints and language modeling loss as optimization objectives. This enables the routing network to implicitly learn to use the recoverability of the token as the basis for routing, rather than just its immediate importance. This forward-looking decision-making mechanism can more intelligently select the key tokens that truly need to be fully computed, thereby achieving better allocation of computing resources.

[0036] (3) Improve the training efficiency of lightweight feature simulation network: Under the premise of freezing the parameters of large language model, this invention aims to learn the input-output mapping of corresponding computing units and trains each lightweight feature simulation network separately to realize behavior simulation. Local feature reconstruction loss is used in training instead of global language modeling loss, so that training can be carried out in parallel and the overall training efficiency of the system is guaranteed.

[0037] (4) Achieving collaborative optimization of lightweight feature simulation network, forward routing network and main model parameters: This invention adopts a three-stage decoupled training strategy of lightweight feature simulation network training - lightweight feature simulation network training - global fine-tuning, which ensures collaborative optimization of simulation network, routing network and main model parameters. It exhibits excellent performance preservation ability at multiple sparsity levels, and reaches or exceeds the existing best methods in language modeling and complex reasoning tasks, providing an effective solution for the efficient deployment of large language models. Attached Figure Description

[0038] Figure 1 This is a flowchart of the large language model reasoning method based on behavior simulation in the embodiment;

[0039] Figure 2 This is a schematic diagram of the reasoning process of the large language model in the embodiment;

[0040] Figure 3 This is a schematic diagram illustrating the lightweight feature simulation network training process in this embodiment;

[0041] Figure 4 This is a schematic diagram illustrating the large-scale reasoning results of existing solutions and the solution of this invention;

[0042] Figure 5 This is a schematic diagram illustrating the perplexity of existing solutions, the original model, and the reasoning results of this invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0044] Example 1

[0045] To address the problems existing in the prior art, this embodiment provides a large language model reasoning method based on behavior simulation. Based on lightweight feature prediction and a forward-looking routing decision mechanism, it aims to significantly reduce computational overhead while effectively mitigating the information loss problem caused by directly skipping computation units.

[0046] In this method, the large language model comprises multiple computational units, each equipped with a lightweight feature simulation network and a prospective routing network. The lightweight feature simulation network is a parameter-simplified feedforward network used to pre-simulate the transformation of the input token by the corresponding computational unit during the learning phase. The prospective routing network is a lightweight binary classification network that dynamically selects whether each token should be fully computed by the original computational unit or approximated by its corresponding simulation network, based on the approximation capability provided by the feature simulation network.

[0047] Preferably, the computing unit may include a self-attention mechanism and a feedforward network module.

[0048] See Figure 1 This method includes the following steps:

[0049] Step S1: With the parameters of the large language model frozen, each lightweight feature simulation network is trained to achieve behavior simulation with the goal of learning the input-output mapping of the corresponding computing unit.

[0050] This step aims to initialize the lightweight feature simulation network, i.e., behavioral simulation. All parameters of the original large model are frozen. A small number of samples are randomly selected from the training corpus; the exact number can be adjusted based on the complexity of the downstream task. Forward propagation is used to collect the input-output feature pairs of each computational unit and cache them to local storage. Then, based on the correspondence between the segmented local computational units and the feature simulation network, the corresponding input-output pairs are indexed for the lightweight feature simulation network. A reconstruction loss, such as a smoothed L1 loss, is then used to train all lightweight feature simulation networks in parallel. This step can also be performed online during the forward pass of the large model, but it is more time-consuming than the caching approach.

[0051] Preferably, the smoothed L1 loss can also be replaced by cosine similarity loss or L2 loss.

[0052] Specifically, this step freezes the parameters of the original large language model and trains all lightweight feature simulation networks in parallel using a small amount of data, enabling them to learn to fit the input-output mapping of each computational unit. This stage employs smoothing... loss:

[0053]

[0054] The original Big Prophecy model output is: The simulated network initialization output is .

[0055] This training phase employs local feature reconstruction loss instead of global language modeling loss, enabling efficient parallel training and ensuring the overall training efficiency of the system.

[0056] For details, see Figure 3 This diagram illustrates the training of the lightweight feature simulation network. Using the nth layer token feature as a baseline, and with the parameters of the original computation unit fixed, the (n+1)th layer token feature is calculated using the original computation unit. Additionally, based on the cached nth layer token feature, the corresponding lightweight feature simulation network is input to calculate the predicted (n+1)th layer token feature. The lightweight feature simulation network is then trained through backpropagation by calculating the reconstruction loss.

[0057] Step S2: With the parameters of the large language model and the lightweight feature simulation network frozen, train the entire prospective routing network with sparsity constraints and language modeling loss as optimization objectives.

[0058] In this step, with the parameters of the original large language model and the feature simulation network frozen, all prospective routing networks are jointly trained. The optimization objective balances language modeling loss and sparsity constraints, enabling routers to learn path allocation based on token recoverability. The language modeling loss uses the next token prediction loss, for a length of... token sequence The predicted probability of a model with parameter θ is defined as follows: Then the language modeling loss is:

[0059]

[0060] For sparsity constraints, given a preset expected sparsity Define the overall sparsity of the model For the forward-looking routing network, select the proportion of tokens in the characteristic simulation network to all tokens, and then adopt... Sparsity constraints in loss calculation The final training loss is the weighted sum of the two. .

[0061] Step S3: With the lightweight feature simulation network parameters and the prospective routing network parameters frozen, a low-resource adapter is constructed and trained, and the parameters of the large language model are fine-tuned by introducing additional trainable parameters.

[0062] A low-resource adapter (LoRA) is introduced and trained to perform lightweight fine-tuning on the large language model, further restoring performance and improving model quality. The loss function used for training is consistent with that used in the routing prospective network training phase.

[0063] Specifically, in the parameter fine-tuning stage, this step freezes the parameters of the large model, as well as the parameters of the feature simulation network and routing network trained in the previous stage. Additional trainable parameters are added using low-rank adaptation, and after training is completed, they are combined with the original parameters through reparameterization.

[0064] When fine-tuning large model parameters, these parameters are frozen during the training phase. LoRA introduces additional trainable parameters. During the inference phase, these trainable parameters can be fused into the large model parameters using reparameterization techniques, achieving the effect of fine-tuning.

[0065] In step S4, the forward routing network dynamically routes each token to the corresponding computing unit or the corresponding lightweight feature simulation network for inference.

[0066] See Figure 2 For a given large language model to be accelerated, the model is first vertically partitioned, for example, based on large model layers or finer-grained self-attention and feedforward neural network units, resulting in multiple computational units. Then, a lightweight feature simulation network and a forward-looking routing network are introduced into the partitioned computational units. For the input token sequence, it first enters the forward-looking routing network, which dynamically determines its computational path based on the features of each token: whether to enter the original computational unit for exact computation or to enter the lightweight feature simulation network for approximate processing. The different tokens assigned to the two paths are then recombined as the input for the next layer after computation.

[0067] The large language model that applies this method can be used for natural language processing. For example, the input text is segmented into multiple tokens, which are then fed into the large language model for reasoning. Finally, a response to the input text is output, thus realizing question answering.

[0068] To verify the effectiveness of this method, the original model, existing static compression methods (SliceGPT, Shortened-LLM, ShortGPT), existing dynamic compression methods (MoD, D-LLM, SkipGPT-Router, SkipGPT-Lora), and the method of this invention (this invention-Router, this invention-Lora) were tested under an inference task. The suffix "Router" indicates that the model was obtained after training with a prospective routing network (i.e., the result after training the routing network, corresponding to S2), and the suffix "Lora" indicates that the model was trained with Lora (corresponding to S3). See Tables 1 and 2 for the inference task accuracy and perplexity at sparsity of 25% and 40%, respectively. The comparison results between SkipGPT-Router and this invention-Router show that this invention alleviates information loss caused by directly skipping computational units. The comparison results between SkipGPT-Lora and this invention-Lora show that the routing network of this invention possesses prospective capabilities.

[0069] Table 1. Test results when sparsity = 25%

[0070]

[0071] Table 2 Test results when sparsity = 40%

[0072]

[0073] See Figure 4 This is a distribution diagram of the feature cosine similarity between existing solutions and the present invention (where cos represents cosine similarity). Figure 5 This is a schematic diagram illustrating the perplexity of existing solutions, the original model, and the reasoning results of this invention. Based on Table 1-2... Figure 4 and Figure 5This method achieves more accurate inference and lower perplexity compared to existing technologies. Further intermediate layer feature similarity analysis demonstrates that feature simulation effectively increases the proportion of high-similarity features (cosine similarity > 0.95). By introducing a lightweight feature simulation network, a buffer path between full computation and direct skipping is provided for each computational unit, while the offline parallel training of the feature simulation network ensures its training efficiency. Compared to existing dynamic computation schemes, this method can omit the efficient parameter fine-tuning stage under lower sparsity requirements, saving over 50% of training time. Simultaneously, this simulation-then-routing paradigm allows the router to implicitly learn and evaluate the recoverability of each token during decision-making, i.e., the ease with which its transformations are accurately predicted by the lightweight feature simulation network. Therefore, the selection criterion for the forward-looking routing network is upgraded from short-sighted immediate importance to a more forward-looking approximate difficulty. This invention exhibits better performance after efficient parameter fine-tuning.

[0074] Example 2

[0075] Based on Example 1, this example provides a large language model reasoning system based on behavior simulation, used to implement the large language model reasoning method based on behavior simulation in Example 1. The system includes:

[0076] (1) Large language model module, which includes multiple computing units, each of which is equipped with a lightweight feature simulation network and a forward routing network.

[0077] (2) Inference module: During the inference process, the forward routing network dynamically routes each token to the corresponding computing unit or the corresponding lightweight feature simulation network for inference.

[0078] This invention presents a collaborative architecture and three-stage training method that combines simulation followed by routing. The scheme first initializes all lightweight feature simulation networks independently and in parallel. Then, it freezes the parameters of the large language model and the lightweight feature simulation networks to train a prospective routing network. Finally, it performs efficient LoRA fine-tuning of the parameters. This decoupled training paradigm solves the problem of training instability in existing greedy routing methods and significantly improves overall training efficiency.

[0079] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A large language model reasoning method based on behavioral simulation, characterized in that, Inference is performed using a large language model comprising multiple computational units. Each computational unit corresponds to a lightweight feature simulation network and a prospective routing network. During inference, the prospective routing network is a lightweight binary classification network. Based on the approximation capability provided by the feature simulation network, it dynamically selects whether each token should be fully computed by the original computational unit or approximated by its corresponding simulation network. The training process of the large language model includes: With the parameters of the large language model frozen, and with the goal of learning the input-output mapping of the corresponding computational unit, each lightweight feature simulation network is trained to achieve behavior simulation. Under the premise of freezing the parameters of the large language model and the parameters of the lightweight feature simulation network, the entire prospective routing network is trained with sparsity constraints and language modeling loss as optimization objectives. By freezing the parameters of the lightweight feature simulation network and the prospective routing network, a low-resource adapter is constructed and trained, and the parameters of the large language model are fine-tuned by introducing additional trainable parameters.

2. The reasoning method for a large language model based on behavioral simulation according to claim 1, characterized in that, During the training of a lightweight feature simulation network, the loss function for: in, , These are the outputs of the original large model and the simulated network, respectively.

3. The reasoning method for a large language model based on behavioral simulation according to claim 1, characterized in that, sparsity constraints for: in, Given the preset desired sparsity, The proportion of tokens routed from the forward-looking routing network to the lightweight feature simulation network out of all tokens.

4. The large language model reasoning method based on behavioral simulation according to claim 1, characterized in that, Language modeling loss for: in, The length of the token sequence. Therefore The predicted probability of a large language model with parameters. Indicates the first A token.

5. The large language model reasoning method based on behavioral simulation according to claim 1, characterized in that, During the training of a prospective routing network, the loss function for: in, , These are language modeling loss and sparsity constraint, respectively. Preset weights.

6. The reasoning method for a large language model based on behavioral simulation according to claim 1, characterized in that, The lightweight feature simulation network is a feedforward network.

7. The reasoning method for a large language model based on behavioral simulation according to claim 1, characterized in that, The prospective routing network is a binary classification network.

8. The large language model reasoning method based on behavior simulation according to claim 1, characterized in that, In training the lightweight feature simulation network, local feature reconstruction loss is used.

9. The large language model reasoning method based on behavioral simulation according to claim 1, characterized in that, The computational unit includes a feedforward network and a self-attention mechanism.

10. A large language model reasoning system based on behavioral simulation, characterized in that, For implementing the behavior simulation-based large language model reasoning method as described in any one of claims 1-9, the system comprises: The large language model module includes multiple computing units, each of which is equipped with a lightweight feature simulation network and a prospective routing network. In the inference module, during the inference process, the forward routing network dynamically routes each token to the corresponding computing unit or the corresponding lightweight feature simulation network for inference.

Images