Large model training methods, inference methods, devices, equipment and storage media

Abstract

This application provides a large-scale model training method, inference method, apparatus, device, and storage medium, relating to the field of artificial intelligence technology. The large-scale model includes an inference module and a depth predictor; the inference module includes an encoder, a multi-layer Transformer module, and a decoder. The method includes: inputting training samples into the encoder and depth predictor respectively; after processing the training samples by the encoder, multi-layer Transformer module, and decoder, obtaining the prediction result of the token to be inferred and the model depth output by the decoder corresponding to each layer of the Transformer module; predicting the model depth based on the training samples obtained by the depth predictor; and optimizing the large-scale model based on the token to be inferred, the prediction result, the model depth, and the predicted model depth to obtain the trained large-scale model. This application, by training a large-scale model containing a depth predictor, can balance inference performance and computational resources.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a large model training method, inference method, device, equipment, and storage medium. Background Technology

[0002] Currently, large-scale reasoning models include ordinary reasoning models and deep thinking reasoning models. Ordinary reasoning models are fast, unconscious, intuitive, and do not require much cognitive resources, while deep thinking reasoning models are slow, rational analytical systems that consume more computational resources.

[0003] When using large-scale model inference, users often cannot determine which inference mode to use, and thus mostly choose the deep thinking inference mode. As a result, every token inference process uses the deep thinking inference mode. In fact, in some inference tasks, it is not necessary to use the deep thinking inference mode for all tokens, resulting in a waste of resources. Summary of the Invention

[0004] The purpose of this application is to provide a large model training method, inference method, apparatus, device, and storage medium to reasonably determine the model depth required for inference during the large model inference process, thereby reducing the waste of computing resources while achieving the desired inference effect.

[0005] In a first aspect, embodiments of this application provide a method for training a large model, the large model including an inference module and a deep predictor; wherein the inference module includes an encoder, a multi-layer Transformer module, and a decoder corresponding to each layer of the Transformer module, the method including:

[0006] The training samples are input into the encoder and the depth predictor, respectively; the training samples include inferred tokens and tokens to be inferred.

[0007] After the training samples are processed by the encoder, multi-layer Transformer modules and decoder, the prediction result of the token to be inferred and the model depth are obtained from the decoder output of each Transformer module.

[0008] The depth of the predicted model is obtained by the depth predictor based on training samples;

[0009] The internal parameters of the large model are optimized based on the token to be inferred, the prediction result, the model depth, and the prediction model depth to obtain the trained large model.

[0010] In this embodiment, the large model includes a depth predictor. By training the depth predictor, the depth predictor can dynamically adjust its depth according to the complexity of different tasks or data, improving inference performance while reducing the waste of computing resources.

[0011] In any embodiment, the internal parameters of the large model are optimized based on the prediction results, model depth, and prediction model depth, including:

[0012] Determine the inference loss based on the token to be inferred and the prediction result;

[0013] Determine the model depth loss based on the model depth and the prediction model depth;

[0014] The internal parameters of large models are optimized based on inference loss and model depth loss.

[0015] This application combines inference loss and model depth loss to optimize the internal parameters of a large model, and evaluates the model from two aspects: the accuracy of the prediction results and the rationality of the model depth, making the optimization process more comprehensive and accurate.

[0016] In any embodiment, the internal parameters of the large model are optimized based on the inference loss and the model depth loss, including:

[0017] Optimize the internal parameters of the inference module based on the inference loss;

[0018] The total loss is determined based on inference loss and model depth loss;

[0019] The internal parameters of the depth predictor are optimized based on the total loss.

[0020] In this embodiment, the internal parameters of the inference module are specifically optimized using inference loss. This targeted optimization method can more accurately adjust the inference module, significantly improving its accuracy and reliability in predicting the token to be inferred. When optimizing the depth predictor, not only inference loss is considered, but also model depth loss is combined to determine the total loss. Then, the internal parameters of the depth predictor are optimized based on the total loss. This comprehensive optimization method considers both the accuracy of the depth predictor in predicting model depth and its synergistic effect with the inference module, enabling the depth predictor to better provide reasonable depth support for the entire model's inference process, thereby improving the model's adaptability and efficiency.

[0021] In any embodiment, the total loss is determined based on the inference loss and the model depth loss, including:

[0022] The sum of the inference loss and model depth loss corresponding to the decoder of each Transformer module is used as the intermediate loss of the corresponding Transformer module.

[0023] The minimum intermediate loss is taken as the total loss.

[0024] In this embodiment, the intermediate loss of each layer is obtained by adding the inference loss of the decoder corresponding to each Transformer module to the model depth loss. In multi-layer Transformer modules, the intermediate losses of different layers may differ. Selecting the minimum intermediate loss as the total loss is actually finding the minimum number of layers in the entire model that can accurately infer tokens, so that the optimized model can simultaneously take into account the accuracy of inference and the reasonableness of depth prediction.

[0025] In any embodiment, during the optimization of the internal parameters of the inference module, the internal parameters of the depth predictor remain unchanged, thus obtaining the intermediate inference module;

[0026] During the optimization of the internal parameters of the depth predictor, the internal parameters of the Transformer module, encoder, and decoder remain unchanged, resulting in an intermediate depth predictor.

[0027] The internal parameters of the intermediate inference module and intermediate depth predictor are further optimized using inference loss and model depth loss.

[0028] In this embodiment, when optimizing the inference module, the parameters of the depth predictor are kept constant, allowing for targeted adjustment of the inference module's internal parameters to achieve optimal performance in predicting the tokens to be inferred. Similarly, when optimizing the depth predictor, fixing the parameters of the Transformer module, encoder, and decoder allows for a focus on improving the depth predictor's accuracy in predicting model depth. This phased optimization approach avoids interference between parameters, ensuring that each module is fully adjusted and improved in its respective optimization phase, and enhancing the efficiency of the entire training process.

[0029] Secondly, embodiments of this application provide a large-model inference method, including:

[0030] Input the reasoned tokens into the large model; the large model is obtained by training using the large model training method in the first aspect.

[0031] The deep predictor in the large model processes the inferred tokens to generate the model depth required for the large model to infer the next token.

[0032] Based on the model depth, the inference module at the corresponding depth outputs the inference result of the next token based on the already inferred tokens.

[0033] This application embodiment uses a depth predictor to predict the model depth required for inferring the next token using a large model, and infers the next token based on the inference model corresponding to the predicted model depth. This large model can automatically adjust the model depth used in the inference process according to the complexity of the task and the characteristics of the data, avoiding the problem of overcomputation of a fixed-depth model in some cases, and improving inference efficiency.

[0034] In any embodiment, the number of reasoned tokens is a preset number of tokens selected from the sequence of reasoned tokens.

[0035] In this embodiment, a preset number of tokens is used to predict the model depth, which ensures the accuracy of the model depth prediction. In addition, when the number of inference tokens is large, unnecessary computation can be reduced.

[0036] Thirdly, embodiments of this application provide a large model training apparatus, the large model including an inference module and a deep predictor; wherein the inference module includes an encoder, a multi-layer Transformer module, and a decoder corresponding to each layer of the Transformer module, and the apparatus includes:

[0037] The sample input module is used to input training samples into the encoder and the depth predictor, respectively; the training samples include inferred tokens and tokens to be inferred.

[0038] The token prediction module is used to obtain the prediction result of the token to be inferred and the model depth of the decoder output of each Transformer module after the training samples have been processed by the encoder, multi-layer Transformer module and decoder.

[0039] The first depth prediction module is used to predict the depth of the model based on training samples obtained by the depth predictor.

[0040] The parameter optimization module is used to optimize the internal parameters of a large model based on the token to be inferred, the prediction result, the model depth, and the prediction model depth, so as to obtain the trained large model.

[0041] Fourthly, embodiments of this application provide a large-scale model inference apparatus, comprising:

[0042] The input module is used to input the reasoned tokens into the large model; the large model is trained using the large model training method in the first aspect.

[0043] The second depth prediction module is used to process the inferred tokens through the depth predictor in the large model and generate the model depth required for the large model to infer the next token.

[0044] The inference module is used to obtain the inference result of the next token output by the inference module at the corresponding depth based on the already inferred tokens, based on the model depth.

[0045] Fifthly, embodiments of this application provide an electronic device, including: a processor, a memory, and a bus, wherein:

[0046] The processor and memory communicate with each other via a bus;

[0047] The memory stores program instructions that can be executed by the processor, and the processor can execute the first or second method by calling the program instructions.

[0048] Sixthly, embodiments of this application provide a non-transitory computer-readable storage medium, comprising:

[0049] A non-transitory computer-readable storage medium stores computer instructions that cause the computer to perform the method of the first aspect or the second aspect.

[0050] In a seventh aspect, embodiments of this application provide a computer program product, including computer program instructions, which, when read and executed by a processor, perform the method of the first aspect or the second aspect.

[0051] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A schematic diagram of a large model architecture provided for an embodiment of this application;

[0054] Figure 2 This is a schematic diagram of a large model training method provided in an embodiment of this application;

[0055] Figure 3 A flowchart of a large model inference method provided in this application embodiment;

[0056] Figure 4 The model inference principle diagram provided for the embodiments of this application;

[0057] Figure 5 This is a schematic diagram of a large model training device provided in an embodiment of this application;

[0058] Figure 6 This is a schematic diagram of a large-scale model inference device provided in an embodiment of this application;

[0059] Figure 7 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0060] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0062] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0063] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0064] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0065] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0066] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0067] With the widespread application of large-scale models in online services, such as search engines, chatbots, and virtual assistants, their deep thinking capabilities enable strategic planning and the provision of higher-value services. Large-scale model technology with deep thinking capabilities can be achieved through the following paths: First, by using OpenAI to expand computing resources on the inference side of the large model and enhancing its deep logic capabilities through sampling and search methods; second, by employing methods similar to Quite Star or DeepSeek, embedding deep logic thinking capabilities into the large model itself through reinforcement learning training; and third, by using training- and post-training compatible models, further enhancing the large model's logical thinking capabilities through post-training sampling and search methods after training to establish a certain level of deep logic thinking.

[0068] Regardless of the method used to enhance the deep logical thinking ability of large models, deep thinking models all face greater demands on computing power and storage, as well as longer contextual information. In the fundamental theories of large models and cognitive science in psychology, large models with deep thinking capabilities are often referred to as "System 2." System 2 is a slow, rational analytical system used to solve complex problems and engage in deep thinking. Large models without deep thinking capabilities are referred to as "System 1." System 1 is a fast, unconscious, intuitive model that doesn't require many cognitive resources and is used to handle simple, familiar, and repetitive tasks. Currently, large model companies are researching "System 1" and "System 2" separately, providing two different large model services. The large model service interfaces of OpenAI or Deepseek typically require users to choose whether to use "deep thinking." However, most users are unsure whether their problems require deep thinking. If users consistently choose "System 2" when using large models, the large models will consume a significant amount of computing power when processing simple tasks. If users only use "System 1," the models cannot provide accurate answers to complex tasks.

[0069] Therefore, it is of great significance to be able to adaptively combine the fast thinking of "System 1" and the slow thinking of "System 2" while providing a large model that integrates "fast" and "slow" thinking.

[0070] To address the aforementioned technical issues, this application proposes an adaptive large-scale model training and inference method that combines "slow" and "fast" thinking. Typically, "slow" thinking requires the large model to perform more iterative calculations before outputting results, thus forming more organized and logical content; while "fast" thinking is more like querying shallow knowledge in the large model's knowledge base. This framework adaptively adjusts the depth of the large model based on the difficulty of the client's query. More difficult problems use deeper neural network models to achieve "slow" thinking; easier problems use shallower neural network models to achieve "fast" thinking. This effectively integrates the "System 1" and "System 2" of the large model, reducing ineffective "slow" thinking and achieving efficient model usage.

[0071] It is understood that the large model training method and large model inference method provided in this application embodiment can be applied to electronic devices, including terminals and servers; wherein the terminal can specifically be a smartphone, tablet computer, computer, personal digital assistant (PDA), etc.; the server can specifically be an application server or a web server. In addition, the model training method and the prediction method can be executed on the same terminal device or on different terminal devices.

[0072] To facilitate understanding, the technical solutions provided in the embodiments of this application will be described below using a terminal device as the execution subject, illustrating the application scenarios of the model training method and inference method provided in the embodiments of this application.

[0073] Figure 1 A schematic diagram of a large model architecture provided for an embodiment of this application, such as Figure 1 As shown, the large model includes an inference module and a deep predictor. The inference module comprises an encoder, multiple Transformer modules, and a decoder. The number of layers in the Transformer module can be set according to actual needs. Each Transformer module corresponds to one decoder. The encoder encodes the input inferred tokens and extracts their feature representations. The multiple Transformer modules further process and transform the encoded features to capture the complex dependencies between tokens. During the model training phase, the decoder corresponding to each Transformer module generates the prediction result of the token to be inferred and the model depth based on the output of that layer. During the inference phase, it generates the prediction result of the token to be inferred based on the output of that layer. The deep predictor predicts the model depth required for the large model to infer the next token based on the inferred tokens. The large model dynamically adjusts the number of Transformer modules in the large model based on the model depth predicted by the deep predictor.

[0074] Figure 2 This is a schematic diagram of a large model training method provided in an embodiment of this application, such as... Figure 2 As shown, the method includes:

[0075] Step 201: Input the training samples into the encoder and the depth predictor respectively; the training samples include inferred tokens and tokens to be inferred.

[0076] Here, "inferred tokens" refers to the sequence of tokens that have already been generated or determined, while "to be inferred tokens" refers to the next token that needs to be predicted by the large model. When training the large model, a large number of training samples can be used. These training samples can be text data on different topics collected from news websites, blogs, forums, natural language processing-related databases, etc.

[0077] Step 202: After the training samples are processed by the encoder, multi-layer Transformer modules and decoder, the prediction results of the token to be inferred and the model depth of the decoder output of each Transformer module are obtained.

[0078] The encoder maps the inferred tokens to the latent space to form a latent space vector. The encoder uses a common MLP or multi-layer fully connected layer for information compression.

[0079] The multi-layer Transformer module processes the encoded features sequentially. Each Transformer module performs more in-depth processing and transformation of the features, capturing semantic and dependency information at different levels.

[0080] Each Transformer module's corresponding decoder generates a prediction of the token to be inferred and its corresponding model depth based on the layer's output. The prediction represents the model's probability distribution for the next token, while the model depth reflects the layer's position within the model.

[0081] It should be noted that the Transformer module can adopt a decoder-only architecture of Transformer, consisting of multiple stacked Transformer modules. Each layer includes an Attention module and an FFN module or a MOE module. The initial Transformer module can consist of 100 stacked layers. In addition, the number of layers in the initial Transformer module can be adjusted according to the actual situation, and this application does not impose a specific limitation on this.

[0082] Step 203: Predict the depth of the model based on the training samples using the depth predictor.

[0083] Deep predictors can employ MLPs or multi-layer fully connected layer architectures. Based on the inferred and pending tokens in the training samples, a deep predictor predicts the depth the entire model should use during inference. This predicted depth takes into account factors such as task complexity and data characteristics.

[0084] Step 204: Optimize the internal parameters of the large model based on the token to be inferred, the prediction result, the model depth, and the prediction model depth to obtain the trained large model.

[0085] Based on the token to be inferred, the prediction result, the model depth, and the predicted model depth, the internal parameters of the large model are optimized. By defining appropriate loss functions, such as cross-entropy loss to measure the difference between the predicted result and the true token to be inferred, and model depth loss to measure the difference between the predicted model depth and the actual model depth, the loss value is calculated. Then, using the backpropagation algorithm and an optimizer (such as Adam), the model's internal parameters are adjusted according to the loss value to minimize the loss, improve the model's prediction accuracy and the reasonableness of the depth prediction, and finally obtain the trained large model.

[0086] In this embodiment, the large model includes a depth predictor. By training the depth predictor, the depth predictor can dynamically adjust its depth according to the complexity of different tasks or data, improving inference performance while reducing the waste of computing resources.

[0087] Based on the above embodiments, the internal parameters of the large model are optimized based on the prediction results, model depth, and prediction model depth, including:

[0088] Determine the inference loss based on the token to be inferred and the prediction result;

[0089] Determine the model depth loss based on the model depth and the prediction model depth;

[0090] The internal parameters of large models are optimized based on inference loss and model depth loss.

[0091] In practical implementation, the inference loss can be calculated using the cross-entropy loss function. Specifically, the predicted result (i.e., the probability distribution of the next token output by the inference module of the large model) and the actual token to be inferred can be substituted into the cross-entropy loss function to calculate the difference between the two. The cross-entropy loss measures the gap between the predicted probability distribution and the actual distribution. The larger the gap, the higher the loss value, indicating that the model's prediction result differs significantly from the actual value.

[0092] The model depth loss can be determined based on the model depth and the predicted model depth. Model depth refers to the number of layers corresponding to each Transformer module. The predicted model depth refers to the model depth predicted by the depth predictor based on the inferred tokens. The model depth loss can be calculated using the cross-entropy loss function or other loss functions.

[0093] After obtaining the inference loss and model depth loss, the internal parameters of the large model are optimized based on these losses. Gradient descent and its variants (e.g., Adam, RMSprop) can be used for parameter optimization. The specific steps are as follows:

[0094] Calculating the gradient: Using the backpropagation algorithm, the gradient of the total loss with respect to each parameter of the model is calculated. The gradient represents the direction and extent of the effect of parameter changes on the loss.

[0095] Update parameters: Adjust the model's internal parameters based on the calculated gradient and the preset learning rate. For parameters in the inference module and deep predictor, update them in the direction of gradient descent to reduce the total loss. For example, the new parameter value = old parameter value - learning rate × gradient.

[0096] Iterative training: Repeat the process of calculating gradients and updating parameters for multiple epochs until the model's loss reaches the expected target or converges. In each epoch, different batches of training samples are used to calculate the loss and gradients, progressively optimizing the model's parameters.

[0097] It should be noted that the decoders corresponding to the Transformer modules of different layers can share model parameters.

[0098] This application combines inference loss and model depth loss to optimize the internal parameters of a large model, and evaluates the model from two aspects: the accuracy of the prediction results and the rationality of the model depth, making the optimization process more comprehensive and accurate.

[0099] Based on the above embodiments, the internal parameters of the large model are optimized according to the inference loss and model depth loss, including:

[0100] Optimize the internal parameters of the inference module based on the inference loss;

[0101] The total loss is determined based on inference loss and model depth loss;

[0102] The internal parameters of the depth predictor are optimized based on the total loss.

[0103] In the specific implementation process, since the inference module mainly predicts the next token based on the already inferred tokens, the internal parameters of the inference module can be optimized according to the inference loss. The specific optimization methods can be found in the above embodiments and will not be repeated here.

[0104] The depth predictor predicts the model depth required to infer the next token based on already inferred tokens. The predicted model depth needs to balance inference accuracy and minimizing model depth. Therefore, the internal parameters of the depth predictor can be optimized based on inference loss and model depth loss. Specifically, the sum of the inference loss and model depth loss for each Transformer layer can be calculated to obtain the intermediate loss for each Transformer layer. The minimum value among these intermediate losses is selected as the total loss. The total loss includes both a measure of the accuracy of the prediction result and a measure of the reasonableness of the model depth prediction. Then, the internal parameters of the depth predictor are optimized based on the total loss.

[0105] The formula for calculating inference loss can be:

[0106] L LM =-logp θ (x i+1 |x0,x1,...,x i-2 ,x i-1,x i )

[0107] The formula for calculating the model depth loss can be:

[0108]

[0109] The formula for calculating intermediate loss is:

[0110]

[0111] Where, x i For the i-th token, the model depth predictor is treated as a classification model, with different model depths considered as different classes. The loss of the model depth predictor is defined using multi-class cross-entropy loss, L. depth For the model depth prediction loss, depth h This represents the depth information of the h-th layer of the stacked Transformer modules, where depth' is the probability of the model's depth predictor outputting depth, H is the total depth of the model (i.e., the total number of Transformer modules), and λ is the weight parameter (e.g., setting λ = 0.0001). i This represents the intermediate loss corresponding to the module of the i-th layer Transformer; It is the inference loss corresponding to the module of the i-th layer Transformer; This is the depth prediction loss of the i-th layer. The value of i is a positive integer, and its maximum value is the total number of layers in the Transformer module.

[0112] In this embodiment, the internal parameters of the inference module are specifically optimized using inference loss. This targeted optimization method can more accurately adjust the inference module, significantly improving its accuracy and reliability in predicting the token to be inferred. When optimizing the depth predictor, not only inference loss is considered, but also model depth loss is combined to determine the total loss. Then, the internal parameters of the depth predictor are optimized based on the total loss. This comprehensive optimization method considers both the accuracy of the depth predictor in predicting model depth and its synergistic effect with the inference module, enabling the depth predictor to better provide reasonable depth support for the entire model's inference process, thereby improving the model's adaptability and efficiency.

[0113] Based on the above embodiments, while optimizing the internal parameters of the inference module, the internal parameters of the depth predictor remain unchanged, thus obtaining the intermediate inference module;

[0114] During the optimization of the internal parameters of the depth predictor, the internal parameters of the Transformer module, encoder, and decoder remain unchanged, resulting in an intermediate depth predictor.

[0115] The internal parameters of the intermediate inference module and intermediate depth predictor are further optimized using inference loss and model depth loss.

[0116] In its specific implementation, this application describes a method for phased optimization of the inference module and deep predictor in a large model. By optimizing different modules at different stages and finally performing joint optimization, the method achieves gradual improvement and comprehensive optimization of model performance. The following is a detailed description:

[0117] Phase 1: Optimize the inference module to improve the accuracy of the inference module's prediction of the next token.

[0118] Parameter optimization scope: Only parameters within the inference module are optimized, including those of the encoder, multi-layer Transformer module, and decoder, while the internal parameters of the depth predictor remain unchanged.

[0119] Loss calculation: The inference loss is used as the basis for optimization. It is calculated by comparing the prediction results of the token to be inferred and the inference module. The cross-entropy loss function is usually used.

[0120] Parameter update: The gradient of the inference loss with respect to each parameter in the inference module is calculated using the backpropagation algorithm. Then, these parameters are updated using the gradient descent algorithm and its variants (such as Adam, RMSprop, etc.) to minimize the inference loss.

[0121] After this stage of optimization, an intermediate inference module with better performance in predicting the next token is obtained.

[0122] Phase 2: Optimize the depth predictor to improve the accuracy of the depth predictor in predicting model depth, while also considering its collaborative work with the inference module.

[0123] Parameter optimization scope: Only the parameters inside the depth predictor are optimized, while the internal parameters of the Transformer module, encoder, and decoder remain unchanged.

[0124] Loss Calculation: The total loss is determined by combining inference loss and model depth loss. Inference loss reflects the prediction accuracy of the inference module, while model depth loss measures the difference between the predicted model depth and the actual model depth, typically using the mean squared error loss function.

[0125] Parameter update: The gradient of the total loss with respect to the individual parameters in the depth predictor is calculated using the backpropagation algorithm. Then, the parameters of the depth predictor are updated using the gradient descent algorithm and its variants to minimize the total loss.

[0126] After this stage of optimization, an intermediate depth predictor with better performance in predicting model depth is obtained.

[0127] The third stage involves jointly optimizing the intermediate inference module and the intermediate deep predictor to further improve the overall performance of the model and achieve better synergy and balance between the inference module and the deep predictor.

[0128] Parameter optimization scope: Simultaneously optimize the internal parameters of the intermediate inference module and the intermediate depth predictor.

[0129] Loss calculation: The inference loss and model depth loss are still used to calculate the total loss, which is the objective function for optimization.

[0130] Parameter update: The gradient of the total loss with respect to the individual parameters in both modules is calculated using the backpropagation algorithm. Then, all parameters are updated using the gradient descent algorithm and its variants to minimize the total loss.

[0131] After this stage of joint optimization, a large model is finally obtained that has been comprehensively improved in terms of prediction accuracy and depth rationality.

[0132] It should be noted that the phased optimization process can be divided into different stages according to actual needs. For example: in the first stage, only the Transformer module is optimized, while the internal parameters of the encoder, decoder, and depth predictor remain unchanged; in the second stage, the encoder, decoder, and depth predictor are optimized, while the internal parameters of the Transformer module remain unchanged; in the third stage, the optimized Transformer module and the optimized encoder, decoder, and depth predictor are jointly optimized to obtain the final model.

[0133] In this embodiment, when optimizing the inference module, the parameters of the depth predictor are kept constant, allowing for targeted adjustment of the inference module's internal parameters to achieve optimal performance in predicting the tokens to be inferred. Similarly, when optimizing the depth predictor, fixing the parameters of the Transformer module, encoder, and decoder allows for a focus on improving the depth predictor's accuracy in predicting model depth. This phased optimization approach avoids interference between parameters, ensuring that each module is fully adjusted and improved in its respective optimization phase, and enhancing the efficiency of the entire training process.

[0134] After training a large model, it can be deployed on a server or terminal, and is compatible with inference frameworks such as VLLM and SGLang during deployment. At the cluster deployment level, it supports techniques such as PD separation. It supports commonly used low-bit quantization precision, including common LLM quantization techniques such as LLM.int8() and soomthquant.

[0135] Based on the large model obtained through the above training, embodiments of this application provide a large model inference method, such as... Figure 3 As shown, the method includes the following steps:

[0136] Step 301: Input the reasoned tokens into the large model;

[0137] Step 302: Process the inferred tokens using the depth predictor in the large model to generate the model depth required for the large model to infer the next token;

[0138] Step 303: Based on the model depth, obtain the inference result of the next token output by the inference module at the corresponding depth based on the already inferred tokens.

[0139] In the specific implementation process Figure 4 The model inference principle diagram provided for the embodiments of this application is as follows: Figure 4 As shown. In scenarios where users ask questions to a large model and the model provides answers, the questions input by the user are also considered inferred tokens. Therefore, inferred tokens include both the user-input questions and the tokens already inferred by the large model during the inference process. In the later stages of inference, the number of inferred tokens increases significantly. The depth predictor may not need all inferred tokens when predicting the depth required to infer the next token. Therefore, it can select the necessary tokens from the inferred tokens. For example, it can select a preset number of recently added tokens, a preset number of the first few tokens, or a preset number of the last few tokens. These selected inferred tokens are used as the basis for prediction. It should be noted that the specific value of the preset number can be set based on experience or determined as a percentage of the inferred tokens. This embodiment does not impose specific limitations on this.

[0140] Similarly, when the inference module performs inference based on the inferred tokens, it may not need all the inferred tokens. The method for selecting the required inferred tokens from the inferred tokens may be to select the most recently added preset number of tokens, or to select the first preset number of tokens and the last preset number of tokens in the inferred tokens, and use the selected inferred tokens as the basis for inferring the next token.

[0141] Therefore, for inference of the next token, the deep predictor can extract and analyze features from the already inferred tokens to understand the complexity of the current task and the characteristics of the data. The deep predictor comprehensively considers factors such as the content of the already inferred tokens, the complexity of the task, and the model's training experience to predict the most suitable model depth required for inferring the next token. Model depth is used to indicate the number of Transformer modules required in the inference module of a large model when inferring the next token.

[0142] Based on the model depth generated by the deep predictor, the Transformer module and its corresponding decoder at the corresponding depth are determined from the inference modules of the large model. For example, if the predicted model depth is 3, the 3rd layer Transformer module and its corresponding decoder are selected as the inference module for the current inference task. The inferred tokens are input into the selected inference module. First, the encoder encodes the inferred tokens, converting them into high-dimensional feature representations. Then, the selected multi-layer Transformer modules process the encoded features sequentially. Each layer of the Transformer module performs further processing and transformation on the features to capture the complex dependencies between tokens. Finally, the decoder of the corresponding layer generates the inference result for the next token based on the output of the Transformer module. The inference result for the next token generated by the inference module can be a probability distribution, representing the likelihood of different tokens being the next token. Based on this probability distribution, the token with the highest probability can be selected as the final inference result output, or other selection strategies, such as sampling, can be adopted according to specific application requirements.

[0143] After reasoning about a token, the reasoned token is placed into the reasoned tokens list. Then, reasoning is performed on the next token. When reasoning about the next token, the same steps are followed until all tokens have been reasoned.

[0144] This application embodiment uses a depth predictor to predict the model depth required for inferring the next token using a large model, and infers the next token based on the inference model corresponding to the predicted model depth. This large model can automatically adjust the model depth used in the inference process according to the complexity of the task and the characteristics of the data, avoiding the problem of overcomputation of a fixed-depth model in some cases, and improving inference efficiency.

[0145] In another embodiment, the large model provided in this application can be implemented as a long context model through YARN or other means, aligned with humans through reinforcement learning, and combined with other applications and model enhancements such as Monte Carlo tree search.

[0146] Figure 5 This is a schematic diagram of a large model training device provided in an embodiment of this application. The device can be a module, program segment, or code on an electronic device. It should be understood that this device is similar to the one described above. Figure 2 The method implementation corresponds to this and can be executed. Figure 2 The specific functions of the device involved in the various steps of the method embodiment can be found in the description above; to avoid repetition, detailed descriptions are omitted here. The device includes: a sample input module 501, a token prediction module 502, a first depth prediction module 503, and a parameter optimization module 504, wherein:

[0147] The sample input module 501 is used to input training samples into the encoder and the depth predictor respectively; the training samples include inferred tokens and tokens to be inferred;

[0148] The token prediction module 502 is used to obtain the prediction result of the token to be inferred and the model depth of the decoder output by each layer of the Transformer module after the training samples are processed by the encoder, the multi-layer Transformer module and the decoder.

[0149] The first depth prediction module 503 is used to predict the depth of the model obtained by the depth predictor based on the training samples.

[0150] The parameter optimization module 504 is used to optimize the internal parameters of the large model based on the token to be inferred, the prediction result, the model depth, and the prediction model depth to obtain the trained large model.

[0151] Based on the above embodiments, the parameter optimization module 504 is specifically used for:

[0152] The inference loss is determined based on the token to be inferred and the prediction result;

[0153] The model depth loss is determined based on the model depth and the predicted model depth.

[0154] The internal parameters of the large model are optimized based on the inference loss and the model depth loss.

[0155] Based on the above embodiments, the parameter optimization module 504 is specifically used for:

[0156] The internal parameters of the inference module are optimized based on the inference loss.

[0157] The total loss is determined based on the inference loss and the model depth loss.

[0158] The internal parameters of the depth predictor are optimized based on the total loss.

[0159] Based on the above embodiments, the parameter optimization module 504 is specifically used for:

[0160] The sum of the inference loss and model depth loss corresponding to the decoder of each Transformer module is used as the intermediate loss of the corresponding Transformer module.

[0161] The minimum intermediate loss is taken as the total loss.

[0162] Based on the above embodiments, the parameter optimization module 504 is specifically used for:

[0163] During the optimization of the internal parameters of the depth predictor, the internal parameters of the Transformer module, the encoder, and the decoder remain unchanged, thus obtaining an intermediate depth predictor;

[0164] The internal parameters of the intermediate inference module and the intermediate depth predictor are further optimized using the inference loss and the model depth loss.

[0165] Figure 6 This is a schematic diagram of a large model inference device provided in an embodiment of this application. The device can be a module, program segment, or code on an electronic device. It should be understood that this device is similar to the one described above. Figure 3 The method implementation corresponds to this and can be executed. Figure 3 The various steps involved in the method embodiment, and the specific functions of the device, can be found in the description above. To avoid repetition, detailed descriptions are appropriately omitted here. The device includes: an input module 601, a second depth prediction module 602, and an inference module 603; wherein:

[0166] Input module 601 is used to input the reasoned tokens into the large model; the large model is obtained by training using the large model training method provided in the above embodiments;

[0167] The second depth prediction module 602 is used to process the inferred tokens through the depth predictor in the large model and generate the model depth required for the large model to infer the next token.

[0168] The inference module 603 is used to obtain the inference result of the next token output by the inference module at the corresponding depth based on the already inferred tokens, based on the model depth.

[0169] Based on the above embodiments, the number of reasoned tokens is a preset number of tokens selected from the sequence of reasoned tokens.

[0170] Figure 7 This is a schematic diagram of the physical structure of the electronic device provided in the embodiments of this application, such as... Figure 7 As shown, the electronic device includes: a processor 701, a memory 702, and a bus 703; wherein:

[0171] The processor 701 and the memory 702 communicate with each other through the bus 703;

[0172] The processor 701 is used to call program instructions in the memory 702 to execute the methods provided in the above-described method embodiments, including, for example,: inputting training samples into the encoder and the depth predictor respectively; the training samples include inferred tokens and tokens to be inferred; after the encoder, the multi-layer Transformer module, and the decoder process the training samples, obtaining the prediction result and model depth of the token to be inferred output by the decoder corresponding to each layer Transformer module; predicting the model depth based on the training samples obtained by the depth predictor; optimizing the internal parameters of the large model based on the token to be inferred, the prediction result, the model depth, and the predicted model depth to obtain the trained large model. Alternatively, inputting inferred tokens into the large model; the large model is obtained by training using the large model training method described in the above-described embodiments; processing the inferred tokens through the depth predictor in the large model to generate the model depth required for the large model to infer the next token; obtaining the inference result of the next token output by the inference module of the corresponding depth based on the inferred tokens based on the model depth.

[0173] The processor 701 can be an integrated circuit chip with signal processing capabilities. The processor 701 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0174] The memory 702 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0175] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can perform the methods provided in the above-described method embodiments, such as: inputting training samples into the encoder and the depth predictor respectively; the training samples include inferred tokens and tokens to be inferred; after the training samples are processed by the encoder, the multi-layer Transformer module, and the decoder, obtaining the prediction result and model depth of the token to be inferred output by the decoder corresponding to each layer Transformer module; predicting the model depth based on the training samples obtained by the depth predictor; and optimizing the internal parameters of the large model based on the token to be inferred, the prediction result, the model depth, and the predicted model depth to obtain the trained large model. Alternatively, the reasoned tokens can be input into the large model; the large model is trained using the large model training method described in the above embodiments; the reasoned tokens are processed by the depth predictor in the large model to generate the model depth required for the large model to infer the next token; based on the model depth, the inference result of the inference module at the corresponding depth for the next token is obtained based on the reasoned tokens.

[0176] This embodiment provides a non-transitory computer-readable storage medium storing computer instructions that cause the computer to execute the methods provided in the above-described method embodiments. These instructions include, for example: inputting training samples into the encoder and the depth predictor respectively; the training samples include inferred tokens and tokens to be inferred; after processing the training samples by the encoder, the multi-layer Transformer module, and the decoder, obtaining the prediction result and model depth of the token to be inferred output by the decoder corresponding to each layer of the Transformer module; predicting the model depth based on the training samples obtained by the depth predictor; and optimizing the internal parameters of the large model based on the token to be inferred, the prediction result, the model depth, and the predicted model depth to obtain the trained large model. Alternatively, the reasoned tokens can be input into the large model; the large model is trained using the large model training method described in the above embodiments; the reasoned tokens are processed by the depth predictor in the large model to generate the model depth required for the large model to infer the next token; based on the model depth, the inference result of the inference module at the corresponding depth for the next token is obtained based on the reasoned tokens.

[0177] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0178] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0179] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0180] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0181] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for training large models, characterized in that, The large model includes an inference module and a deep predictor; wherein the inference module includes an encoder, a multi-layer Transformer module, and a decoder corresponding to each layer of the Transformer module, and the method includes: Training samples are input into the encoder and the depth predictor, respectively; the training samples include inferred tokens and tokens to be inferred; the training samples are text data on different topics. After the encoder, the multi-layer Transformer module, and the decoder process the training samples, the prediction result of the token to be inferred and the model depth of the decoder output corresponding to each layer Transformer module are obtained. The depth of the predicted model is obtained by the depth predictor based on the training samples; The internal parameters of the large model are optimized based on the token to be inferred, the prediction result, the model depth, and the prediction model depth to obtain the trained large model. The optimization of the internal parameters of the large model based on the prediction results, the model depth, and the prediction model depth includes: The inference loss is determined based on the token to be inferred and the prediction result; The model depth loss is determined based on the model depth and the predicted model depth. The internal parameters of the large model are optimized based on the inference loss and the model depth loss. The optimization of the internal parameters of the large model based on the inference loss and the model depth loss includes: The internal parameters of the inference module are optimized based on the inference loss. The total loss is determined based on the inference loss and the model depth loss. The internal parameters of the depth predictor are optimized based on the total loss. During the optimization of the internal parameters of the inference module, the internal parameters of the depth predictor remain unchanged, thus obtaining the intermediate inference module; During the optimization of the internal parameters of the depth predictor, the internal parameters of the Transformer module, the encoder, and the decoder remain unchanged, thus obtaining an intermediate depth predictor; The internal parameters of the intermediate inference module and the intermediate depth predictor are further optimized using the inference loss and the model depth loss.

2. The method according to claim 1, characterized in that, The step of determining the total loss based on the inference loss and the model depth loss includes: The sum of the inference loss and model depth loss corresponding to the decoder of each Transformer module is used as the intermediate loss of the corresponding Transformer module. The minimum intermediate loss is taken as the total loss.

3. A large-scale model reasoning method, characterized in that, include: Input the inferred tokens into the large model; The large model is obtained by training using the large model training method as described in any one of claims 1-2; The already inferred tokens are processed by the depth predictor in the large model to generate the model depth required for the next token to be inferred. Based on the model depth, the inference module at the corresponding depth outputs the inference result for the next token based on the already inferred tokens.

4. The method according to claim 3, characterized in that, The number of reasoned tokens is a preset number of tokens selected from the sequence of reasoned tokens.

5. A large-scale model training device, characterized in that, The large model includes an inference module and a deep predictor; wherein the inference module includes an encoder, a multi-layer Transformer module, and a decoder corresponding to each layer of the Transformer module, and the device includes: The sample input module is used to input training samples into the encoder and the depth predictor respectively; the training samples include inferred tokens and tokens to be inferred; the training samples are text data on different topics; The token prediction module is used to obtain the prediction result of the token to be inferred and the model depth of the decoder output by the encoder, the multi-layer Transformer module and the decoder after the training samples are processed by the encoder and the decoder. The first depth prediction module is used to predict the depth of the model obtained by the depth predictor based on the training samples. The parameter optimization module is used to optimize the internal parameters of the large model based on the token to be inferred, the prediction result, the model depth, and the prediction model depth to obtain the trained large model. The parameter optimization module is specifically used for: The inference loss is determined based on the token to be inferred and the prediction result; The model depth loss is determined based on the model depth and the predicted model depth. The internal parameters of the large model are optimized based on the inference loss and the model depth loss. The optimization of the internal parameters of the large model based on the inference loss and the model depth loss includes: The internal parameters of the inference module are optimized based on the inference loss. The total loss is determined based on the inference loss and the model depth loss. The internal parameters of the depth predictor are optimized based on the total loss. During the optimization of the internal parameters of the inference module, the internal parameters of the depth predictor remain unchanged, thus obtaining the intermediate inference module; During the optimization of the internal parameters of the depth predictor, the internal parameters of the Transformer module, the encoder, and the decoder remain unchanged, thus obtaining an intermediate depth predictor; The internal parameters of the intermediate inference module and the intermediate depth predictor are further optimized using the inference loss and the model depth loss.

6. A large-scale model reasoning device, characterized in that, include: The input module is used to input the reasoned tokens into the large model; The large model is obtained by training using the large model training method as described in any one of claims 1-2; The second depth prediction module is used to process the reasoned tokens through the depth predictor in the large model to generate the model depth required for the large model to infer the next token; The inference module is used to obtain the inference result of the next token output by the inference module at the corresponding depth based on the already inferred tokens, based on the model depth.

7. An electronic device, characterized in that, include: Processor, memory, and bus, among which: The processor and the memory communicate with each other via the bus; The memory stores program instructions that can be executed by the processor, and the processor can execute the method as described in any one of claims 1-4 by calling the program instructions.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-4.

9. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1-4.

Images