Model-based data processing method and device and model reasoning acceleration method and device

Abstract

The invention provides a model-based data processing method and device and a model reasoning acceleration method and device. According to the scheme, the target model is obtained by inserting the extension parameters into the original parameters of the machine learning model, and the inserted extension parameters do not change the reasoning result of the original parameters of the machine learning model. During reasoning, feature representation of input data is obtained through original parameter reasoning of a target model, and calibration information is synchronously generated through extension parameters; the reasoning result generated according to the feature representation is calibrated according to the calibration information, the calibration result is obtained, the feature representation and the calibration information of the input data can be obtained at the same time through one reasoning process, the time expenditure for obtaining the calibration information is reduced, and therefore the influence of the calibration process on the model output response timeliness is reduced; moreover, only a small number of extension parameters need to be inserted into the original model, and an additional reward model with a relatively large parameter scale does not need to be used, so that the space overhead of the calibration scheme is reduced.

Description

Technical Field

[0001] This application relates to computer technology, and more particularly to a method and apparatus for model-based data processing and model inference acceleration. Background Technology

[0002] With the continuous development of artificial intelligence technology, generative models (such as large language models) have performed well in Natural Language Processing (NLP) tasks, but they also have some problems. One approach is to intervene in model inference by using an additional reward model to generate reward signals. These reward signals are used to calibrate the output during model inference, which can improve the quality of the model's generated results. When calibration is no longer needed, the reward model can be removed, allowing the model to revert to its original output.

[0003] Traditional inference-based intervention schemes largely rely on training an additional reward model. When intervening in the model through inference, the output of the model being intervened upon is fed into the reward model. The reward model then infers and generates a calibration signal based on the output, and uses this calibration signal to calibrate the input. However, the large parameter size of the reward model leads to a sharp increase in space complexity and a long inference time for the calibration signal. Therefore, existing model inference-based intervention schemes incur significant space and time overhead, severely impacting the timeliness of the model's output response. Summary of the Invention

[0004] This application provides a model-based data processing and model inference acceleration method and apparatus to solve the problem that existing model inference calibration schemes have large space and time overhead, which seriously affects the timeliness of model output response.

[0005] In a first aspect, this application provides a model-based data processing method, comprising: inputting input data into a target model, wherein the target model is obtained by inserting extended parameters into the original parameters of a machine learning model, and the extended parameters do not change the inference result of the original parameters of the machine learning model; obtaining a feature representation of the input data through inference using the original parameters in the target model, and synchronously generating calibration information using the extended parameters; and calibrating the inference result generated based on the feature representation according to the calibration information to obtain a calibration result.

[0006] Secondly, this application provides a method for accelerating model inference, comprising: inputting input data into a target model, performing inference at multiple time steps through the target model to generate target text; wherein the target model is obtained by inserting extended parameters into the original parameters of a text generation model, and the extended parameters do not change the inference result of the original parameters of the text generation model; wherein the inference process at each time step comprises: inputting the input data and the generated text tokens into the target model, obtaining an input feature representation through inference using the original parameters of the target model, and synchronously generating an extended feature representation through the extended parameters; generating the next text token based on the input feature representation, and generating at least one text token after the next text token based on the extended feature representation.

[0007] Thirdly, this application provides a model-based data processing method, comprising: in response to a natural language processing task request, acquiring input data; inputting the input data into a target model, wherein the target model is obtained by inserting extended parameters into the original parameters of the natural language processing model, and the extended parameters do not change the inference result of the original parameters of the natural language processing model; performing inference through the original parameters in the target model to obtain a feature representation of the input data, and synchronously generating calibration information through the extended parameters; calibrating the natural language processing result generated based on the feature representation according to the calibration information, and obtaining a calibrated natural language processing result.

[0008] Fourthly, this application provides a model-based data processing method, comprising: receiving query information sent by a terminal device; inputting the query information into a target language model, wherein the target language model is obtained by inserting extended parameters into the original parameters of a pre-trained language model, and the extended parameters do not change the inference result of the original parameters of the pre-trained language model; performing inference through the original parameters in the target language model to obtain a feature representation of the query information, and synchronously generating calibration information through the extended parameters; calibrating the response result generated based on the feature representation of the query information according to the calibration information to obtain a calibrated response result; and outputting the calibrated response result.

[0009] Fifthly, this application provides a server, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the server to perform the methods provided in any of the foregoing aspects.

[0010] Sixthly, this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method provided in any of the foregoing aspects.

[0011] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the methods provided in any of the foregoing aspects.

[0012] The method and apparatus for model-based data processing and model inference acceleration provided in this application obtain a target model by inserting extended parameters into the original parameters of a machine learning model, and the inserted extended parameters do not change the inference result of the original parameters of the machine learning model. During inference, input data is input into the target model, and the feature representation of the input data is obtained through inference using the original parameters in the target model. Calibration information is simultaneously generated using the extended parameters. Based on the calibration information, the inference result generated from the feature representation is calibrated to obtain a calibration result. The method of this application only requires inserting a small number of extended parameters into the original machine learning model. During the process of obtaining the feature representation of the input data through inference in the original machine learning model, calibration information can be generated simultaneously using the extended parameters. The feature representation of the input data and calibration information can be obtained simultaneously in a single inference process, reducing the time overhead of obtaining calibration information and thus reducing the impact of the calibration process on the timeliness of the model's output response. Furthermore, it eliminates the need for an additional reward model with a large parameter scale, reducing the space overhead of the calibration scheme. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0014] Figure 1 This is a framework diagram of a model reasoning intervention scheme based on a reward model;

[0015] Figure 2 A framework diagram of the model reasoning intervention scheme provided in this application;

[0016] Figure 3 This is a schematic diagram of an example system architecture to which this application applies;

[0017] Figure 4 A flowchart illustrating a model-based data processing method provided in an exemplary embodiment of this application;

[0018] Figure 5 A flowchart illustrating a method for obtaining a target model provided in an exemplary embodiment of this application;

[0019] Figure 6A schematic diagram of the architecture of an extended model provided for an exemplary embodiment of this application;

[0020] Figure 7 A schematic diagram of an extended converter block provided for an exemplary embodiment of this application;

[0021] Figure 8 An extended schematic diagram of a linear mapping matrix provided for an exemplary embodiment of this application;

[0022] Figure 9 A flowchart illustrating a model training method provided in an exemplary embodiment of this application;

[0023] Figure 10 A flowchart of a model-based data processing method provided as another exemplary embodiment of this application;

[0024] Figure 11 A flowchart of a model-based data processing method provided as another exemplary embodiment of this application;

[0025] Figure 12 This is a schematic diagram of the structure of a server provided in an embodiment of this application.

[0026] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0027] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0028] It should be noted that the user information (including but not limited to user device information, user attribute information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0029] First, let me explain the terms used in this application:

[0030] Non-intrusive: This means that the newly added parameters will not change the original model architecture / parameters / computation process, etc. The original model will maintain its original output.

[0031] A token, also known as a discrete token or discrete representation, refers to a discrete representation of speech or text. To distinguish between the discrete representations of text and speech, the discrete representation of speech is called a speech token, and the discrete representation of text is called a text token. The vocabulary used by the model contains both speech and text tokens that may appear in the prediction results. Each speech token in the vocabulary corresponds to a speech unit, such as a phoneme, syllable, or speech segment. Each text token in the vocabulary corresponds to a text unit, such as a character or word. This embodiment uses text tokens.

[0032] Transformer is a deep learning architecture widely used in natural language processing tasks, which can effectively solve the problem of modeling sequential data. Its core is the attention mechanism.

[0033] Transformer Block: This refers to the basic structural unit of the Transformer.

[0034] Inference intervention: refers to interfering with the model's output by modifying / intervening in the probability distribution of the next one or more tokens predicted by the model during the inference process.

[0035] Time cost: refers to the time required to perform a task.

[0036] Space overhead: refers to the resources such as GPU and memory required to perform a task.

[0037] Large Language Model (LLM): Generally refers to an autoregressive language model based on the Transformer architecture.

[0038] Visual question answering task: Based on the input image and the question, determine the answer to the question from the visual information of the input image.

[0039] Image description task: Generate descriptive text for the input image.

[0040] Visual entailment task: Predict the semantic relevance between input images and text, i.e., entailment, neutrality, or contradiction.

[0041] The task of expression and comprehension involves locating the image region in the input image that corresponds to the input text.

[0042] Image generation task: Generate an image based on the input descriptive text.

[0043] Text-based sentiment classification task: Predict the sentiment classification information of input text.

[0044] Text summarization task: Generate a summary of the input text.

[0045] Multimodal tasks refer to downstream tasks that involve multiple modalities of data, such as images and text, in their input and output. Examples include visual question answering, image description, visual entailment, representation and understanding, and image generation.

[0046] Multimodal pre-trained models refer to pre-trained models whose input and output data involve multiple modalities such as images and text. After fine-tuning and training, they can be applied to multimodal task processing.

[0047] Pre-trained language model: A pre-trained model obtained by pre-training a language model (LM).

[0048] Large models refer to deep learning models with a massive number of parameters, typically containing hundreds of millions, tens of billions, or even trillions of parameters. Large models are also known as foundation models (FM), which are pre-trained on large-scale unlabeled corpora to produce pre-trained models with hundreds of millions of parameters. These models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and Multi-modal Pre-training Models.

[0049] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0050] Generative models, such as large language models, are widely used in text generation tasks and can generate text according to customer needs. Here, models used for text generation are collectively referred to as text generation models, which can specifically include various language models, large models, natural language processing models, multimodal models, etc.

[0051] Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of natural language processing tasks, including text generation, translation, and summarization. Extensive research from multiple perspectives has revealed that LLMs can sometimes produce unsafe or harmful outputs; regarding reasoning capabilities, they can generate unreliable mathematical proofs.

[0052] To address the aforementioned issues and avoid catastrophic forgetting and knowledge illusions caused by continuous model training, plug-and-play inference intervention schemes have emerged. Existing inference intervention schemes employ an additional reward model to generate reward signals, which are used to calibrate the output during model inference. When intervention is no longer needed, the reward model can be removed, allowing the model to revert to its original output.

[0053] For example, such as Figure 1 As shown, in existing inference intervention schemes, input data is fed into the original model being intervened for inference to generate prediction results. The prediction results are then fed into an additionally trained reward model for inference to obtain a calibration signal; finally, the calibration signal is used to calibrate the original prediction results to obtain the calibrated output results.

[0054] Therefore, existing inference-based intervention schemes rely on training additional reward models, which are typically large-scale models, leading to a sharp increase in space complexity. Furthermore, existing inference-based intervention schemes generate calibration signals through inference using reward models after the original model outputs a prediction, and the inference time is usually quite long. Thus, existing model-based inference intervention schemes incur significant space and time overhead, severely impacting the timeliness of the model's output response.

[0055] This application provides a model-based data processing method aimed at solving the problems of high time and space overhead in traditional inference intervention schemes. The proposed solution obtains a target model by inserting extended parameters into the original parameters of a machine learning model, without altering the inference results of the original parameters. During inference, input data is fed into the target model, and feature representations of the input data are obtained through inference using the original parameters in the target model. Calibration information is simultaneously generated using the extended parameters. Based on the calibration information, the inference results generated from the feature representations are calibrated to obtain calibration results. In this embodiment, the target model is obtained by inserting extended parameters into the original machine learning model and training these extended parameters. The extended parameters are used to simultaneously generate calibration information during the process of obtaining feature representations of the input data through inference in the original machine learning model. Furthermore, the addition of extended parameters is non-intrusive and does not affect the process of obtaining feature representations of the input data through inference in the original machine learning model. That is, the feature representations of the input data obtained through inference in the target model are consistent with the feature representations obtained by the original machine learning model based on the same input data before extension.

[0056] The machine learning model can be any type of natural language processing model, including but not limited to various large language models (LLM), pre-trained language models, Transformer-based natural language models, and language models combined with retrieval enhancement.

[0057] For example, such as Figure 2 As shown, the method provided in this application obtains a target model by inserting extended parameters into the original parameters of a machine learning model. Input data is then input into the target model, and feature representations of the input data are obtained through inference using the original parameters in the target model. Calibration information is simultaneously generated using the extended parameters. Furthermore, the inference results generated from the feature representations of the input data are calibrated based on the calibration information to obtain a calibration result.

[0058] The method of this application obtains the target model by inserting extended parameters into the original parameters of the machine learning model, and the inserted extended parameters do not change the inference results of the original parameters of the machine learning model. During inference, the target model generates calibration information simultaneously while obtaining the feature representation of the input data. The feature representation and calibration information of the input data can be obtained simultaneously in a single inference process, saving the time spent obtaining calibration information and reducing time overhead, thereby reducing the impact of the calibration process on the timeliness of the model's output response. Furthermore, only a small number of parameters need to be inserted into the original model, eliminating the need for an additional reward model with a large parameter scale, thus reducing space overhead.

[0059] It should be noted that the solution presented in this application can also be applied to retrieval-enhanced machine learning models. When applied to retrieval-enhanced machine learning models, the method of extending the machine learning model to obtain the target model remains unchanged. No changes are needed in the retrieval enhancement stage; the retrieval results are used as part of the input data and fed into the target model. The target model then performs inference based on the input data to obtain the feature representation and calibration information of the input data. Further, the inference results generated from the feature representation of the input data are calibrated based on the calibration information to obtain the calibration results. By combining model inference intervention with retrieval-enhanced generation, external knowledge can be effectively utilized in real time, thereby further improving the quality of the inference results generated by the target model.

[0060] Figure 3 This is a schematic diagram of an example system architecture to which this application applies. Figure 3 As shown, the system architecture includes a first server, a second server, and end-side devices. The first server and the end-side devices have a communication link, enabling communication between them. Similarly, the second server and the first server have a communication link, enabling communication between them.

[0061] The endpoint device can be an electronic device running downstream applications. Specifically, it can be a hardware device with network communication, computing, and information display functions, including but not limited to smartphones, tablets, desktop computers, local servers, and cloud servers. The downstream applications running on the endpoint device can be dialogue / question-answering systems, content creation, machine translation, code generation, and copywriting generation (such as product descriptions, charts, and text summaries). When the endpoint device needs to perform natural language processing tasks while running downstream applications, it provides input data to the first server.

[0062] The primary server is a computing-capable device deployed in the cloud or locally, such as a cloud cluster. It stores machine learning models for performing natural language processing (NLP) tasks and provides NLP services. The primary server is responsible for using machine learning models to perform inference (i.e., NLP) based on given input data and obtaining the inference results. These NLP tasks include, but are not limited to, text generation, text summarization, machine translation, and natural language understanding. They can be applied to scenarios such as intelligent search, intelligent customer service, copywriting generation, intelligent recommendation, and multilingual intelligent translation, without further specific limitations.

[0063] The second server is a computing-capable device deployed in the cloud or locally, such as a cloud cluster. The second server is used to insert extended parameters into the machine learning model used to perform the natural language processing task (referred to as the original model or the model to be calibrated), obtaining an extended model. Further, using the dataset corresponding to the calibration task, the extended parameters in the extended model are adjusted to obtain the target model.

[0064] Furthermore, the second server can insert the expanded parameters trained in the target model into the original model running on the first server to hot-fix the original model into the target model. Alternatively, the second server can also replace the original model running on the first server with the target model to upgrade the original model into the target model.

[0065] Furthermore, the first server uses the upgraded target model to provide natural language processing services to the outside world.

[0066] Specifically, when the edge device needs to perform a natural language processing task while running downstream applications, it provides input data to the first server. The first server receives the input data sent by the edge device, inputs the input data into the target model, obtains the feature representation of the input data through inference using the target model, and simultaneously generates calibration information; based on the calibration information, it calibrates the inference result generated from the feature representation to obtain the calibration result.

[0067] Furthermore, the first server returns the calibration result to the end device. The end device receives the calibration result returned by the first server and executes subsequent processing logic based on the calibration result.

[0068] It should be noted that in practical applications, this solution can be applied to different calibration tasks, i.e., inference intervention tasks with different inference intervention objectives. For example, a calibration task can be an inference intervention task aimed at reducing unsafe or harmful outputs, an inference intervention task aimed at aligning the model with beneficial and harmless preferences, or an inference intervention task aimed at accelerating model inference.

[0069] In an example scenario, taking intelligent question answering as an example, the model's input data is the query information entered by the user. The model's inference result is the response result generated by the model to the query information. The edge device runs an intelligent question answering system. After the user enters the query information, the edge device sends the query information to the server. The server inputs the query information into the target model, generates a feature representation of the query information through the target model, and simultaneously generates calibration information. Based on the calibration information, the inference result generated from the feature representation is calibrated to obtain the calibration result. This calibration result is the response result to the query information.

[0070] For example, taking the calibration task's objective of reducing unsafe or harmful outputs as an example, the calibration information synchronously generated by the target model can represent the probability information that the inference result generated based on the feature representation obtained at the current time step contains unsafe or harmful outputs without calibration. If the calibration information is greater than or equal to the probability threshold, i.e., it is determined that the inference result generated at the current time step contains unsafe or harmful outputs, the inference result at the current time step can be discarded, and the inference process at the current time step can be re-executed; or at least one time step can be rolled back, and the inference process at least one time step can be re-executed to obtain a new inference result, thereby achieving the effect of retrieving unsafe or harmful outputs by calibrating the inference result.

[0071] For example, taking the goal of inference calibration to reduce unsafe or harmful outputs as an example, the calibration information generated synchronously by the target model can also represent the calibration information of the feature representation obtained at the current time step. The feature representation of the input data obtained at the current time step is calibrated based on the calibration information to obtain the calibrated feature representation. Further, through the task header of the target model, the calibrated feature representation is converted into an inference result, and by calibrating the inference result, the effect of retrieving unsafe or harmful outputs can be achieved.

[0072] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0073] Figure 4 This is a flowchart illustrating a model-based data processing method provided as an exemplary embodiment of this application. The execution entity in this embodiment is the first server in the aforementioned system architecture. Figure 4 As shown, the specific steps of this method are as follows:

[0074] Step S401: Input the input data into the target model. The target model is obtained by inserting extended parameters into the original parameters of the machine learning model. The extended parameters do not change the inference results of the original parameters of the machine learning model.

[0075] In this embodiment, the target model is obtained by inserting extended parameters into the original parameters of the original machine learning model. These extended parameters are obtained after training. The extended parameters are used to synchronously generate calibration information during the process of the original machine learning model inferring the feature representation of the input data. Furthermore, the addition of extended parameters does not affect the process of the original machine learning model inferring the feature representation of the input data. That is, the feature representation of the input data obtained by the target model is consistent with the feature representation obtained by the original machine learning model based on the same input data before extension.

[0076] The machine learning model can be any type of natural language processing model, including but not limited to various large language models (LLM), pre-trained language models, and Transformer-based natural language models.

[0077] Step S402: Obtain the feature representation of the input data through inference using the original parameters in the target model, and generate calibration information synchronously by extending the parameters.

[0078] In this step, the input data is fed into the target model, and the feature representation of the input data is obtained by reasoning through the original parameters in the target model. Furthermore, calibration information is generated synchronously through the inserted extended parameters.

[0079] Step S403: Based on the calibration information, calibrate the inference result generated based on the feature representation to obtain the calibration result.

[0080] After obtaining calibration information, the inference results generated from the feature representation of the input data are calibrated based on the calibration information to obtain calibration results, thereby achieving the goal of intervening in the inference of the model and improving the quality of the inference results.

[0081] In an alternative embodiment, extended parameters can be inserted into the original parameters of the embedding layer and the transformer block in the original machine learning model to extend the parameters of the embedding layer and the transformer block. These extended parameters are used to generate extended representations.

[0082] During inference, input data is fed into the target model. Inference using the target model simultaneously generates both the input feature representation and its extended representation. This extended representation serves as calibration information, used to calibrate the synchronously generated feature representation of the input data, resulting in a calibrated feature representation. Further, an output result is generated based on the calibrated feature representation, serving as the calibration result.

[0083] In another optional embodiment, extended parameters can be inserted into the original parameters of the embedding layer and transformer block in the original machine learning model to extend the parameters of the embedding layer and transformer block. These extended parameters are used to generate extended representations. At least one extension head is added to generate calibration information based on the extended representations. The extended parameters of the target model include the extended parameters inserted in the embedding layer and transformer block, as well as the parameters of the added extension head.

[0084] During inference, the input data is fed into the target model, and inference through the target model can simultaneously generate the feature representation and extended representation of the input; calibration information is generated based on the extended representation through the extended header; inference results are generated through the original task header of the machine learning model; and the inference results are calibrated based on the calibration information to obtain the calibration result.

[0085] For example, a machine learning model typically includes an embedding layer, multiple stacked transformer blocks, and a task head. The target model is obtained by inserting extended parameters into the original parameters of the machine learning model, including: concatenating embedded extended parameters to the original parameters of the embedding layer of the machine learning model to obtain an extended embedding layer; concatenating transformation extended parameters to the original parameters of each of the multiple stacked transformer blocks to obtain multiple stacked extended transformer blocks; and adding an extension head after the multiple stacked extended transformer blocks to obtain the target model. The parameters of the embedding extended parameters, transformation extended parameters, and extension head are obtained through training.

[0086] During inference, input data is fed into the extended embedding layer of the target model. The original parameters of the extended embedding layer map the input data to the original embedding vector, and the extended parameters map the input data to the incremental embedding vector. The complete output of the extended embedding layer in the target model (referred to here as the extended embedding vector) is equivalent to appending an incremental embedding vector to the original embedding vector output by the original embedding layer before expansion. The original embedding vector refers to the portion that maintains the same shape as the embedding vector obtained by mapping the same input data through the original embedding layer.

[0087] The extended embedding vector output from the extended embedding layer is input into a multi-layered stacked extended transformer block. This stacked block processes the extended embedding vector, with the final layer outputting a complete vector representation. This complete vector representation includes both the feature representation and the extended representation of the input data. Specifically, the feature representation of the input data is obtained by calculating the original embedding vector using the original parameters of the extended transformer block; and the extended representation is obtained by calculating the original and incremental embedding vectors using the transformed extension parameters. The complete vector representation output by the final extended transformer block in the target model is equivalent to concatenating the extended representation with the feature representation of the input data output by the final transformer layer in the original machine learning model. The feature representation of the input data output by the final extended transformer block in the target model is consistent with the feature representation of the same input data output by the final transformer layer in the original machine learning model.

[0088] The extended representation output of the last layer extended transformer block in the target model is input to the extended header, which maps the extended representation to calibration information.

[0089] In addition, if the feature representation of the input data output by the last layer of the extended transformer block in the target model is directly input into the task head of the target model (i.e., the task head of the original machine learning model), the feature representation can be mapped to the prediction result through the task head. The prediction result obtained in this way is consistent with the prediction result of the original machine learning model.

[0090] In this embodiment, calibration information can be generated synchronously without changing the feature representation of the input data through one inference of the target model.

[0091] In one example scenario, calibration information can be used as the calibration information of the feature representation of the input data. The process of mapping the extended representation to calibration information through the extended header can be represented as: Among them, H m H' represents the calibration information synchronously generated by the target model, and H′ represents the extended representation output by the last layer extended transformer block of the target model. The parameters of the extended head (which is a mapping matrix) are learnable during training.

[0092] In this step, the feature representation of the input data is calibrated based on the calibration information to obtain the calibrated feature representation; the calibrated feature representation is then converted into inference results through the task header of the target model to achieve the effect of calibrating the inference results and realize the goal of model inference calibration.

[0093] Optionally, when calibrating the feature representation of the input data based on the calibration information, the calibration information and the feature representation of the input data can be summed to obtain the calibrated feature representation.

[0094] For example, let H0 represent the feature representation of the input data generated by the target model, and let H... m This represents the calibration information generated synchronously by the target model. Therefore, based on this calibration information, the process of generating inference results from feature representations is calibrated, and the process of obtaining the inference results can be represented as: O otter =lm_head(H m +H0). Here, lm_head represents the task header of the target model, which is also the task header of the original machine learning model.

[0095] Optionally, weighting coefficients can be set for the calibration information and feature representation, respectively. The calibration information and the feature representation of the input data are then weighted and summed to obtain the calibrated feature representation. The degree of calibration of the feature representation by the calibration information can be adjusted by configuring the weighting coefficients. The weighting coefficients for the calibration information and feature representation can be configured and adjusted according to actual application requirements and empirical values; no specific limitations are imposed here.

[0096] In an example scenario, calibration information can represent the confidence level of the inference result generated based on the feature representation of the currently obtained input data without calibration. For example, it could represent the probability that the generated inference result does not meet requirements. The process of mapping the extended representation to calibration information via the extended header can be represented as follows: Among them, O′ otter H' represents the calibration information synchronously generated by the target model, and H′ represents the extended representation output by the last layer extended transformer block of the target model. The parameters of the extended head are learnable during training. σ(·) represents the Sigmoid function, which maps variables to the interval (0,1).

[0097] In this step, the calibration information is determined to meet the calibration conditions based on whether it reaches a preset probability threshold. If the calibration information is greater than or equal to the confidence threshold (i.e., the inference result generated from the feature representation of the currently obtained input data is highly likely to fail to meet the requirements), the calibration information is determined to meet the calibration conditions. The inference result generated from the feature representation is then calibrated based on the calibration information to intervene in the inference process of the target model and obtain a calibration result. Otherwise, if the calibration information is less than the confidence threshold (i.e., the inference result generated from the feature representation of the currently obtained input data is highly likely to meet the requirements), the calibration information is determined to fail to meet the calibration conditions. The feature representation of the input data is then converted into an inference result through the task header of the target model. The confidence threshold can be configured and adjusted according to actual application needs and empirical values; no specific limitations are imposed here.

[0098] Optionally, when calibrating the inference result generated from the feature representation based on the calibration information, the input data can be re-input into the target model, and the target model can be used to re-infer the new feature representation of the input data, and new calibration information can be generated simultaneously. Based on the new calibration information, the process of generating the inference result from the new feature representation is calibrated until the new calibration information obtained is less than the confidence threshold (i.e., the calibration conditions are not met and no calibration is required), and the latest obtained inference result is taken as the calibration result.

[0099] It should be noted that, in this embodiment, the method can be calibrated multiple times if the generated calibration information meets the calibration conditions, so as to repeat the reasoning and calibration process of steps S401-S403; until the new calibration information obtained does not meet the calibration conditions (i.e., no calibration is required), the feature representation of the input data is converted into the reasoning result through the task head of the target model to obtain the final calibration result.

[0100] Optionally, when calibrating the inference result generated from the feature representation based on the calibration information, if the calibration information is greater than or equal to the confidence threshold, at least one time step can be rolled back, and the inference process of at least one time step can be re-executed using the target model.

[0101] The method in this embodiment can be applied to each time step of model inference. When executing at any time step, if the calibration information generated at the current time step meets the calibration conditions (calibration information is greater than or equal to the confidence threshold), at least one time step can be rolled back (e.g., rolled back T time steps). The portion before T time steps is extracted from the generated token sequence and input into the target model. The target model then re-infers and generates tokens for subsequent time steps (i.e., the inference results of subsequent time steps) based on the token sequence before T time steps. The number of time steps rolled back, T, can be configured and adjusted according to actual application needs and empirical values, and is not specifically limited here.

[0102] In one example scenario, the goal of the calibration task could be to accelerate inference. At least one extension head can be added to the target model. The input data's feature representation and extended representation are based on the output of the last layer of the target model's extended transformer block. The next token is predicted using the target model's task head (i.e., the original machine learning model's task head) based on the input data's feature representation. Then, using at least one extension head of the target model, one to the next token are predicted based on the extended representation.

[0103] When calibrating the inference results generated from the feature representation based on the calibration information, the next token predicted by the task head and the next token predicted by at least one extended head are used together as the calibration result. Multiple tokens can be predicted in one inference, thereby achieving the goal of accelerating inference.

[0104] Optionally, for the prediction results of each extension head of the target model, the similarity between the (t+1)th token predicted by the task head of the target model and the prediction results of each extension head can be calculated. Prediction results with similarity greater than or equal to the similarity threshold are retained, while prediction results with similarity less than the similarity threshold are discarded. The similarity threshold can be configured and adjusted according to actual application requirements, and is not specifically limited here.

[0105] For example, assume the target model contains three extension heads. When predicting the (t+1)th token based on the first t tokens, the first t tokens are input into the target model for processing. The last layer of the target model's extension transformer block outputs a feature representation and an extension representation. The (t+1)th token is predicted based on the feature representation using the target model's task head (the original machine learning model's task head). Furthermore, by inputting the extension representations into the three extension heads respectively, the (t+2), (t+3), and (t+4)th tokens are predicted using the three extension heads. The subsequent four tokens can be obtained through a single inference.

[0106] Optionally, the (t+1), (t+2), (t+3), and (t+4)th tokens can be input together into the target model, which will then provide the probability of each token. Based on the probabilities of each token, the results are filtered, and tokens with probabilities higher than a preset threshold are used as the calibration results. For example, if the probabilities of the first three tokens (t+1, t+2, and t+3) are higher than the preset threshold, then the (t+1), (t+2), and (t+3)th tokens can be retained as the calibration results. The preset threshold can be configured and adjusted based on actual application needs and empirical values; no specific limitations are imposed here.

[0107] The method of this application obtains the target model by inserting extended parameters into the original parameters of the machine learning model, and the inserted extended parameters do not change the inference results of the original parameters of the machine learning model. During inference, the input data is input into the target model, and the feature representation of the input data is obtained through inference using the original parameters in the target model. Calibration information is generated synchronously through the extended parameters. Based on the calibration information, the inference results generated from the feature representation are calibrated to obtain the calibration result. The method of this application only needs to insert a small number of extended parameters into the original machine learning model. During the process of obtaining the feature representation of the input data through inference of the original machine learning model, calibration information can be generated synchronously through the extended parameters. The feature representation of the input data and calibration information can be obtained simultaneously in a single inference process, saving the time spent on obtaining calibration information and reducing time overhead, thereby reducing the impact of the calibration process on the timeliness of the model output response. Furthermore, it eliminates the need for an additional reward model with a large parameter scale, greatly reducing the space overhead of the calibration scheme. In addition, the calibration scheme can improve the quality of the model inference results, enabling the model to generate more accurate and secure inference results, while reducing latency and improving customer satisfaction.

[0108] To more clearly illustrate the processing flow of the target model, the process of obtaining the target model is explained in detail below. Figure 5A flowchart illustrating a method for obtaining a target model provided in an exemplary embodiment of this application. Figure 5 As shown, the target model used in the aforementioned embodiments can be obtained through the following steps:

[0109] Step S501: Construct a backup model of the machine learning model. The backup model has the same structure and parameters as the machine learning model.

[0110] Step S502: Insert embedded extension parameters into the embedding layer of the backup model, insert transformation extension parameters into the linear layer contained in the transformer block of the backup model, and add an extension head to obtain the extended model.

[0111] In this embodiment, to obtain the trained extended parameters, a backup model with a structure and parameters completely identical to the machine learning model is first constructed. Further, embedded extended parameters are inserted into the embedding layer of the backup model, transformed extended parameters are inserted into the linear layers contained in the transformer block of the backup model, and an extension head is added to obtain the extended model.

[0112] For example, embedding extension parameters are concatenated onto the original parameters of the embedding layer of the backup model to obtain an extended embedding layer; transformation extension parameters are concatenated onto the original parameters of each multi-layer stacked transformer block to obtain a multi-layer stacked extended transformer block; and an extension head is added after the multi-layer stacked extended transformer block to obtain an extended model. The embedding extension parameters, transformation extension parameters, and extension head parameters are trainable parameters and can be randomly initialized.

[0113] Specifically, additional extension parameters are inserted into the linear layers contained in the embedding layer and transformer block of the backup model (i.e., the backup model). This allows the extended model to synchronously generate extended representations for generating calibration information during the inference process of generating feature representations of the input data. Furthermore, an extension head is added to the backup model to map the extended representations to calibration information. This calibration information is used to calibrate the inference results generated by the task head based on the feature representations of the input data, thereby improving the quality of the generated inference results and achieving the goal of model inference intervention.

[0114] In one optional embodiment, embedding expansion parameters are inserted into the embedding layer of the backup model, expanding the dimension of the embedding vector output by the embedding layer from m to (m+k). Here, m is the dimension of the original embedding vector output by the embedding layer before expansion, and k is the dimension of the incremental embedding vector; both m and k are positive integers. This expansion of the parameters of the embedding layer of the backup model results in an expanded embedding vector output by the embedding layer, which is equivalent to appending an incremental embedding vector to the original embedding vector output by the embedding layer before expansion. The original embedding vector refers to the portion that maintains consistency with the embedding vector mapped from the same input data by the embedding layer before expansion.

[0115] For any linear layer contained in any transformer block of the backup model, an extended mapping matrix is ​​concatenated onto the linear mapping matrix of the linear layer, and an extended vector is concatenated onto the bias vector of the linear layer to obtain the corresponding extended linear layer. After expanding each linear layer contained in the transformer block of the backup model into the corresponding extended linear layer, the corresponding extended transformer block is obtained. The transformation extension parameters include the extended mapping matrix in the extended transformer block. The original parameters in the extended transformer block of the backup model are used to determine the feature representation based on the original embedding vector of the input data, and the transformation extension parameters are used to determine the extended representation based on the original embedding vector and the incremental embedding vector. Here, the parameters of the linear layers in each transformer block of the backup model are extended to obtain the extended transformer block. Multiple stacked extended transformer blocks process the extended embedding vector to obtain the complete vector representation output by the last extended transformer block. The complete vector representation output by the last extended transformer block includes the feature representation and the extended representation of the input data. Specifically, the complete vector representation output by the last extended transformer block in the extended model is equivalent to concatenating the extended representation onto the feature representation of the input data output by the last transformer block in the backup model. The feature representation of the input data output by the last layer of the extended converter block in the extended model is consistent with the feature representation of the same input data output by the last layer of the converter in the backup model.

[0116] An extension head is added to the backup model to generate calibration information based on the complete vector representation output by the last layer transformer block.

[0117] For example, Figure 6 This is a schematic diagram of the architecture of the extended model provided in this embodiment. For example... Figure 6 As shown, the input data is fed into the extended embedding layer of the extended model, and the extended embedding vector is obtained through mapping by the extended embedding layer, including the original embedding vector (such as...). Figure 6 h0) and incremental embedding vector (as shown in the figure) and incremental embedding vector (as shown in the figure) Figure 6 (as shown in h'0). The extended embedded vector input is processed by a multi-layer stacked extended transform block, and the last extended transform block outputs the complete vector representation, including the feature representation of the input data (such as h'0). Figure 6 H0 as shown) and extended representation (such as Figure 6 H′ shown in the figure. Figure 6 In this context, 's' represents the number of layers of the extended transformer blocks contained in the extended model, which is also the number of layers of stacked transformer blocks in the backup model. Further, the extension represents the mapping of the input extended head to obtain calibration information (as shown in the figure). This calibration information is used to calibrate the process by which the task head maps the feature representation of the input data to the inference result, thus obtaining the final inference result.

[0118] In practical applications, backup models typically contain multiple stacked transformer blocks. Each transformer block is parametrically expanded to obtain the corresponding expanded transformer block. In the expanded model, the input to the first layer of the expanded transformer block is the expanded embedding vector output by the expanded embedding layer, including the original embedding vector (e.g., ...). Figure 6 h0) and incremental embedding vector (as shown in the figure) and incremental embedding vector (as shown in the figure) Figure 6 (h'0 shown in the diagram). The complete hidden state vector output by the first layer extended transformer block includes the original hidden state vector (denoted by h1) and the incremental hidden state vector (denoted by h′1). The output of the previous layer extended transformer block serves as the input to the next layer extended transformer block. The output of the last layer extended transformer block is a complete vector representation, including the feature representation H0 and the extended representation H′ of the input data.

[0119] Typically, a transformer block includes a feed-forward network (FFN), a multi-head attention (MHA), and residual connections and normalization layers (Add & Norm). Both the FFN and MHA contain multiple linear layers. Extending each linear layer in the FFN yields an extended FFN layer. Extending each linear layer in the MHA yields an extended MHA layer. After extending all the linear layers in the transformer block, the extended transformer block is obtained.

[0120] Figure 7 This is a schematic diagram of an extended converter block provided for an exemplary embodiment of this application. Figure 7 As shown, taking the i-th layer extended transformer block as an example, using... This represents the input hidden state vector of the i-th layer extended transformer block, including the original hidden state vector h. i and the extended hidden state vector h′ i .use This represents the output hidden state vector of the i-th layer extended transformer block, including the original hidden state vector h. i+1 and the extended hidden state vector h′ i+1 Wherein, the original hidden state vector h i During inference with the backup model based on the same input data, the input to the i-th layer transformer block remains consistent. The original hidden state vector h... i+1 During the inference process based on the same input data as the backup model, the hidden state vector output by the i-th layer transformer block remains consistent.

[0121] Specifically, the hidden state vector output from the previous layer... (including the original hidden state vector h) i and the extended hidden state vector h′ iThe input is processed through the extended MHA of the i-th layer extended transformer block, and the output is the hidden state vector. (including the original hidden state vector h) mha and the extended hidden state vector h′ mha Furthermore, after Add&Norm is applied to the extended MHA output... and After summing, normalization is performed, and the hidden state vector is output. (including the original hidden state vector h) an and the extended hidden state vector h′ an Furthermore, the output of Add&Norm will be... The input is an extended FFN, which is processed to output a hidden state vector. (including the original hidden state vector h) ffn and the extended hidden state vector h′ ffn Furthermore, after the next Add&Norm operation, the extended FFN output is... and After summing and normalizing, the hidden state vector output by the i-th layer extended transformer block is obtained. (including the original hidden state vector h) i+1 and the extended hidden state vector h′ i+1 ).

[0122] Specifically, the transformation extension parameters are inserted into any linear layer contained in the transformer block of the backup model, which can be achieved in the following way:

[0123] For any linear layer contained in the transformer block of the backup model, an extended mapping matrix is ​​concatenated on the linear mapping matrix of the linear layer, and an extended vector is concatenated on the bias vector of the linear layer to obtain the corresponding extended linear layer.

[0124] Specifically, when concatenating the extended mapping matrix onto the linear mapping matrix of the linear layer, a zero matrix of dimension (k,n) is concatenated onto the linear mapping matrix of the linear layer, expanding the dimension of the linear mapping matrix from (m,n) to (m+k,n), resulting in a (m+k,n)-dimensional mapping matrix to align the dimensions of the linear mapping matrix of the linear layer with those of the extended mapping matrix. Here, n is a positive integer. Further, an extended mapping matrix of dimension (m+k,k) is concatenated onto the (m+k,n)-dimensional mapping matrix to obtain the extended linear mapping matrix of the linear layer. The transformation and expansion parameters include the extended mapping matrix.

[0125] For example, Figure 8 This is an extended schematic diagram of the linear mapping matrix provided in this embodiment. For example... Figure 8As shown, the process of expanding the linear mapping matrix for each linear layer is as follows: Let W represent the linear mapping matrix before expansion, with dimensions (m,n). First, a zero matrix of dimension (k,n) is concatenated below the linear mapping matrix W of dimension (m,n) to obtain a mapping matrix of dimension (m+k,n). Then, an expanded mapping matrix of dimension (m+k,k) is concatenated to the right of the mapping matrix of dimension (m+k,n). Figure 8 In W′), we obtain the extended linear mapping matrix (which can be represented as W′). The extended mapping matrix is ​​learnable.

[0126] like Figure 8 As shown, using This represents the hidden state vector of the input extended linear layer, including the original hidden state vector h. in and the extended hidden state vector h′ in , can be represented as Through the extended linear mapping matrix The hidden state vector of the input Perform linear mapping (i.e., calculate) ) obtained Equivalent to h out =W×h in and The concatenation. The inserted extended mapping matrix W′ does not affect the original hidden state vector h after the linear mapping. out The mapping result of the expanded linear mapping matrix. This is equivalent to the mapping result h of the linear mapping matrix before expansion. out The extended hidden state vector h′ is concatenated above. out This extension method preserves the value from h. in to h out The linear mapping remains unchanged, and a new one is created from to h′ out The linear mapping remains unchanged after the expansion; the only difference is that the dimension of the hidden state vector increases before and after the mapping.

[0127] For example, for MHA in the converter block, the MHA processing procedure can be represented as follows:

[0128] Q = W Q k i K = W K h i V = W V h i ;

[0129] head j =Attn(qj ,k j ,v j );q j ,k j ,v j It is the part that is split into Q, K, and V and then input into the j-th attention head;

[0130] h mha =Concat(head1,head2,…)W O .

[0131] Among them, W Q W K W V and W O These are the linear mapping matrices of the linear layers contained in the MHA. Attn(·) represents the attention calculation process. Concat(·) represents the concatenation operation. h i h is the hidden state vector input to MHA. mha It is the hidden state vector output by MHA. head1, head2, ... represent the multiple attention heads included in MHA.

[0132] In this embodiment, the four linear mapping matrices W contained in the MHA of the transformer block of the backup model are... Q W K W V and W O Perform as follows Figure 8 The extension shown involves adding a learnable extended mapping matrix, resulting in extended MHA. This extension only increases the size of the hidden state vectors without modifying the MHA architecture, thus achieving a non-intrusive and efficient attention implementation.

[0133] Optionally, for a linear layer using a bias vector, the dimension of the bias vector is the same as the dimension of the hidden state vector output by the linear layer, both being m. An expansion vector of dimension k is concatenated onto the m-dimensional bias vector to obtain the expanded bias vector of the linear layer, which is referred to as the expansion vector in this embodiment. The transformation expansion parameters include the expansion vector.

[0134] For example, an FFN in a converter block typically includes three linear layers. The processing of the FFN can be represented as: h g =W g h an +b g h u =W u h an +b u h ffn =W d (σ(h g )*hu )+b d Where σ(·) is the sigmoid function. an h is the hidden state vector of the input FFN. ffn W is the hidden state vector output by FFN. g W u and W d It is the linear mapping matrix contained in the three linear layers of FFN. b g b u and b d It is the bias vector contained in the three linear layers of FFN.

[0135] In this embodiment, the linear mapping matrix W contained in the three linear layers of each FFN in the backup model is... g W u and W d Perform as follows Figure 8 The extension shown adds a learnable extended mapping matrix; and the extended FFN is obtained by adding a learnable extended vector to the bias vectors contained in the three linear layers of each FFN.

[0136] Furthermore, the complete hidden state vector output by the extended linear layer includes: the original hidden state vector of dimension m and the extended hidden vector of dimension k. For the normalization layer in the Add&Norm connected to the extended MHA and extended FFN (i.e., connected to the last extended linear layer in MHA and FFN), in order to avoid the normalization operation destroying the original hidden state vector, the normalization layer connected to each extended linear layer is adjusted. The adjusted normalization layer uses the mean and / or variance of the original hidden state vector output by the extended linear layer to normalize the complete hidden state vector output by the extended linear layer.

[0137] For example, consider a normalization layer following the extended FFN, where the normalization layer uses root mean square normalization (RMSNorm). In the backup model before expansion, the input to the normalization layer following the FFN is the original hidden state vector h output by the FFN. ffn With the original hidden state vector h of the input FFN an The sum, denoted as h norm-in The normalization process can be represented as follows: Here, mean(·) represents the operation of calculating the average. γ is the scaling factor. ∈ is a small constant to avoid zero denominators, for example, ∈ = 0.00001. It can be configured and adjusted according to actual application needs and empirical values, and no specific limitation is made here.

[0138] The input to the normalization layer in the extended model is the output hidden state vector of the extended FFN layer. With the hidden state vector of the input extended FFN layer The sum of these is denoted as Without adjusting the normalization layer, the normalization operation is performed as follows: After adjusting the normalization layer, the processing procedure for the adjusted normalization operation is as follows: The denominator uses the original hidden state vector h from the input normalized layer. norm-in The root mean square value.

[0139] Additionally, in some example scenarios, the backup model's normalization layer normalizes the input hidden state vector using the mean of the input hidden state vector. The normalization layer is then adjusted to normalize the input hidden state vector using the mean of the original hidden state vector within the input hidden state vector.

[0140] In some example scenarios, the normalization layer of the backup model normalizes the input hidden state vector using the variance of the input hidden state vector. The normalization layer is then adjusted to normalize the input hidden state vector using the variance of the original hidden state vector within the input hidden state vector.

[0141] In another optional embodiment, a new embedding layer can be added to the backup model in this step. This new embedding layer maps the input data to an incremental embedding vector of dimension k. The original embedding layer still maps the input data to an original embedding vector of dimension m. Transformation extension parameters are inserted into the linear layers contained in the transformer block of the backup model. The original parameters in the extended transformer block are used to determine the feature representation based on the original embedding vector of the input data, and the transformation extension parameters are used to determine the extended representation based on the original embedding vector and the incremental embedding vector. The original embedding vector of dimension m obtained from the original embedding layer is concatenated with the incremental embedding vector of dimension k obtained from the new embedding layer, and this concatenation is used as the input to the subsequent extended transformer block. An extension head is added to the backup model. The extension head is used to generate calibration information based on the complete vector representation output by the last extended transformer block. The processing of the extended transformer block and the extension head is described in the foregoing related content and will not be repeated here.

[0142] By extending the backup model as described above, the extended model is obtained.

[0143] Step S503: Using the dataset, adjust the parameters of the embedding extension parameters, transformation extension parameters, and extension head in the extended model to obtain the trained embedding extension parameters, transformation extension parameters, and extension head.

[0144] In different application scenarios, the original machine learning model has different defects, so the goals of the calibration task (i.e., inference intervention) will also be different. The method in this embodiment can be applied to a variety of different calibration tasks, including but not limited to: reducing redundant generation, reducing unsafe or harmful responses, aligning beneficial and harmless preferences, and accelerating inference.

[0145] Repeated generation refers to a model's tendency to endlessly copy the input text or excessively repeat the same sentences or phrases when generating text. This phenomenon results in a lack of diversity and creativity in the model's output, leading to a poor user experience. Reducing repeated generation aims to minimize or avoid this behavior.

[0146] Unsafe or harmful responses refer to content that a model may generate that has a negative impact when producing text. Reducing harmful responses aims to minimize the harmful output of the model. For example, a model may generate incorrect information or misleading statements, especially when dealing with complex or specialized problems (such as mathematical reasoning). Additionally, malicious users may exploit the model for nefarious activities. These actions can not only damage the reputation of individuals and organizations but also pose a threat to society as a whole.

[0147] Beneficial-harmless alignment refers to ensuring that the model generates text that aligns with human expectations and values, guaranteeing that the output is both useful and harmless. Accelerated inference refers to improving the model's inference speed.

[0148] In this embodiment, in order to achieve the corresponding calibration task, it is necessary to obtain the dataset corresponding to the calibration task, which is used to train the extended model and obtain the extended parameters for generating the target model with the corresponding calibration task capability.

[0149] For example, taking the calibration task's objective of reducing duplicate generation as an example, the corresponding dataset includes: query information and the response results of the query information. The response results of the query information are a sequence of output text tokens with annotation information, including the label of each text token, indicating whether duplicate generation occurs at the text token location.

[0150] For example, taking the calibration task's objective as beneficial and harmless preference alignment, the corresponding dataset includes samples such as query information, positive responses to the query information, and negative responses. Positive responses refer to those that meet the requirements of beneficial and harmless preference alignment. Negative responses refer to those that do not meet the requirements of beneficial and harmless preference alignment.

[0151] For example, taking the calibration task with the goal of inference acceleration as an example, the samples in the corresponding dataset include: query information and the response results of the query information.

[0152] In this embodiment, the machine learning model to be calibrated refers to the model that requires inference intervention. Specifically, it can be various natural language processing models, including but not limited to various large language models (LLM), pre-trained language models, and Transformer-based natural language models. The machine learning model can specifically be a text generation model, an intelligent dialogue model, etc., without specific limitations here.

[0153] After obtaining the dataset and extended model corresponding to the calibration task, the extended model is trained using the dataset. During training, the original parameters of the backup model in the extended model are fixed, while the extended parameters in the extended model are adjusted, including the embedding extension parameters, the transformation extension parameters, and the extension head. After training is complete, the trained embedding extension parameters, transformation extension parameters, and extension head are obtained.

[0154] Step S504: Concatenate the embedded extension parameters to the original parameters of the embedding layer of the machine learning model to obtain the extended embedding layer; concatenate the transformation extension parameters to the original parameters of each multi-layer stacked transformer block to obtain the multi-layer stacked extended transformer block; add an extension head after the multi-layer stacked extended transformer block to obtain the target model.

[0155] After obtaining the extended parameters after training (embedding extended parameters, transforming extended parameters, and extending heads), the extended parameters are inserted into the original parameters of the machine learning model to obtain the target model, thus realizing hot upgrade / hot repair of the machine learning model.

[0156] It should be noted that the principle of inserting the extended parameters into the machine learning model after obtaining the extended parameters (including embedded extended parameters, transformed extended parameters, and extended heads) is consistent with the specific implementation principle of inserting embedded extended parameters into the embedding layer of the backup model, inserting transformed extended parameters into the linear layer contained in the transformer block of the backup model, and adding an extended head to obtain the extended model in the aforementioned step S502. It will not be repeated here.

[0157] This embodiment constructs a backup model with the same structure and parameters as the machine learning model. Embedding extension parameters are inserted into the embedding layer of the backup model, transformation extension parameters are inserted into the linear layers contained in the transformer block of the backup model, and an extension head is added to obtain an extended model. Using a dataset, the parameters of the embedding extension parameters, transformation extension parameters, and extension head in the extended model are adjusted to obtain trained embedding extension parameters, transformation extension parameters, and extension head. Inserting these trained embedding extension parameters, transformation extension parameters, and extension head into the machine learning model yields a target model with calibration capabilities (i.e., inference intervention capabilities), achieving a hot upgrade of the original machine learning model without affecting the normal operation of the inference task. The obtained target model can synchronously generate calibration information during the process of obtaining the feature representation of the input data during inference. The feature representation and calibration information of the input data can be obtained simultaneously in a single inference process, saving time spent acquiring calibration information and reducing time overhead, thereby reducing the impact of inference calibration on the timeliness of the model's output response. Furthermore, only a small number of parameters need to be inserted into the original model and fine-tuned, eliminating the need for an additional reward model and significantly reducing the space overhead of inference calibration.

[0158] Figure 9 A flowchart illustrating a model training method provided for an exemplary embodiment of this application. In an optional embodiment, as shown... Figure 9 As shown, step S503 above uses the dataset to adjust the parameters of the embedded extension parameters, transformed extension parameters, and extension head in the extended model. Specifically, this can be achieved through the following steps:

[0159] Step S901: Input the sample data in the dataset into the extended embedding layer of the extended model for mapping to obtain the extended embedding vector. The extended embedding vector includes the original embedding vector and the incremental embedding vector of the sample data.

[0160] In the extended model, the extended embedding vector output by the extended embedding layer is equivalent to appending an incremental embedding vector to the original embedding vector output by the embedding layer before extension. The original embedding vector refers to the portion that maintains consistency with the embedding vector obtained by mapping the same sample data through the embedding layer before extension.

[0161] Step S902: The extended embedding vector is processed by the extended transformer block to obtain the complete vector representation, which includes the feature representation and extended representation of the sample data.

[0162] In this extended model, the complete vector representation output by the last layer of the extended transformer block is equivalent to concatenating the extended representation onto the feature representation of the sample data output by the last layer of the transformer in the original machine learning model. The feature representation of the sample data output by the last layer of the extended transformer block in the extended model remains consistent with the feature representation of the same sample data output by the last layer of the transformer in the original machine learning model.

[0163] Step S903: Generate calibration information based on the extended representation using the extended head of the extended model.

[0164] The extended representation output of the last layer transformer block (after extension) in the extended model is input to the extension header, which maps the extended representation to calibration information.

[0165] Step S904: Generate inference prediction results based on the feature representation of the sample data by extending the task head of the model.

[0166] Step S905: Calculate the calibration task loss based on the inference prediction results and / or calibration information.

[0167] In practical applications, the methods for calculating calibration task losses can differ depending on the target of the calibration task.

[0168] In an example scenario, taking the calibration task's objective of reducing duplicate generation as an example, the samples in the corresponding dataset include: query information and the response results of the query information. The response results of the query information are a sequence of output text tokens with annotation information, including a truth value label for each text token, indicating whether a duplicate generation occurred at that text token.

[0169] The process of training the extended model includes:

[0170] The query information is input into the extended model for inference, resulting in the feature representation and extended representation of the query information output by the last layer extended transformer block. The task head of the extended model generates an inference prediction result based on the feature representation of the query information. Finally, the extended representation is mapped to calibration information through the extended model's extension head. This inference prediction result includes an output text tag sequence. The calibration information includes the predicted label corresponding to each text tag in the inference prediction result. The predicted label represents the probability of repeated generation at that text tag.

[0171] Furthermore, the calibration task loss is calculated using the following formula (1) based on the inference prediction results and the response results of the query information in the dataset:

[0172]

[0173] Among them, L taskThis represents the calibration task loss. r represents the number of text tags included in the query response. i This represents the predicted label of the i-th text tag in the response result of the query information, that is, the probability that the i-th text tag in the response result of the query information will be generated repeatedly. The truth label of the i-th text tag in the response results of the query information.

[0174] Furthermore, in scenarios where the goal of the calibration task is to reduce repeated generation, the calibration task loss function can be modified based on formula (1), for example, for... Adding logarithmic or exponential operations, etc., can be configured and adjusted according to actual application needs, and no specific limitations are made here.

[0175] In an example scenario, taking the calibration task's objective as beneficial and harmless preference alignment, the corresponding dataset includes samples such as: query information (denoted by x), and positive responses to the query information (denoted by y). w (represented by) and negative response (using y) l (This is indicated by the symbol). A positive response refers to a response that meets the requirement of aligning preferences for benefits without harm. A negative response refers to a response that does not meet the requirement of aligning preferences for benefits without harm.

[0176] The process of training the extended model includes:

[0177] Query information x and positive response y w After concatenation, the input to the extended model is used for inference, resulting in the first feature representation and the first extended representation output by the last layer extended transformer block. The first extended representation is then mapped to the first calibration information through the extended head of the extended model, denoted as r[x,y]. w The query information x and the negative response y are compared. l After concatenation, the input to the extended model is used for inference, resulting in the second feature representation and the second extended representation output from the last layer of the extended transformer block. The second extended representation is then mapped to the second calibration information through the extended head of the extended model, denoted as r[x,y]. l ].

[0178] Furthermore, using the following formula (2), based on the obtained r[x,y] w ] and r[x,y l ] Calculate the calibration task loss:

[0179] L task =-logσ(r[x,y) w ]-r[x,y l (2)

[0180] Among them, L taskThis represents the calibration task loss. σ(·) is the sigmoid function.

[0181] Furthermore, in scenarios where the goal of the calibration task is to align preferences in a beneficial and harmless manner, the calibration task loss function can be modified based on formula (2), for example, for r[x,y w ]-r[x,y l Adding exponential operations or multiplying by a scaling factor, etc., can be configured and adjusted according to actual application needs, and no specific limitations are made here.

[0182] In an example scenario, taking the calibration task as an example of inference acceleration, the corresponding dataset includes: query information and the response results of the query information.

[0183] The process of training the extended model includes:

[0184] The query information is input into the extended model for inference, resulting in the feature representation and extended representation of the query information output by the last layer of the extended transformer block. The task head of the extended model generates the inference prediction result based on the feature representation of the query information, i.e., the prediction distribution of the next token. The extended representation is mapped to calibration information through at least one extended head of the extended model to obtain the prediction distribution of at least one token after the next token.

[0185] Furthermore, the calibration task loss is calculated using the following formula (3) based on the obtained inference prediction results and calibration information:

[0186]

[0187] Among them, L task This represents the calibration task loss. K represents the number of extension heads. λ k This represents the weight coefficient corresponding to the k-th extension header. Different extension headers may have different weight coefficients. The specific configuration and adjustment can be made according to actual application needs and experience values. No specific restrictions are made here. This means that the yth token is predicted from the first t tokens in the response result based on the query information, using the kth extended header. t+k+1 The probability of each token.

[0188] Furthermore, in scenarios where the goal of the calibration task is to accelerate inference, the calibration task loss function can be modified based on formula (3), for example, for... Adding exponential operations or multiplying by a scaling factor can be configured and adjusted according to actual application needs, and no specific limitations are made here.

[0189] Step S906: Based on the calibration task loss, adjust the embedded extension parameters, transformation extension parameters, and extension head parameters in the extended model.

[0190] After calculating the calibration task loss, the extended parameters in the extended model (including the embedding extended parameters, transformation extended parameters, and the parameters of the extended head) can be adjusted through backpropagation based on the calibration task loss to train the extended model. During the training of the extended model, the parameters of the original machine learning model are fixed, and only the inserted extended parameters are adjusted. After training is completed, the trained embedding extended parameters, transformation extended parameters, and extended head are obtained.

[0191] In one optional embodiment, the regularization loss is calculated based on the complete hidden state vector and the original hidden state vector output by each layer of the extended transformer block of the extended model; the extended parameters in the extended model are adjusted based on the regularization loss.

[0192] For example, the regularization loss can be calculated using the following formula (4), based on the root mean square difference between the complete hidden state vector output by each layer of the extended transformer block of the extended model and the original hidden state vector:

[0193]

[0194] Among them, L reg This represents the regularization loss. S represents the number of extended transformer blocks in the extended model. This represents the complete hidden state vector output by the i-th layer extended transformer block, including the original hidden state vector h. i+1 and the extended hidden state vector h′ i+1 The `mean(·)` operator calculates the average value. ∈ is a small constant, for example, ∈ = 0.00001, which can be configured and adjusted according to actual application needs and empirical values; no specific limitation is made here.

[0195] Furthermore, when calculating the regularization loss, the regularization loss function used can be modified based on formula (4), for example, by replacing the root mean square value in formula (4) with the mean. Replace with mean(h) i+1 ),Will Replace with Alternatively, the root mean square value in formula (4) can be replaced with the variance. Replace with h i+1 The variance, will Replace with The variance. The regularization loss function can be configured and adjusted according to the actual application requirements, and no specific limitations are made here.

[0196] Furthermore, based on the regularization loss, the extended parameters in the extended model (including the parameters of the embedded extended parameters, the transformed extended parameters, and the parameters of the extended head) are adjusted through backpropagation to achieve the training of the extended model.

[0197] Optionally, the calibration task loss and regularization loss can be weighted and summed to obtain a comprehensive loss. Based on the comprehensive loss, the expansion parameters in the extended model (including the embedded expansion parameters, the transformed expansion parameters, and the parameters of the expansion head) can be adjusted. The weighting coefficients of the calibration task loss and regularization loss can be configured and adjusted according to actual application needs and experience, and are not specifically limited here.

[0198] Furthermore, after obtaining the target model, the system notifies the insertion of post-trained embedding extension parameters, transformation extension parameters, and extension heads into the currently used machine learning model, thus hot-fixing the original machine learning model into the target model and achieving hot upgrade of the machine learning model. This hot-fix / hot-upgrade process does not require large-scale full-parameter training and is much faster than normal model iteration.

[0199] The method in this embodiment extends and trains the original machine learning model through a non-intrusive parameter insertion approach, preserving the feature representations obtained from model inference and ensuring the integrity and stability of the original model output. When inference intervention is required, calibration information is generated synchronously through the extended parameters and the extended head to calibrate the inference results generated based on the feature representations, thereby enabling model inference intervention, improving model performance, and significantly reducing additional space and time overhead. When inference intervention is not required, the inserted extended parameters are removed to restore the original machine learning model. This plug-and-play approach is particularly suitable for application scenarios requiring rapid model iteration and deployment, while maintaining compatibility with existing model architectures without requiring adjustments or retraining of the original machine learning model. The simple parameter insertion enables model functionality extension, seamlessly integrating with existing models and simplifying the deployment process. The parameter insertion method is highly versatile, applicable to various NLP tasks and domains, without requiring customization for specific applications.

[0200] This application provides a method for accelerating model inference, comprising: inputting input data into a target model, performing inference through multiple time steps via iteration of the target model, and generating target text; wherein the target model is obtained by inserting extended parameters into the original parameters of the text generation model, and the extended parameters do not change the inference result of the original parameters of the text generation model.

[0201] The reasoning process at each time step includes: inputting the input data and the generated text tags into the target model, obtaining the input feature representation through reasoning using the original parameters of the target model, and synchronously generating the extended feature representation through the extended parameters; generating the next text tag based on the input feature representation, and generating at least one text tag after the next text tag based on the extended feature representation.

[0202] In this process, at least one text tag following the next text tag generated based on the extended feature representation is used as calibration information. The next text tag generated based on the input feature representation is used as the original inference result. The original inference result and the calibration information are concatenated to form the calibration result. Compared to the original inference result, the calibration result contains more prediction results, which can accelerate the model's inference.

[0203] In practical applications, text generation models consist of an embedding layer, multiple stacked transformer blocks, and a text generation head. The target model is obtained by inserting extended parameters into the original parameters of the text generation model. The specific implementation is as follows:

[0204] Embedded extension parameters are concatenated onto the original parameters of the embedding layer of the text generation model to obtain an extended embedding layer; transform extension parameters are concatenated onto the original parameters of each multi-layer stacked transform block to obtain a multi-layer stacked extended transform block; at least one extension head is added after the multi-layer stacked extended transform block to obtain the target model.

[0205] The parameters of the embedded extension parameters, the transformed extension parameters, and at least one extension head are obtained through training. The target model is obtained by inserting extension parameters into the original parameters of the text generation model, similar to the principle of inserting extension parameters into the machine learning model (or backup model) in the previous embodiments. For details, please refer to the relevant content in the previous embodiments; this embodiment will not repeat it here.

[0206] For example, the input feature representation is obtained through reasoning using the original parameters of the target model, and the extended feature representation is generated synchronously through the extended parameters. Specifically, this can be achieved as follows: Input data is input into the extended embedding layer of the target model. The original parameters of the extended embedding layer are used to map the input data into the original embedding vector, and the extended parameters are used to map the input data into the incremental embedding vector. The original embedding vector and the incremental embedding vector are input into a multi-layer stacked extended transformer block. The original parameters of the extended transformer block are used to calculate the original embedding vector to obtain the input feature representation. The extended parameters are then used to calculate the original embedding vector and the incremental embedding vector to obtain the extended feature representation.

[0207] In this embodiment, the implementation principle of obtaining the input feature representation through reasoning using the original parameters of the target model and synchronously generating the extended feature representation through extended parameters is the same as that in the previous embodiment, which describes the implementation principle of obtaining the feature representation of the input data through reasoning using the original parameters of the target model and synchronously generating the extended representation through extended parameters. It will not be repeated here.

[0208] Furthermore, the input feature representation is input into the text generation head, and the text generation head predicts the next text token based on the input feature representation. An extended representation is input into at least one extended head, and at least one text token following the next text token is predicted based on the extended representation using the at least one extended head.

[0209] In this embodiment, the extension header can adopt the same structure as the text generation header, and the initial parameters of the extension header can be obtained through random initialization.

[0210] In this embodiment, the extended parameters inserted into the text generation model (including the trained embedding extended parameters, transformation extended parameters, and parameters of at least one extended head) are used to obtain the specific implementation principle of the target model, as described above. Figure 5 The relevant details of the corresponding embodiments will not be repeated here.

[0211] The method in this embodiment obtains a target model by inserting extended parameters into the original parameters of the text generation model, and the inserted extended parameters do not change the inference results of the original parameters of the text generation model. During text generation, input data is fed into the target model, and the target model iterates through multiple time steps to generate the target text. In each time step, the input data and the already generated text tags are input into the target model. The input feature representation is obtained through inference using the original parameters of the target model, and an extended feature representation is generated synchronously using the extended parameters. The next text tag is generated based on the input feature representation, and at least one subsequent text tag is generated based on the extended feature representation. Multiple text tags can be generated simultaneously in one time step, accelerating model inference and improving the speed of text generation.

[0212] The model-based data processing solution provided in this application can be applied to the field of natural language processing. Figure 10 A flowchart illustrating a model-based data processing method provided as another exemplary embodiment of this application. (See attached flowchart.) Figure 10 As shown, when applied to the field of natural language processing, the specific steps of this method are as follows:

[0213] Step S1001: In response to the natural language processing task request, obtain input data.

[0214] Natural language processing tasks include, but are not limited to, text generation, text summarization, machine translation, and natural language understanding. They can be applied to scenarios such as intelligent search, intelligent customer service, copywriting generation, intelligent recommendation, and multilingual intelligent translation, without specific limitations here.

[0215] When applied to different natural language processing tasks, the input data can be different. The specific configuration and adjustment can be made according to the actual application requirements, and no specific limitations are made here.

[0216] Step S1002: Input the input data into the target model. The target model is obtained by inserting extended parameters into the original parameters of the natural language processing model. The extended parameters do not change the inference results of the original parameters of the natural language processing model.

[0217] The natural language processing model includes an embedding layer, multi-layered stacked transformer blocks, and a task head. The target model is obtained by inserting extended parameters into the original parameters of the natural language processing model, including:

[0218] An extended embedding layer is obtained by concatenating embedding extension parameters to the original parameters of the embedding layer of the natural language processing model; a multi-layered stacked extended transformer block is obtained by concatenating transformation extension parameters to the original parameters of each transformer block; and an extension head is added after the multi-layered stacked extended transformer block to obtain the target model. The parameters of the embedding extension parameters, transformation extension parameters, and extension head are obtained through training.

[0219] In this embodiment, the extended parameters inserted into the natural language processing model (including the trained embedding extended parameters, transformation extended parameters, and parameters of the extended head) are used to obtain the specific implementation principle of the target model, as described above. Figure 5 The relevant details of the corresponding embodiments will not be repeated here.

[0220] Step S1003: Infer the feature representation of the input data by using the original parameters in the target model, and generate calibration information synchronously by extending the parameters.

[0221] Step S1004: Based on the calibration information, calibrate the natural language processing result generated based on the feature representation to obtain the calibrated natural language processing result.

[0222] The natural language processing model in this embodiment is the original machine learning model. For the specific implementation principle of this embodiment, please refer to the content of the foregoing embodiment, which will not be repeated here.

[0223] The method in this embodiment obtains the target model by inserting extended parameters into the original parameters of the natural language processing model, and the inserted extended parameters do not change the inference results of the original parameters of the natural language processing model. During inference, input data is input into the target model, and the feature representation of the input data is obtained through inference using the original parameters in the target model. Calibration information is generated synchronously through the extended parameters. Based on the calibration information, the inference results generated from the feature representation are calibrated to obtain the calibration result. The method of this application only needs to insert a small number of extended parameters into the original natural language processing model. During the process of obtaining the feature representation of the input data through inference in the original natural language processing model, calibration information can be generated synchronously through the extended parameters. The feature representation of the input data and calibration information can be obtained simultaneously in one inference process, saving the time spent obtaining calibration information, reducing time overhead, and thus reducing the impact of the calibration process on the timeliness of the model output response. Furthermore, it eliminates the need for an additional reward model with a large parameter scale, reducing the space overhead of the calibration scheme. In addition, the calibration scheme can improve the quality of the model inference results, enabling the model to generate more accurate and secure inference results, while reducing latency and improving customer satisfaction.

[0224] The following example illustrates the method and process of reasoning calibration for large language models, using a scenario where a large language model (LLM) generates response results based on query information.

[0225] Figure 11 A flowchart illustrating a model-based data processing method provided as another exemplary embodiment of this application. (See attached flowchart.) Figure 11 As shown, the specific steps of this method are as follows:

[0226] Step S1101: Receive query information sent by the receiving end device.

[0227] Step S1102: Input the query information into the target language model. The target language model is obtained by inserting extended parameters into the original parameters of the pre-trained language model, and the extended parameters do not change the inference results of the original parameters of the pre-trained language model.

[0228] The target language model is obtained by inserting extended parameters into the original pre-trained language model (LLM). These extended parameters are used to simultaneously generate calibration information during the process of the original language model inferring the feature representation of the query information. Furthermore, the addition of extended parameters does not affect the process of the original pre-trained language model inferring the feature representation of the query information. In other words, the feature representation of the query information obtained by the extended target model is consistent with the feature representation obtained by the original pre-trained language model based on the same query information.

[0229] Step S1103: Infer the feature representation of the query information by using the original parameters in the target language model, and generate calibration information synchronously by extending the parameters.

[0230] The implementation principle of this step is the same as that of the aforementioned step S402. For the specific implementation principle and technical effects, please refer to the relevant content of the aforementioned embodiments, which will not be repeated here.

[0231] Step S1104: Based on the calibration information, calibrate the response result generated from the feature representation based on the query information to obtain the calibrated response result.

[0232] The implementation principle of this step is the same as that of step S403 mentioned above. For the specific implementation principle and technical effects, please refer to the relevant content of the aforementioned embodiments, which will not be repeated here.

[0233] Step S1105: Output the calibrated response result.

[0234] After generating the response results of the query information through inference calibration, the response results of the query information are returned to the end device.

[0235] The method in this application obtains the target language model by inserting extended parameters into the original parameters of the pre-trained language model, without altering the inference results of the original parameters. During inference, calibration information is simultaneously generated by the target language model as it obtains the feature representation of the query information. This allows for the simultaneous acquisition of both the feature representation and calibration information in a single inference process, saving time spent acquiring calibration information and reducing time overhead, thereby minimizing the impact of the calibration process on the timeliness of the model's output response. Furthermore, only a small number of parameters need to be inserted into the original pre-trained language model, eliminating the need for an additional reward model and reducing the space overhead of the solution. In addition, the calibration scheme improves the quality of the response results generated by the large language model, enabling it to generate more accurate and secure responses while reducing latency and increasing customer satisfaction.

[0236] Figure 12 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Figure 12 As shown, the server includes a memory 1201 and a processor 1202. The memory 1201 stores computer-executable instructions and can be configured to store various other data to support operations on the server. The processor 1202 is communicatively connected to the memory 1201 and executes the computer-executable instructions stored in the memory 1201 to implement the technical solutions provided in any of the above method embodiments. Their specific functions and the technical effects they achieve are similar and will not be repeated here.

[0237] Optional, such as Figure 12As shown, the server also includes other components such as a firewall 1203, a load balancer 1204, a communication component 1205, and a power supply component 1206. Figure 12 The diagram only shows some components and does not mean that the server only includes... Figure 12 The components shown. Figure 12 This example uses a cloud server deployed in the cloud as an example, but the server can also be deployed locally. This embodiment does not make any specific limitations here.

[0238] This application also provides a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the method of any of the foregoing embodiments. The specific functions and technical effects to be achieved are not described here.

[0239] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method of any of the foregoing embodiments. The computer program is stored in a readable storage medium, and at least one processor of the server can read the computer program from the readable storage medium. The execution of the computer program by the at least one processor causes the server to perform the technical solution provided in any of the above method embodiments. The specific functions and the technical effects that can be achieved are not described here.

[0240] This application provides a chip, including a processing module and a communication interface. The processing module is capable of executing the technical solution of the server in the aforementioned method embodiments. Optionally, the chip further includes a storage module (e.g., a memory), which stores instructions. The processing module executes the instructions stored in the storage module, and the execution of the instructions stored in the storage module causes the processing module to execute the technical solution provided in any of the aforementioned method embodiments.

[0241] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0242] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules from at least one processor.

[0243] The memory may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk drive, and may also be a USB flash drive, external hard drive, read-only memory, disk, or optical disc. The aforementioned memory can be object storage service (OSS). The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk, or optical disc.

[0244] The aforementioned communication components are configured to facilitate wired or wireless communication between the device containing the communication components and other devices. The device containing the communication components can access wireless networks based on communication standards, such as mobile hotspots (WiFi), second-generation (2G), third-generation (3G), fourth-generation (4G) / Long Term Evolution (LTE), fifth-generation (5G), or combinations thereof. In one exemplary embodiment, the communication components receive broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication components also include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be based on Radio Frequency Identification (RFID), infrared, Ultra Wide Band (UWB), Bluetooth, and other technologies.

[0245] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0246] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0247] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside within an application-specific integrated circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components within an electronic device or host device.

[0248] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0249] The order of the embodiments described above is merely for illustrative purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The sequence numbers are merely used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types. "Multiple" means two or more, unless otherwise explicitly specified.

[0250] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0251] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0252] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A model-based data processing method, characterized in that, include: Input data is fed into the target model, which is obtained by inserting extended parameters into the original parameters of the machine learning model, and the extended parameters do not change the inference results of the original parameters of the machine learning model. The feature representation of the input data is obtained by reasoning through the original parameters in the target model, and calibration information is generated synchronously through the extended parameters. Based on the calibration information, the inference result generated based on the feature representation is calibrated to obtain the calibration result.

2. The method according to claim 1, characterized in that, The machine learning model includes an embedding layer, multi-layered stacked transformer blocks, and a task head. The target model is obtained by inserting extended parameters into the original parameters of the machine learning model, including: An extended embedding layer is obtained by concatenating extended parameters with the original parameters of the embedding layer of the machine learning model. Transformation extension parameters are spliced ​​onto the original parameters of each of the multi-layer stacked converter blocks to obtain multi-layer stacked extended converter blocks; An extension head is added after the multi-layered stacked extension converter block to obtain the target model; The embedding extension parameters, the transformation extension parameters, and the parameters of the extension head are obtained through training.

3. The method according to claim 2, characterized in that, The feature representation of the input data is obtained by reasoning through the original parameters in the target model, and calibration information is generated synchronously through the extended parameters, including: The input data is input into the extended embedding layer of the target model. The original parameters of the extended embedding layer are used to map the input data into the original embedding vector, and the extended embedding parameters are used to map the input data into the incremental embedding vector. The original embedding vector and the incremental embedding vector are input into a multi-layer stacked extended transformer block. The original embedding vector is calculated using the original parameters of the extended transformer block to obtain the feature representation of the input data. The extended representation is then calculated using the transformation extension parameters on the original embedding vector and the incremental embedding vector. The extended representation is input into the extended header, and the extended representation is mapped to calibration information through the extended header.

4. The method according to any one of claims 1-3, characterized in that, The step of calibrating the inference result generated based on the feature representation according to the calibration information to obtain the calibration result includes: Based on the calibration information, the feature representation of the input data is calibrated to obtain the calibrated feature representation; The calibrated feature representation is converted into inference results using the task head of the target model.

5. The method according to any one of claims 1-3, characterized in that, The calibration information represents the confidence level of the inference result. The step of calibrating the inference result generated based on the feature representation according to the calibration information to obtain the calibration result includes: If the calibration information is greater than or equal to the confidence threshold, the input data is re-inputted into the target model for re-inference to obtain new inference results and new calibration information. This process continues until the new calibration information is less than the confidence threshold, at which point the latest obtained inference result is taken as the calibration result. or, If the calibration information is greater than or equal to the confidence threshold, backtrack by at least one time step and re-execute the inference process of the at least one time step using the target model.

6. The method according to claim 2, characterized in that, Also includes: Construct a backup model of the machine learning model, wherein the backup model has the same structure and parameters as the machine learning model; Insert embedded extension parameters into the embedding layer of the backup model, insert transformation extension parameters into the linear layer contained in the transformer block of the backup model, and add an extension head to obtain the extended model. Using the dataset, adjust the parameters of the embedding extension parameters, the transformation extension parameters, and the extension head in the extended model to obtain the trained embedding extension parameters, transformation extension parameters, and extension head.

7. The method according to claim 6, characterized in that, Inserting transform extension parameters into the linear layers contained in the transformer block of the backup model, including: For any linear layer contained in any transformer block of the backup model, an extended mapping matrix is ​​concatenated on the linear mapping matrix of the linear layer, and an extended vector is concatenated on the bias vector of the linear layer to obtain the corresponding extended linear layer. After expanding each linear layer contained in the transformer block of the backup model into a corresponding extended linear layer, the corresponding extended transformer block is obtained. The transformation extension parameters include the extended mapping matrix in the extended transformer block.

8. The method according to claim 7, characterized in that, The complete hidden state vector output by the extended linear layer includes: the original hidden state vector and the extended hidden vector. The method further includes: The normalization layer connected to the extended linear layer in the extended transformer block is adjusted. The adjusted normalization layer uses the mean and / or variance of the original hidden state vector output by the extended linear layer to normalize the complete hidden state vector output by the extended linear layer.

9. The method according to claim 6, characterized in that, Using the dataset, the parameters of the embedding expansion parameters, the transform expansion parameters, and the expansion head in the extended model are adjusted to obtain the trained embedding expansion parameters, transform expansion parameters, and expansion head, including: The sample data in the dataset is input into the extended embedding layer of the extended model for mapping to obtain the extended embedding vector, which includes the original embedding vector and the incremental embedding vector of the sample data. The extended embedding vector is processed by the extended transformer block to obtain a complete vector representation, which includes the feature representation and extended representation of the sample data. The extended model's task header generates inference prediction results based on the feature representation of the sample data. Calibration information is generated based on the extended representation using the extended head of the extended model; Calculate the calibration task loss based on the inference prediction results and / or the calibration information; Based on the calibration task loss, adjust the embedding extension parameters, the transformation extension parameters, and the parameters of the extension head in the extended model.

10. The method according to claim 9, characterized in that, Also includes: The regularization loss is calculated based on the complete hidden state vector and the original hidden state vector output by each extended transformer block of the extended model. Based on the regularization loss, adjust the embedding extension parameters, the transformation extension parameters, and the parameters of the extension head in the extended model.

11. The method according to claim 10, characterized in that, The step of calculating the regularization loss based on the complete hidden state vector and the original hidden state vector output by each extended transformer block of the extended model includes: The regularization loss is calculated based on the root mean square difference between the complete hidden state vector output by each extended transformer block of the extended model and the original hidden state vector.

12. A method for accelerating model inference, characterized in that, include: The input data is fed into the target model, and the target model is used to iterate and perform inference at multiple time steps to generate the target text; The target model is obtained by inserting extended parameters into the original parameters of the text generation model, and the extended parameters do not change the inference results of the original parameters of the text generation model. The reasoning process for each time step includes: The input data and the generated text tags are input into the target model. The input feature representation is obtained by reasoning through the original parameters of the target model, and the extended feature representation is generated synchronously through the extended parameters. The next text tag is generated based on the input feature representation, and at least one text tag following the next text tag is generated based on the extended feature representation.

13. The method according to claim 12, characterized in that, The text generation model includes an embedding layer, multiple stacked transformer blocks, and a text generation head. The target model is obtained by inserting extended parameters into the original parameters of the text generation model, including: Extended parameters are concatenated with the original parameters of the embedding layer of the text generation model to obtain the extended embedding layer; Transformation extension parameters are spliced ​​onto the original parameters of each of the multi-layer stacked converter blocks to obtain multi-layer stacked extended converter blocks; At least one extension head is added after the multi-layered stacked extension converter block to obtain the target model; The embedding extension parameters, the transformation extension parameters, and the parameters of the at least one extension head are obtained through training.

14. The method according to claim 13, characterized in that, The input feature representation is obtained through inference using the original parameters of the target model, and an extended feature representation is generated synchronously using the extended parameters, including: The input data is input into the extended embedding layer of the target model. The original parameters of the extended embedding layer are used to map the input data into the original embedding vector, and the extended embedding parameters are used to map the input data into the incremental embedding vector. The original embedding vector and the incremental embedding vector are input into a multi-layer stacked extended transformer block. The original embedding vector is calculated using the original parameters of the extended transformer block to obtain the input feature representation. The extended feature representation is then calculated using the transformation extension parameters on the original embedding vector and the incremental embedding vector.

15. The method according to claim 14, characterized in that, Generating the next text tag based on the input feature representation, and generating at least one text tag following the next text tag based on the extended feature representation, including: The input feature representation is input into the text generation head, and the text generation head predicts the next text tag based on the input feature representation. The extended representation is input into the at least one extended header, and the at least one text tag following the next text tag is predicted based on the extended representation using the at least one extended header.

16. A model-based data processing method, characterized in that, include: In response to natural language processing task requests, obtain input data; The input data is input into the target model, which is obtained by inserting extended parameters into the original parameters of the natural language processing model, and the extended parameters do not change the inference results of the original parameters of the natural language processing model. By reasoning through the original parameters in the target model, the feature representation of the input data is obtained, and calibration information is generated synchronously through the extended parameters. Based on the calibration information, the natural language processing result generated based on the feature representation is calibrated to obtain the calibrated natural language processing result.

17. The method according to claim 16, characterized in that, The natural language processing model includes: an embedding layer, multi-layered stacked transformer blocks, and a task head. The target model is obtained by inserting extended parameters into the original parameters of the natural language processing model, including: An extended embedding layer is obtained by concatenating extended parameters with the original parameters of the embedding layer of the natural language processing model. Transformation extension parameters are spliced ​​onto the original parameters of each of the multi-layer stacked converter blocks to obtain multi-layer stacked extended converter blocks; An extension head is added after the multi-layered stacked extension converter block to obtain the target model; The embedding extension parameters, the transformation extension parameters, and the parameters of the extension head are obtained through training.

18. A model-based data processing method, characterized in that, include: Receive query information sent by the receiving end device; The query information is input into the target language model, which is obtained by inserting extended parameters into the original parameters of the pre-trained language model, and the extended parameters do not change the inference results of the original parameters of the pre-trained language model. The feature representation of the query information is obtained by reasoning through the original parameters in the target language model, and calibration information is generated synchronously through the extended parameters. Based on the calibration information, the response result generated from the feature representation based on the query information is calibrated to obtain the calibrated response result; Output the calibrated response result.

19. The method according to claim 18, characterized in that, The pre-trained language model includes: an embedding layer, multi-layered stacked transformer blocks, and a task head. The target language model is obtained by inserting extended parameters into the original parameters of the pre-trained language model, including: By concatenating extended parameters with the original parameters of the embedding layer of the pre-trained language model, an extended embedding layer is obtained. Transformation extension parameters are spliced ​​onto the original parameters of each of the multi-layer stacked converter blocks to obtain multi-layer stacked extended converter blocks; An extension header is added after the multi-layered stacked extension transformer block to obtain the target language model; The embedding extension parameters, the transformation extension parameters, and the parameters of the extension head are obtained through training.

20. A server, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the server to perform the method according to any one of claims 1-19.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-19.

22. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-19.

Images