Model training method, model reasoning method and related devices

Abstract

The embodiment of the invention provides a model training method, a model reasoning method and a related device, which are used for reasoning input data through a trained combination model to obtain a reasoning result, and a small language model in the trained combination model interacts based on a cross attention mechanism, so that the quality of the reasoning result generated by the model is improved. The method comprises the following steps: acquiring a first intermediate state; obtaining a second intermediate state; based on a cross attention mechanism, performing cross attention operation on the first intermediate state and the second intermediate state to obtain a first operation result; analyzing the first operation result and the second intermediate state through a second small language model to obtain a first reasoning result; according to the first reasoning result, parameters of the combination model are trained, a trained combination model is obtained, and the trained combination model is used for conducting reasoning according to the input text to obtain question and answer pair data of the input text.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a model training method, a model inference method, and related apparatus. Background Technology

[0002] To improve the performance of Large Language Models (LLMs) in specific domains, they can be trained using domain-specific data. However, collecting domain-specific question-answer pairs (Q&A) data requires significant manpower and time. With the rapid development of LLMs, it is now possible to generate domain-specific Q&A data using LLMs, thereby improving the efficiency of Q&A generation in specific domains.

[0003] Currently, to ensure the quality of generated question-and-answer pairs, large-scale language models are typically used for data generation. However, generating data using large-scale language models consumes significant computing resources and incurs substantial costs. Therefore, how to reduce computing resource consumption while maintaining the quality of generated data has become an urgent problem to be solved. Summary of the Invention

[0004] This application provides a model training method, a model inference method, and related apparatus, which are used to infer inference results from input data through a trained combined model. The small language models in the trained combined model interact based on a cross-attention mechanism, thereby improving the quality of the inference results generated by the model.

[0005] In a first aspect, this application provides a model training method, comprising: obtaining a first intermediate state, wherein the first intermediate state is the output result of a first intermediate layer obtained based on first input data, the first intermediate layer being an intermediate layer of a first small language model in a combined model, wherein the combined model includes at least two small language models; obtaining a second intermediate state, wherein the second intermediate state is the output result of a second intermediate layer obtained based on second input data, the second intermediate state being an intermediate layer of a second small language model in the combined model, the second input data being obtained based on the first input data; and wherein when multiple small language models in the combined model perform step-by-step inference, the small language models can interact to obtain the first small language model. After the first intermediate state and the second intermediate state of the second small language model, a cross-attention operation can be performed on the first and second intermediate states based on the cross-attention mechanism to obtain a first operation result. This first operation result is used to represent the importance of the first intermediate state in generating the second intermediate state. The first operation result and the second intermediate state are analyzed through the second small language model to obtain a first inference result. Based on the first inference result, the parameters of the combined model are trained to obtain the trained combined model. The trained combined model is used to infer the question-answer pair data of the input text based on the input text. The question-answer pair data includes the question obtained in response to the input text and the answer to the question.

[0006] In this embodiment, the trained ensemble model can perform inference based on the input text to obtain question-answer pairs. Similarly, the trained ensemble model includes at least two smaller language models. Because the ensemble model can perform step-by-step inference based on multiple smaller language models within it, and each subsequent model (the second smaller language model) can interact with the preceding model (the first smaller language model), the second smaller language model can determine the importance of the contextual information in the first smaller language model to its generated result. This allows the second smaller language model to not only combine relevant contextual information during inference but also to generate inference results based on the importance of the contextual information, thereby improving the quality of the generated inference results. Therefore, using the trained ensemble model to infer from the input text consumes less computational resources and generates high-quality inference results compared to a large language model.

[0007] In one possible implementation, the aforementioned combined model further includes a third small language model. The method may further include: obtaining updated second and third intermediate states, wherein the updated second intermediate state is obtained by updating the second intermediate state by the second intermediate layer based on the first operation result, and the third intermediate state is the output result obtained by the third intermediate layer based on the third input data. The third intermediate layer is an intermediate layer of the third small language model, and the third input data is obtained based on the second input data; performing cross-attention operation on the updated second and third intermediate states to obtain a second operation result, wherein the second operation result is used to represent the importance of the updated second intermediate state in generating the third intermediate state; and analyzing the second operation result and the third intermediate state through the third small language model to obtain a second inference result.

[0008] In this embodiment of the application, when the combined model includes two or more small language models, the two adjacent small language models can still interact and can perform operations based on the cross-attention mechanism, so that the latter model can obtain relevant contextual information from the former model and perform inference based on the importance of the contextual information, thereby improving the quality of the inference results generated by the small language models.

[0009] In one possible implementation, the aforementioned process of training the parameters of the combined model based on the first inference result to obtain the trained combined model may include: training the parameters of the combined model based on the first inference result and the second inference result to obtain the trained combined model.

[0010] In this embodiment of the application, when the combined model includes two or more small language models, the parameters in the combined model will be trained based on the inference results of the multiple small language models.

[0011] In one possible implementation, the aforementioned process of training the parameters of the combined model based on the first inference result and the second inference result to obtain the trained combined model may include: training the parameters of the combined model based on the error between the first inference result and the first actual result, and the error between the second inference result and the second actual result, to obtain the trained combined model.

[0012] In one possible implementation, before obtaining the second intermediate state, the method may further include: reasoning about the first input data using a first small language model to obtain a third reasoning result; and using the third reasoning result and the first input data as the second input data.

[0013] In this embodiment, the input data of the subsequent model can be obtained from the input data of the previous model, so that the subsequent model can continue to reason based on the reasoning of the previous model, thereby realizing step-by-step reasoning of the input data through combined models.

[0014] In one possible implementation, the aforementioned analysis of the first operation result and the second intermediate state through the second small language model to obtain the first inference result may include: updating the second intermediate state through the second intermediate layer of the second small language model based on the first operation result to obtain the updated second intermediate state; and analyzing the updated second intermediate state through the output layer of the second small language model to obtain the first inference result.

[0015] In this embodiment of the application, the first calculation result can represent the importance of the first intermediate state to the generation of the current second intermediate state. Based on the first calculation result, the second small language model can focus on contextual information with high importance when performing reasoning, so as to generate a higher quality reasoning result.

[0016] Secondly, this application provides a model reasoning method, including: acquiring input text; reasoning on the input text through a trained ensemble model to obtain question-answer pair data, wherein the question-answer pair data includes questions and answers to questions, and the trained ensemble model includes at least two small language models.

[0017] In this embodiment, the trained small language model includes multiple small language models. Using the trained small language models for inference reduces the computational resources required.

[0018] In one possible implementation, the aforementioned at least two small language models include a first small language model and a second small language model. The aforementioned reasoning on the input text through the trained combined model to obtain question-answer pair data may include: reasoning on the input text through the first small language model to obtain the question; and reasoning on the question and the input text through the second small language model to obtain the answer to the question.

[0019] In one possible implementation, before reasoning about the question and input text using the second small language model to obtain the answer to the question, the method may further include: obtaining a first intermediate result and a second intermediate result, wherein the first intermediate result is the output result obtained by the first intermediate layer based on the input text, the first intermediate layer is an intermediate layer of the first small language model, and the second intermediate result is the output result obtained by the second intermediate layer based on the question and input text, the second intermediate layer is an intermediate layer of the second small language model; and performing cross-attention operation on the first intermediate result and the second intermediate result based on a cross-attention mechanism to obtain an intermediate operation result, wherein the intermediate operation result is used to represent the importance of the first intermediate result in generating the second intermediate result.

[0020] In this embodiment, cross-attention operation is performed on the first intermediate result and the second intermediate result to find the correlation between the first intermediate result and the second intermediate result, and to obtain the importance of the first intermediate result to the second intermediate result. This allows the intermediate layer of the subsequent second small language model to consider more relevant information when performing analysis, and to combine more relevant and important parts of the question into the answer generated by the second small language model according to the importance of the relevant information, thereby improving the quality of the answer generated by the second small language model.

[0021] In one possible implementation, the aforementioned reasoning process using the second small language model to obtain the answer to the question and the input text may include: updating the second intermediate result through the intermediate layer of the second small language model based on the intermediate calculation result to obtain the updated second intermediate result; and analyzing the updated second intermediate result through the output layer of the second small language model to obtain the answer to the question.

[0022] In one possible implementation, the aforementioned trained combined model is obtained by training the combined model using the model training method provided in the first aspect.

[0023] Thirdly, this application provides a model training apparatus, comprising: an acquisition module, configured to acquire a first intermediate state, wherein the first intermediate state is the output result obtained by a first intermediate layer based on first input data, the first intermediate layer being an intermediate layer of a first small language model in a combined model, and the combined model including at least two small language models; the aforementioned acquisition module is further configured to acquire a second intermediate state, wherein the second intermediate state is the output result obtained by a second intermediate layer based on second input data, the second intermediate layer being an intermediate layer of a second small language model in the combined model, and the second input data being obtained based on the first input data; a computation module, configured to perform cross-attention computation on the first intermediate state and the second intermediate state based on a cross-attention mechanism to obtain a first computation result, the first computation result being used to represent the importance of the first intermediate state in generating the second intermediate state; an analysis module, configured to analyze the first computation result and the second intermediate state through the second small language model to obtain a first inference result; and a training module, configured to train the parameters of the combined model based on the first inference result to obtain a trained combined model.

[0024] In one possible implementation, the aforementioned combined model further includes a third small language model. The aforementioned acquisition module is also used to acquire the updated second intermediate state and the third intermediate state. The updated second intermediate state is obtained by the second intermediate layer updating the second intermediate state based on the first calculation result. The third intermediate state is the output result obtained by the third intermediate layer based on the third input data. The third intermediate layer is the intermediate layer of the third small language model, and the third input data is obtained based on the second input data. The aforementioned calculation module is also used to perform cross-attention calculation on the updated second intermediate state and the third intermediate state to obtain a second calculation result. The second calculation result is used to represent the importance of the updated second intermediate state in generating the third intermediate state. The aforementioned analysis module is also used to analyze the second calculation result and the third intermediate state through the third small language model to obtain a second inference result.

[0025] In one possible implementation, the aforementioned training module is specifically used to: train the parameters of the combined model based on the first inference result and the second inference result to obtain the trained combined model.

[0026] In one possible implementation, the aforementioned training module is specifically used to: train the parameters of the combined model based on the error between the first inference result and the first actual result, and the error between the second inference result and the second actual result, to obtain the trained combined model.

[0027] In one possible implementation, before obtaining the second intermediate state, the device may further include: a reasoning module for reasoning about the first input data using a first small language model to obtain a third reasoning result; and a determination module for using the third reasoning result and the first input data as the second input data.

[0028] In one possible implementation, the aforementioned analysis module is specifically used to: update the second intermediate state through the second intermediate layer of the second small language model based on the first calculation result, to obtain the updated second intermediate state; and analyze the updated second intermediate state through the output layer of the second small language model to obtain the first inference result.

[0029] Fourthly, this application provides a model reasoning device, comprising: an acquisition module for acquiring input text; and a reasoning module for reasoning on the input text using a trained combined model to obtain question-answer pair data, wherein the question-answer pair data includes questions and answers to questions, and the trained combined model includes at least two small language models.

[0030] In one possible implementation, the aforementioned at least two small language models include a first small language model and a second small language model. The aforementioned reasoning module is specifically used to: reason about the input text using the first small language model to obtain the question; and reason about the question and the input text using the second small language model to obtain the answer to the question.

[0031] In one possible implementation, before the aforementioned reasoning of the question and input text using the second small language model to obtain the answer to the question, the aforementioned acquisition module is further used to acquire a first intermediate result and a second intermediate result. The first intermediate result is the output result obtained by the first intermediate layer based on the input text, and the first intermediate layer is an intermediate layer of the first small language model. The second intermediate result is the output result obtained by the second intermediate layer based on the question and input text, and the second intermediate layer is an intermediate layer of the second small language model. The device may further include a computation module, specifically used to perform cross-attention computation on the first intermediate result and the second intermediate result based on a cross-attention mechanism to obtain an intermediate computation result. The intermediate computation result is used to represent the importance of the first intermediate result in generating the second intermediate result.

[0032] In one possible implementation, the aforementioned reasoning module is specifically used to: update the second intermediate result through the intermediate layer of the second small language model based on the intermediate calculation result, to obtain the updated second intermediate result; and analyze the updated second intermediate result through the output layer of the second small language model to obtain the answer to the question.

[0033] In one possible implementation, the aforementioned trained combined model is obtained by training the combined model using the method described in the first aspect above.

[0034] Fifthly, this application provides a model training apparatus, which includes: a processor, a memory, an input / output device, and a bus; the memory stores computer instructions; when the processor executes the computer instructions in the memory, the memory stores computer instructions; when the processor executes the computer instructions in the memory, it is used to implement the first aspect and any one of the implementations of the first aspect.

[0035] In a sixth aspect, this application provides a model inference apparatus, which includes: a processor, a memory, an input / output device, and a bus; the memory stores computer instructions; when the processor executes the computer instructions in the memory, the memory stores computer instructions; when the processor executes the computer instructions in the memory, it is used to implement the second aspect and any one of the implementations of the second aspect.

[0036] In a seventh aspect, embodiments of this application provide a computer-readable storage medium including computer program instructions, which, when executed by a cluster of computing devices, enable the cluster of computing devices to perform the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0037] Eighthly, embodiments of this application provide a computer program product containing instructions that, when executed by a computing device cluster, cause the computing device cluster to perform the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect. Attached Figure Description

[0038] Figure 1 A schematic diagram of a system architecture provided for an embodiment of this application;

[0039] Figure 2 An example diagram of the architecture of a combined model provided in this application embodiment;

[0040] Figure 3 A schematic flowchart illustrating a model training method provided in an embodiment of this application;

[0041] Figure 4 A flowchart illustrating a model reasoning method provided in an embodiment of this application;

[0042] Figure 5 This is a schematic diagram of the structure of a model training device provided in an embodiment of this application;

[0043] Figure 6 This is a schematic diagram of the structure of a model inference device provided in an embodiment of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0045] To facilitate understanding, some concepts involved in this application will be introduced first.

[0046] 1. Large Language Model (LLM)

[0047] LLM (Language Modeling) refers to a language model with hundreds of millions (or more) parameters trained on massive amounts of text data. It's a deep learning-based natural language processing model. These models can process large amounts of text data to learn the grammatical and semantic rules of natural language. LLMs can be applied to text generation, machine translation, question answering systems, text summarization, and sentiment analysis, offering advantages such as strong generative capabilities, high adaptability, accurate prediction, and scalability. For example, in movie recommendation scenarios, large language models can generate descriptions of movie scenes, including genre, main actors, and plot, enabling the system to better recommend similar films. Large language models can also generate recommendation reasons; for instance, e-commerce websites can use large language models to generate reasons for recommending products, such as product quality, price, and features, allowing users to better understand the value of the products.

[0048] 2. Small Language Models (SLM)

[0049] SLM is a technique in the field of artificial intelligence, referring to a smaller model with fewer parameters compared to large language models (LLM).

[0050] 3. Cross-attention mechanism

[0051] Cross-attention is an importance allocation mechanism that focuses on the relationships between different modalities or sequences. It involves computation between two sequences, where the query vector of one sequence is compared with the key and value vectors of another sequence to dynamically adjust the attention each sequence gives to the others, achieving comprehensive information utilization. Cross-attention is commonly applied in machine translation, question-answering systems, image-text tasks, or multimodal learning scenarios. For example, in question-answering tasks, where the question and paragraph are two different inputs, cross-attention allows the model to treat the question as a query when generating an answer, dynamically focusing on different parts of the paragraph to locate the most relevant content and generate the answer.

[0052] With the development of large language models, it is possible to generate corresponding question-answer pairs for specific domains, thereby avoiding the need for extensive manual collection of such data. However, due to the large scale of large language models, inference will consume a significant amount of computational resources, resulting in substantial computational costs.

[0053] Therefore, embodiments of this application propose a combined model, which includes at least two smaller language models. For this combined model, embodiments of this application provide a model training method and a model inference method. This allows the trained combined model to be used to infer question-answer pairs from input questions.

[0054] To make the technical solution of this application clearer and easier to understand, the system architecture of this application will be described below with reference to the accompanying drawings.

[0055] See Figure 1 This application provides a system architecture 100. As shown in the system architecture 100, the data acquisition device 160 can be used to collect training data. After the data acquisition device 160 collects the training data, it stores the training data in the database 130. The training device 120 trains the target model / rule 101 based on the training data maintained in the database 130.

[0056] The target model / rule 101 in this embodiment can specifically be a combined model, which includes two or more smaller language models. It should be noted that in practical applications, the training data maintained in the database 130 may not all come from the data acquisition device 160; it may also be received from other devices. Furthermore, it should be noted that the training device 120 may not necessarily train the target model / rule 101 entirely based on the training data maintained in the database 130; it may also obtain training data from the cloud or other sources for model training. The above description should not be construed as limiting the embodiments of this application.

[0057] The computation module 111 includes a target model / rule 101. The model training process mentioned in this application can be executed by the training device 120, and the model inference process mentioned in this application can be executed by the target model / rule 101. The target model / rule 101 trained by the training device 120, i.e., the trained combined model, can be applied to different devices, such as... Figure 1 The execution device 110 shown can be a terminal device such as a mobile phone, watch, wristband, tablet computer, and vehicle-mounted device, or a device with computing capabilities such as a CPU, GPU, and FPGA. Figure 1 In this embodiment, the execution device 110 is equipped with a transceiver 112, which may include an input / output (I / O) interface or other wireless or wired communication interfaces for data interaction with external devices. Taking the I / O interface as an example, the user can input text into the I / O interface through the client device 140. For example, in the following embodiments of this application, the client device 140 can send the question-answer pair data obtained by the trained combined model based on the input text to the execution device 110.

[0058] During the preprocessing of input text by the execution device 110, or the calculation module 111 of the execution device 110 performing calculations and other related processes, the execution device 110 can call data, code, etc. in the data storage system 150 for corresponding processing, or store the data, instructions, etc. obtained from the corresponding processing into the data storage system 150.

[0059] Finally, I / O interface 112 returns the processing result to client device 140, thereby providing it to the user.

[0060] In addition, Figure 1 This is merely a schematic diagram of a system architecture provided in an embodiment of this application. The positional relationships between the devices, components, modules, etc., shown in the diagram do not constitute any limitation. For example, in Figure 1 In this context, the data storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 may also be placed within the execution device 110.

[0061] Figure 2 This is a framework example diagram of a combined model provided in an embodiment of this application. For example... Figure 2 As shown, the combined model comprises three small language models, with adjacent small language models connected by a cross-attention module. This cross-attention module allows the subsequent model in a given pair of small language models to consider the importance of the output of the preceding model to the reasoning result generated by the subsequent model during the reasoning process. Specifically, the first small language model can infer a key paragraph from the input text based on a first prompt, where the first prompt can be finding a key paragraph in the input text; the second small language model can derive a question from the input text based on a second prompt, where the second prompt can be generating a question from the input text based on the key paragraph; and the third small language model can find the answer to the question from the input text based on a third prompt, where the third prompt is deriving the answer to the question from the input text based on the question. The aforementioned three small language models are combined through the cross-attention module to obtain a combined model that can generate question-answer pairs data, including questions and answers, from the input text.

[0062] Understandable Figure 2 The framework of the composition model shown is merely an example and is not intended to limit its specific implementation to this example. For example, in other possible frameworks, the composition model may also include two or more small language models, etc.

[0063] The method flow provided in this application will be described below in conjunction with the aforementioned system architecture.

[0064] This application provides a model training method and a model inference method. It can be understood that the model training method and the model inference method provided in this application are respectively referred to as the training stage and the inference stage. The training stage is the model training method provided in this application, and the inference stage is the process of the model inference method provided in this application.

[0065] I. Training Phase

[0066] See Figure 3 The flowchart of a model training method provided in this application is as follows.

[0067] Step 301: Obtain the first intermediate state;

[0068] To reduce the computational resources consumed by model inference, this application provides a combined model comprising at least two smaller language models. When using the combined model to infer from input data, the inference process can be divided into multiple steps. Specifically, multiple smaller language models within the combined model perform inference based on the input data. Furthermore, the inference of a subsequent model within the combined model can consider relevant information from the inference process of the preceding model, thereby increasing the relevant basis for the subsequent model's inference and improving the quality of the inference results generated by the subsequent model.

[0069] In this embodiment, the first intermediate state is the output result obtained by the first intermediate layer based on the first input data. This first intermediate layer is the intermediate layer of the first small language model in the combined model. The obtained first intermediate state can be used by the intermediate layer of the second small language model to update the second intermediate state.

[0070] Step 302: Obtain the second intermediate state;

[0071] Before obtaining the second intermediate state, the first small language model can first perform inference based on the first input data to generate a third inference result. Then, this third inference result and the first input data are used as the second input data. For example, when the first input data is input text, the first small language model can extract a question related to the input text, which is the aforementioned third inference result. This question and the first input data are then used as the input data for the second small language model.

[0072] After obtaining the second input data, it can be analyzed by the second intermediate layer of the second minor language model to obtain the second intermediate state. The second intermediate layer is the intermediate layer of the second minor language model in the combined model, and the second intermediate state is the output result obtained by the second intermediate layer based on the second input data.

[0073] Step 303: Based on the cross-attention mechanism, perform cross-attention operation on the first intermediate state and the second intermediate state to obtain the first operation result;

[0074] Because cross-attention mechanisms can be used for information interaction between different inputs, enabling models to effectively align and focus on contexts from different sources, thereby helping the model better capture the correlation between two inputs. Therefore, in this embodiment, based on the cross-attention mechanism, operations can be performed on the first intermediate state and the second intermediate state to obtain a first operation result. This first operation result can be used to represent the importance of the first intermediate state to the generated second intermediate state. This allows the subsequent second intermediate layer to combine the contextual information related to the generation of the second intermediate state in the first small language model during inference, and, based on the importance of this contextual information to the second intermediate state, allow the second intermediate layer to focus on more important related information, thereby improving the quality of the inference results generated by the subsequent second small language model.

[0075] Specifically, R f R L W represents the first intermediate state and the second intermediate state, respectively. Q W K W V W O These represent the Q, K, V, and O matrix parameters in the cross-attention mechanism. The query Q originates from the second intermediate state, while the key K and value V both originate from the first intermediate state. The process of performing cross-attention operations on the first and second intermediate states is as follows:

[0076] f proj (R f ) = R mid

[0077] K,V=R mid W K ,R mid W V

[0078] Q = R L W Q

[0079] f cross =Attention(Q,K,V)W O

[0080] The formula for calculating attention is as follows:

[0081] Among them, f pro j f represents the linear mapping layer in the cross-attention module. cross W represents the result of the first operation.O This is used to map the calculated attention values ​​to the same value range as the subsequent model (the second small language model), d k The dimension of the key.

[0082] Step 304: Analyze the first operation result and the second intermediate state using the second small language model to obtain the first reasoning result;

[0083] After obtaining the first calculation result, the second intermediate layer of the second small language model can consider the influence of the intermediate state of the previous model (the first language model) on the intermediate state generated by the second intermediate layer during the reasoning process, so that the second intermediate layer can consider more relevant information when reasoning, thereby improving the generation quality.

[0084] Optionally, based on the first calculation result, the second intermediate state can be updated through the second intermediate layer of the second small language model to obtain the updated second intermediate state; subsequently, the updated second intermediate state can be analyzed through the output layer of the second small language model to obtain the first inference result.

[0085] Step 305: Based on the first inference result, train the parameters of the combined model to obtain the trained combined model.

[0086] In this embodiment of the application, the trained ensemble model can be used to infer from the input text to obtain question-answer pair data related to the input text, and can also be applied to multiple specific domains to obtain question-answer pair datasets for a specific domain by using the trained ensemble model based on the text of that specific domain.

[0087] The combined model includes at least two smaller language models. When the combined model includes a third smaller language model in addition to the first and second smaller language models, the influence of the second smaller language model on the third smaller language model's reasoning must also be considered when the third smaller language model is making inferences.

[0088] Similarly, when the combined model includes more than three small language models, the subsequent model still needs to consider the influence of the previous model on its reasoning, which will not be elaborated here.

[0089] Optionally, updated second and third intermediate states can be obtained. The third intermediate state is the output of the third intermediate layer based on the third input data. This third intermediate layer is the intermediate layer of the third small language model, and the third input data is obtained based on the second input data.

[0090] Subsequently, based on the cross-attention mechanism, cross-attention operations can be performed on the updated second and third intermediate states to obtain a second operation result. This second operation result represents the importance of the updated second intermediate state to the third intermediate state. The second operation result and the third intermediate state are then analyzed using a third small language model to obtain a second inference result.

[0091] Optionally, the parameters of the combined model can be trained based on the first and second inference results to obtain the trained combined model. Similarly, when the combined model includes multiple small language models, the parameters of the combined model can be trained based on the inference results of the multiple small language models to obtain the trained combined model.

[0092] Optionally, the parameters of the combined model can be trained based on the error between the first inference result and the first actual result, as well as the error between the second inference result and the second actual result, to obtain the trained combined model.

[0093] Specifically, a loss function can be constructed based on the first inference result and the first actual result, and the error can be passed to the parameters of the combined model through the backpropagation algorithm, thereby reducing the error by adjusting the parameters.

[0094] In this combined model, two adjacent smaller language models can be connected via a cross-attention module. When training the combined model, the parameters of the cross-attention module can be adjusted without changing the parameters of the smaller language models, thus reducing error. Adjusting the parameters of the cross-attention module to modify the parameters of the combined model avoids gradient vanishing during backpropagation, which could lead to training failure.

[0095] II. Reasoning Stage

[0096] See Figure 4 The flowchart of a model reasoning method provided in this application is as follows.

[0097] Step 401: Obtain the input text;

[0098] The input text can be text from a specific field, such as medicine, finance, auditing, or law. The input text can be medical articles, financial reports, or legal documents, etc.

[0099] Step 402: Use the trained combined model to reason about the input text to obtain question-answer pairs.

[0100] After obtaining the input text, the trained ensemble model can be used to infer question-answer pairs from the input text. The trained ensemble model includes at least two smaller language models, and it is trained using the aforementioned model training method.

[0101] Optionally, when at least two small language models include a first small language model and a second small language model, the input text can be reasoned through the first small language model to obtain a question related to the input text; subsequently, the question and the input text can be reasoned through the second small language model, and the answer to the question can be obtained from the input text by querying the input text based on the question obtained from the input text by the first small language model.

[0102] Optionally, when at least two small language models include a first small language model, a second small language model, and a third small language model, the first small language model can be used to reason about the input text to obtain key paragraphs in the input text; then the second small language model can be used to obtain the question from the key paragraphs based on the prompts; and finally, the third small language model can be used to obtain the answer to the question from the input text based on the question.

[0103] It is understandable that the number of small language models in the combined model can be two, three, or more; there is no limitation here. Based on the actual number of small language models in the combined model, the question-answer pair data of the input text can be obtained through step-by-step reasoning using multiple small language models.

[0104] Optionally, before reasoning about the question and input text using the second small language model to obtain the answer, a first intermediate result and a second intermediate result can be obtained. The first intermediate result is the output of the first intermediate layer of the first small language model based on the input text, and the second intermediate result is the output of the second intermediate layer of the second small language model based on the question and input text. Subsequently, a cross-attention mechanism can be used to perform cross-attention operations on the first and second intermediate results to obtain an intermediate operation result. This intermediate operation result represents the importance of the first intermediate result in generating the second intermediate result, providing relevant contextual information for generating the updated second intermediate result. Furthermore, the updated second intermediate result can be generated based on the importance of the first intermediate result in generating the second intermediate result. This allows the second small language model to focus on relevant and highly important parts of the input text when generating the answer, and combines this with contextual information relevant to the current task, thereby improving the quality of the answer generated by the second small language model.

[0105] Optionally, after obtaining the intermediate calculation results, the second intermediate layer can update the second intermediate result based on the intermediate calculation results to obtain the updated second intermediate result; subsequently, the output layer of the second small language model can analyze the updated second intermediate result to obtain the answer to the question.

[0106] In this embodiment, question-answer pairs obtained from the input text can be used to perform step-by-step reasoning through multiple small language models in a trained ensemble model. During reasoning, each subsequent model can consider the influence of the previous model's output on its own reasoning, adding information from the previous model relevant to the subsequent model's reasoning. Based on a cross-attention mechanism, the importance of relevant information to the subsequent model is calculated, allowing the subsequent model to combine relevant and highly important information for reasoning, thus improving the quality of the generated reasoning results.

[0107] For example, based on multiple datasets such as MS_MARCO, SQuAD, ASCN, Nuclesar, and Financial, the collaborative small language models (CSLM) trained using the model training method provided in this application embodiment are compared with several large language models, including InternLM2-1.8B, QWen1.5-1.8B, QWen1.5-4B, InternLM2-7B, and QWen1.5-7B, from multiple metrics such as relevance, comprehensiveness, correctness, and coherence. The comparison results are shown in Table 1.

[0108] Table 1

[0109]

[0110] Table 2

[0111]

[0112] As shown in Tables 1 and 2, the combined model proposed in this application outperforms other large language models in all metrics based on datasets from different domains. This indicates that the combined model trained using the model training method proposed in this application can obtain higher-quality question-answer pair datasets with less computational resources.

[0113] Furthermore, to verify the impact of the cross-attention mechanism on the quality of the model's generated results, the proposed ensemble model (CSLM) was compared with models that removed one cross-attention module and models that removed both cross-attention modules. The comparison results are shown in Table 3. For example, the ensemble model in this case includes three small language models, with adjacent small language models connected by a cross-attention model. This allows the latter model in two adjacent small language models to interact with the former model and perform computations based on the cross-attention mechanism. This enables the latter model to consider contextual information from the former model that is relevant to its reasoning during inference, and to focus more on the more important parts of that contextual information, thereby improving the quality of the inference results generated by the latter model.

[0114] Table 3

[0115]

[0116] As shown in Table 3, when using the combined model to perform step-by-step reasoning on the input data, the interaction between the small language models and the influence of the previous model on the next model will improve the quality of the reasoning results generated by the combined model.

[0117] The method flow provided in this application has been described above. The apparatus provided in this application will now be described based on the aforementioned method flow.

[0118] See Figure 5 The structural schematic diagram of a model training device provided in this application is as follows.

[0119] The acquisition module 501 is used to acquire a first intermediate state, which is the output result obtained by the first intermediate layer based on the first input data. The first intermediate layer is the intermediate layer of the first small language model in the combined model. The combined model includes at least two small language models.

[0120] The aforementioned acquisition module 501 is also used to acquire a second intermediate state, which is the output result obtained by the second intermediate layer based on the second input data. The second intermediate layer is the intermediate layer of the second small language model in the combined model, and the second input data is obtained based on the first input data.

[0121] The operation module 502 is used to perform cross-attention operation on the first intermediate state and the second intermediate state based on the cross-attention mechanism to obtain a first operation result. The first operation result is used to represent the importance of the first intermediate state in generating the second intermediate state.

[0122] Analysis module 503 is used to analyze the first operation result and the second intermediate state through the second small language model to obtain the first reasoning result;

[0123] Training module 504 is used to train the parameters of the combined model based on the first inference result, so as to obtain the trained combined model.

[0124] In one possible implementation, the aforementioned combined model further includes a third small language model. The aforementioned acquisition module 501 is also used to acquire the updated second intermediate state and the third intermediate state. The updated second intermediate state is obtained by the second intermediate layer updating the second intermediate state based on the first calculation result. The third intermediate state is the output result obtained by the third intermediate layer based on the third input data. The third intermediate layer is the intermediate layer of the third small language model, and the third input data is obtained based on the second input data. The aforementioned calculation module 502 is also used to perform cross-attention calculation on the updated second intermediate state and the third intermediate state to obtain a second calculation result. The second calculation result is used to represent the importance of the updated second intermediate state in generating the third intermediate state. The aforementioned analysis module 503 is also used to analyze the second calculation result and the third intermediate state through the third small language model to obtain a second inference result.

[0125] In one possible implementation, the aforementioned training module 504 is specifically used to: train the parameters of the combined model based on the first inference result and the second inference result to obtain the trained combined model.

[0126] In one possible implementation, the aforementioned training module 504 is specifically used to: train the parameters of the combined model based on the error between the first inference result and the first actual result, and the error between the second inference result and the second actual result, to obtain the trained combined model.

[0127] In one possible implementation, before obtaining the second intermediate state, the device may further include: a reasoning module 505, used to reason about the first input data through a first small language model to obtain a third reasoning result; and a determination module 506, used to use the third reasoning result and the first input data as the second input data.

[0128] In one possible implementation, the aforementioned analysis module 503 is specifically used to: update the second intermediate state through the second intermediate layer of the second small language model according to the first calculation result, to obtain the updated second intermediate state; and analyze the updated second intermediate state through the output layer of the second small language model to obtain the first inference result.

[0129] See Figure 6 The structural schematic diagram of a model inference device provided in this application is as follows.

[0130] Module 601 is used to acquire input text;

[0131] The reasoning module 602 is used to reason about the input text through the trained ensemble model to obtain question-answer pairs, which include questions and answers. The trained ensemble model includes at least two small language models.

[0132] In one possible implementation, the aforementioned at least two small language models include a first small language model and a second small language model. The aforementioned reasoning module 602 is specifically used to: reason about the input text through the first small language model to obtain the question; and reason about the question and the input text through the second small language model to obtain the answer to the question.

[0133] In one possible implementation, before the aforementioned reasoning of the question and input text using the second small language model to obtain the answer to the question, the aforementioned acquisition module 601 is further used to acquire a first intermediate result and a second intermediate result. The first intermediate result is the output result obtained by the first intermediate layer based on the input text, and the first intermediate layer is the intermediate layer of the first small language model. The second intermediate result is the output result obtained by the second intermediate layer based on the question and input text, and the second intermediate layer is the intermediate layer of the second small language model. The device may also include: a calculation module 603, specifically used to: perform cross-attention calculation on the first intermediate result and the second intermediate result based on a cross-attention mechanism to obtain an intermediate calculation result, which is used to represent the importance of the first intermediate result in generating the second intermediate result.

[0134] In one possible implementation, the aforementioned reasoning module 602 is specifically used to: update the second intermediate result through the intermediate layer of the second small language model based on the intermediate calculation result, to obtain the updated second intermediate result; and analyze the updated second intermediate result through the output layer of the second small language model to obtain the answer to the question.

[0135] In one possible implementation, the aforementioned trained combined model is obtained by training the combined model using the aforementioned model training method.

[0136] This application also provides a computer-readable storage medium storing a program that, when run on a computer, causes the computer to perform the aforementioned actions. Figure 3 or Figure 4 The steps in the method described in the illustrated embodiment.

[0137] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned actions. Figure 3 or Figure 4 The method steps described in the illustrated embodiment.

[0138] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process of the systems, devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0141] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0143] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0144] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0145] Finally, it should be noted that the above are only specific embodiments of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application.

Claims

1. A model training method, characterized in that, include: Obtain a first intermediate state, which is the output result of the first intermediate layer based on the first input data. The first intermediate layer is the intermediate layer of the first small language model in the combined model. The combined model includes at least two small language models. Obtain a second intermediate state, which is the output result of the second intermediate layer based on the second input data. The second intermediate layer is the intermediate layer of the second small language model in the combined model, and the second input data is obtained based on the first input data. Based on the cross-attention mechanism, cross-attention operation is performed on the first intermediate state and the second intermediate state to obtain a first operation result. The first operation result is used to represent the importance of the first intermediate state in generating the second intermediate state. The first inference result is obtained by analyzing the first calculation result and the second intermediate state using the second small language model; Based on the first reasoning result, the parameters of the combined model are trained to obtain the trained combined model. The trained combined model is used to reason based on the input text to obtain the question-answer pair data of the input text.

2. The method according to claim 1, characterized in that, The combined model also includes a third minor language model, and the method further includes: The updated second intermediate state and third intermediate state are obtained. The updated second intermediate state is obtained by the second intermediate layer updating the second intermediate state according to the first calculation result. The third intermediate state is the output result obtained by the third intermediate layer according to the third input data. The third intermediate layer is the intermediate layer of the third small language model. The third input data is obtained according to the second input data. A cross-attention operation is performed on the updated second intermediate state and the third intermediate state to obtain a second operation result, which is used to represent the importance of the updated second intermediate state in generating the third intermediate state. The second inference result is obtained by analyzing the second operation result and the third intermediate state through the third small language model.

3. The method according to claim 2, characterized in that, The step of training the parameters of the combined model based on the first inference result to obtain the trained combined model includes: Based on the first reasoning result and the second reasoning result, the parameters of the combined model are trained to obtain the trained combined model.

4. The method according to claim 3, characterized in that, The step of training the parameters of the combined model based on the first inference result and the second inference result to obtain the trained combined model includes: Based on the error between the first inference result and the first actual result, and the error between the second inference result and the second actual result, the parameters of the combined model are trained to obtain the trained combined model.

5. The method according to any one of claims 1 to 4, characterized in that, Before obtaining the second intermediate state, the method further includes: The first small language model is used to reason about the first input data to obtain a third reasoning result; The third inference result and the first input data are used as the second input data.

6. The method according to any one of claims 1 to 5, characterized in that, The step of analyzing the first computation result and the second intermediate state through the second small language model to obtain the first reasoning result includes: Based on the first calculation result, the second intermediate state is updated through the second intermediate layer of the second small language model to obtain the updated second intermediate state; The updated second intermediate state is analyzed by the output layer of the second small language model to obtain the first inference result.

7. A model reasoning method, characterized in that, include: Get the input text; The input text is inferred by the trained ensemble model to obtain question-answer pairs, which include questions and answers to the questions. The trained ensemble model includes at least two small language models.

8. The method according to claim 7, characterized in that, The at least two small language models include a first small language model and a second small language model. The step of reasoning about the input text using a trained combined model to obtain question-answer pair data includes: The question is obtained by reasoning through the input text using the first small language model; The second small language model is used to reason about the question and the input text to obtain the answer to the question.

9. The method according to claim 8, characterized in that, Before the step of reasoning about the question and the input text using the second small language model to obtain the answer to the question, the method further includes: Obtain a first intermediate result and a second intermediate result. The first intermediate result is the output result obtained by the first intermediate layer based on the input text. The first intermediate layer is the intermediate layer of the first small language model. The second intermediate result is the output result obtained by the second intermediate layer based on the question and the input text. The second intermediate layer is the intermediate layer of the second small language model. Based on the cross-attention mechanism, cross-attention operation is performed on the first intermediate result and the second intermediate result to obtain an intermediate operation result. The intermediate operation result is used to represent the importance of the first intermediate result in generating the second intermediate result.

10. The method according to claim 9, characterized in that, The step of reasoning about the question and the input text using the second small language model to obtain the answer to the question includes: Based on the intermediate calculation results, the second intermediate results are updated through the intermediate layer of the second small language model to obtain the updated second intermediate results; The updated second intermediate result is analyzed by the output layer of the second small language model to obtain the answer to the question.

11. The method according to any one of claims 7 to 10, characterized in that, The trained combined model is obtained by training the combined model using the methods described in claims 1 to 6.

12. A model training device, characterized in that, include: The acquisition module is used to acquire a first intermediate state, which is the output result obtained by the first intermediate layer based on the first input data. The first intermediate layer is the intermediate layer of the first small language model in the combined model, and the combined model includes at least two small language models. The acquisition module is further configured to acquire a second intermediate state, which is the output result obtained by the second intermediate layer based on the second input data. The second intermediate layer is the intermediate layer of the second small language model in the combined model, and the second input data is obtained based on the first input data. The computation module is used to perform cross-attention operations on the first intermediate state and the second intermediate state based on the cross-attention mechanism to obtain a first computation result, which is used to represent the importance of the first intermediate state in generating the second intermediate state. The analysis module is used to analyze the first calculation result and the second intermediate state through the second small language model to obtain the first reasoning result; The training module is used to train the parameters of the combined model based on the first inference result to obtain the trained combined model.

13. The apparatus according to claim 12, characterized in that, The combined model also includes a third small language model. The acquisition module is further configured to acquire the updated second intermediate state and the third intermediate state. The updated second intermediate state is obtained by the second intermediate layer updating the second intermediate state based on the first calculation result. The third intermediate state is the output result obtained by the third intermediate layer based on the third input data. The third intermediate layer is the intermediate layer of the third small language model. The third input data is obtained based on the second input data. The calculation module is further configured to perform cross-attention operation on the updated second intermediate state and the third intermediate state to obtain a second calculation result, which is used to represent the importance of the updated second intermediate state in generating the third intermediate state. The analysis module is further used to analyze the second operation result and the third intermediate state through the third small language model to obtain the second reasoning result.

14. The apparatus according to claim 13, characterized in that, The training module is specifically used for: Based on the first reasoning result and the second reasoning result, the parameters of the combined model are trained to obtain the trained combined model.

15. The apparatus according to claim 14, characterized in that, The training module is specifically used for: Based on the error between the first inference result and the first actual result, and the error between the second inference result and the second actual result, the parameters of the combined model are trained to obtain the trained combined model.

16. The apparatus according to any one of claims 12 to 15, characterized in that, Before obtaining the second intermediate state, the device further includes: The reasoning module is used to reason about the first input data through the first small language model to obtain a third reasoning result; The determining module is used to take the third inference result and the first input data as the second input data.

17. The apparatus according to any one of claims 12 to 16, characterized in that, The analysis module is specifically used for: Based on the first calculation result, the second intermediate state is updated through the second intermediate layer of the second small language model to obtain the updated second intermediate state; The updated second intermediate state is analyzed by the output layer of the second small language model to obtain the first inference result.

18. A model reasoning device, characterized in that, include: The acquisition module is used to acquire the input text; The reasoning module is used to reason about the input text through the trained ensemble model to obtain question-answer pairs, which include questions and answers to the questions. The trained ensemble model includes at least two small language models.

19. The apparatus according to claim 18, characterized in that, The at least two small language models include a first small language model and a second small language model, and the inference module is specifically used for: The question is obtained by reasoning through the input text using the first small language model; The second small language model is used to reason about the question and the input text to obtain the answer to the question.

20. The apparatus according to claim 19, characterized in that, Before the step of reasoning about the question and the input text using the second small language model to obtain the answer to the question, the aforementioned acquisition module is further configured to: Obtain a first intermediate result and a second intermediate result. The first intermediate result is the output result obtained by the first intermediate layer based on the input text. The first intermediate layer is the intermediate layer of the first small language model. The second intermediate result is the output result obtained by the second intermediate layer based on the question and the input text. The second intermediate layer is the intermediate layer of the second small language model. The device further includes: The computation module is specifically used to perform cross-attention operations on the first intermediate result and the second intermediate result based on the cross-attention mechanism to obtain an intermediate computation result, wherein the intermediate computation result is used to represent the importance of the first intermediate result in generating the second intermediate result.

21. The apparatus according to claim 20, characterized in that, The reasoning module is specifically used for: Based on the intermediate calculation results, the second intermediate results are updated through the intermediate layer of the second small language model to obtain the updated second intermediate results; The updated second intermediate result is analyzed by the output layer of the second small language model to obtain the answer to the question.

22. The apparatus according to any one of claims 18 to 21, characterized in that, The trained combined model is obtained by training the combined model using the methods described in claims 1 to 6.

23. A model training device, characterized in that, include: A processor and a memory, wherein the processor is coupled to the memory; The memory is used to store programs; The processor is configured to execute a program in the memory, causing the method as described in any one of claims 1 to 6 to be performed.

24. A model reasoning device, characterized in that, include: A processor and a memory, wherein the processor is coupled to the memory; The memory is used to store programs; The processor is configured to execute a program in the memory, such that the method described in any one of claims 7 to 11 is performed.

25. A computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the method as claimed in any one of claims 1 to 6 or 7 to 11.

26. A computer program product comprising instructions that, when run on a computer, cause the computer to perform the method as described in any one of claims 1 to 6 or 7 to 11.

Images