Data processing method and device, equipment, storage medium and product

Abstract

The invention provides a data processing method and device, equipment, a storage medium and a product, and the method comprises the steps: obtaining a reasoning path of a reference model for generating a reference answer based on a training question, and a reasoning path of a to-be-trained model for generating a reasoning answer based on the training question, and based on the reference model and the to-be-trained model, respectively executing the attention information calculated in the respective reasoning paths, and determining the attention intensity corresponding to the respective reasoning paths. And training the to-be-trained model based on the difference between the reference answer and the reasoning answer and the difference between the attention intensities of the respective reasoning paths of the reference model and the to-be-trained model for the training question to obtain a target model. According to the technical scheme, the attention intensity of the influence degree of the reasoning step on the answer is represented by aligning, so that the to-be-trained model learns the reasoning step playing a key role in the reasoning process, the operation and processing of the redundant reasoning step are reduced, the model reasoning efficiency is improved, and meanwhile, the calculation overhead is reduced.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a data processing method, apparatus, device, storage medium, and product. Background Technology

[0002] With the rapid development of artificial intelligence technology, Large Language Models (LLMs), with their powerful reasoning capabilities, are widely used in various scenarios such as text understanding, question answering, and complex logic analysis. LLMs can typically perform multi-stage reasoning processing on input information and generate relatively accurate reasoning results. However, in practical applications, the reasoning process of LLMs often relies on the computation of a large number of model parameters and multiple consecutive reasoning steps, resulting in a long response time and low reasoning efficiency.

[0003] Against this backdrop, knowledge distillation technology has emerged. It refers to transferring complex knowledge from a large model to a smaller model to achieve reasoning capabilities close to those of the large model with a smaller parameter scale, thereby reducing the model's computational complexity and deployment costs. In related technologies, the output of the large model is typically used as the learning target, guiding the output of the small model to be as close as possible to that of the large model. While this approach can compress the model size to some extent, it only mimics the reasoning results; the reasoning process itself still requires a relatively complex inference procedure, resulting in low reasoning efficiency.

[0004] Therefore, how to train small models based on knowledge distillation to improve their reasoning efficiency in multi-step reasoning tasks has become a pressing technical problem. Summary of the Invention

[0005] This application provides a data processing method, apparatus, device, storage medium, and product. By aligning the attention intensity of two models to characterize the degree of influence of reasoning steps, the model to be trained can learn the reasoning steps that have a decisive influence on the answer during the reasoning process, reducing the computation and processing of redundant reasoning steps. This allows the model to be trained to execute more focused and streamlined reasoning steps, which is beneficial to improving the model's reasoning efficiency, increasing the reasoning speed of the target model in question-and-answer scenarios, and reducing unnecessary computational overhead.

[0006] In a first aspect, embodiments of this application provide a data processing method, the method comprising: Obtain the first inference path of the reference model generating a reference answer based on the training question, and obtain the second inference path of the model to be trained generating an inference answer based on the training question; Based on the first attention information calculated by the reference model during the execution of the first inference path, the first attention intensity corresponding to the first inference path is determined. The first attention intensity is used to represent the degree of influence of each inference step in the first inference path on the reference answer. Based on the second attention information calculated by the model to be trained during the execution of the second inference path, the second attention intensity corresponding to the second inference path is determined. The second attention intensity is used to represent the degree of influence of each inference step in the second inference path on the inference answer. Based on the difference between the reference answer and the inferred answer, and the difference between the first attention intensity and the second attention intensity, the model to be trained is trained to obtain the target model.

[0007] Secondly, embodiments of this application provide a data processing apparatus, which includes: The acquisition unit is used to acquire a first inference path of the reference model generating a reference answer based on the training question, and to acquire a second inference path of the model to be trained generating an inference answer based on the training question; The determining unit is used to determine the first attention intensity corresponding to the first inference path based on the first attention information calculated by the reference model during the execution of the first inference path. The first attention intensity is used to represent the degree of influence of each inference step in the first inference path on the reference answer. The determining unit is further configured to determine the second attention intensity corresponding to the second inference path based on the second attention information calculated by the model to be trained during the execution of the second inference path. The second attention intensity is used to represent the degree of influence of each inference step in the second inference path on the inference answer. The training unit is used to train the model to be trained based on the difference between the reference answer and the inferred answer, and the difference between the first attention intensity and the second attention intensity, to obtain the target model.

[0008] Thirdly, embodiments of this application provide an electronic device, which includes one or more processors; and a memory for storing one or more computer programs, wherein when the one or more computer programs are executed by the one or more processors, the electronic device implements the data processing method of the first aspect described above.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the data processing method of the first aspect described above.

[0010] Fifthly, embodiments of this application provide a computer program product, which includes a computer program or computer instructions, and when the computer program or computer instructions are executed by a processor, they implement the data processing method as described in the first aspect.

[0011] In some embodiments of this application, the technical solutions involve obtaining a first inference path from a reference model to generate a reference answer based on a training question, and obtaining a second inference path from a model to be trained to generate an inferred answer based on a training question. Then, based on the first attention information computed by the reference model during the execution of the first inference path, a first attention intensity corresponding to the first inference path can be determined. Furthermore, based on the second attention information computed by the model to be trained during the execution of the second inference path, a second attention intensity corresponding to the second inference path can be determined. These two attention intensities can represent the degree of influence of each inference step in the inference path on obtaining its respective answer. Finally, based on the differences between the reference answer and the inferred answer, and the differences between the first and second attention intensities, the model to be trained is trained to obtain the target model.

[0012] This demonstrates that, under the same training problem, the reference model allocates more attention to more useful reasoning steps in the reasoning path. If the model to be trained disperses its attention to non-critical steps, its attention structure will deviate from that of the reference model, resulting in additional loss. By using the difference between the first and second attention intensities as the training objective, we can strengthen the model to allocate more attention to key reasoning steps. This helps the model to be trained to mimic the decisive reasoning steps and decision-making logic in the reference model, rather than just imitating the final answer. This approach helps the model to be trained to establish a more efficient reasoning path while retaining its core reasoning capabilities.

[0013] Furthermore, since the reference model devotes less attention to redundant reasoning steps such as backtracking and adjusting answers, by aligning the final answers output by the two models and the attention intensity used to represent the differences in reasoning steps, the model to be trained can not only learn to output the correct output, but also learn to reduce the computation and processing of redundant reasoning steps that are not emphasized by the reference model. This makes the reasoning steps more focused and direct, and while retaining the core reasoning steps, it simplifies redundant reasoning steps, which is conducive to improving reasoning efficiency and reducing unnecessary computational overhead to a certain extent.

[0014] Furthermore, by using the trained target model to reason about the input question, after executing the simplified reasoning process, not only can an accurate answer be obtained, but compared with the reference model answering the question, the reasoning speed can also be improved and the response time of the reasoning process can be reduced. While ensuring the accuracy of the answer to the input question, the efficiency of reasoning can be improved. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a data processing method provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the principle of aligned attention weights provided in an embodiment of this application; Figure 4 This is a comparative diagram of the inference process performed by a reference model and a target model, as provided in an embodiment of this application. Figure 5 This is an architecture diagram of knowledge distillation provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a computer system for an electronic device provided in an embodiment of this application. Detailed Implementation

[0017] It should be noted in advance that, in order to enable those skilled in the art to better understand the technical solutions proposed in the embodiments of this application, the embodiments of this application will be described clearly and completely in conjunction with one or more accompanying drawings. Furthermore, the various drawings shown in the embodiments of this application are merely illustrative examples; for example, the execution order of each step in the drawings can be adaptively adjusted according to the actual application scenario. In addition, in the embodiments of this application, the block diagrams shown in the various drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0018] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0019] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0020] Knowledge distillation (KD) is a model training method, also known as model compression. Its core idea is to transfer the reasoning results of a large model with powerful performance (also known as a teacher model) to a model with a simpler structure and smaller parameter size (also known as a student model), so that the small model can obtain reasoning capabilities close to those of the large model with less computational overhead.

[0021] In related technologies, distillation methods typically reduce the computational cost and burden of the model during the inference phase by transferring the inference results of a large model to a small model. For example, the output of the large model can be directly used as the training target to guide the small model to imitate the behavior of the large model. While this approach can improve the performance of the small model and is easy to implement and deploy, it only focuses on the consistency of the inference results and ignores the optimization of the inference process itself. This results in many unnecessary inference steps being copied to the small model, leading to lower inference efficiency in the small model and wasting computational resources, thus failing to fully utilize the advantages of the small model.

[0022] Specifically, the reasoning process of large models generating the final output is complex and lengthy, typically involving multiple reasoning steps and a large number of inference operations. Within the reasoning path of a large model, there may be intermediate reasoning steps that have little impact on the result or can be simplified. For example, many reasoning steps are simply to solve detailed problems, or to perform backtracking, exploratory attempts, and corrective actions. These reasoning steps do not play a decisive role in forming the final answer, but they consume a significant amount of computational resources. Therefore, knowledge distillation techniques in related technologies merely replicate model behavior without in-depth analysis of the reasoning process. That is, they do not adequately focus on streamlining redundant reasoning steps in the reasoning path, nor do they deeply analyze and optimize the reasoning steps that have a decisive impact on the reasoning result. This results in smaller models still executing some irrelevant reasoning steps when answering certain questions, affecting the reasoning efficiency of smaller models and increasing computational costs, especially in complex tasks with multi-step reasoning or requiring a large amount of computation, failing to achieve true reasoning simplification.

[0023] Based on this, this application provides a data processing method. This method can obtain the answers and inference paths of a large model and a small model for the same training question, and determine the attention intensity corresponding to each inference path of the two models based on the attention information of the two models in executing their respective inference paths. Since the attention information calculated by the model during the inference process can represent the degree of attention between each inference step and other inference steps in the model, the attention intensity determined based on the attention information can indicate the criticality of each inference step in the inference path, which is also the degree of influence on the final answer. The model to be trained uses the answer and attention intensity output by the reference model as the training target, which can help the small model imitate the inference behavior of the large model. While learning the correct output, it optimizes its own inference process, thereby improving the inference speed and efficiency of the model by simplifying redundant inference steps, thus achieving inference distillation. When the small model is applied to question answering platforms such as text and image platforms, it can improve the speed of answering questions while ensuring the accuracy of the small model's answers to user input questions, thereby improving the platform's response speed.

[0024] Based on the above description, please refer to Figure 1 , Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application, such as... Figure 1 As shown, the implementation environment includes a model training device 101 and a model deployment device 102, which can be directly or indirectly connected via wired or wireless means. It should be noted that... Figure 1 The number and form of the devices shown are for illustrative purposes only and do not constitute a limitation on the embodiments of this application.

[0025] In some embodiments, there can be multiple model training devices 101, which can be deployed to perform different tasks, including reference models and training models. The training models are trained using knowledge distillation to obtain the target models corresponding to each training model.

[0026] In some embodiments, the number of model deployment devices 102 can also be multiple, and the target models trained by the model training device 101 can be deployed separately. The number of target models can be one or multiple. When there are multiple target models, they can be used to perform different tasks. Multiple model deployment devices 102 can deploy target models for performing the same or different tasks respectively.

[0027] In some embodiments, the model training device 101 and the model deployment device 102 can be the same electronic device, that is, the electronic device can not only train the model to be trained, but also deploy the trained target model for the inference stage of the model, such as the stage of generating answers based on input questions.

[0028] The model training device 101 can be an electronic device used for model training based on knowledge distillation. It can be a high-performance electronic device capable of deploying a reference model with a large parameter scale and requiring significant computational resources for inference operations, such as an LLM. The model training device 101 can also deploy a lightweight model to be trained (which can also be an LLM), with a smaller parameter scale and requiring fewer computational resources for inference compared to the reference model. The model training device 101 can train the model to be trained by running the deployed reference model and the model to be trained through knowledge distillation, thereby obtaining the target model.

[0029] The model deployment device 102 can be an electronic device used by a user, or other electronic devices used to perform specific reasoning tasks, and its hardware performance is weaker than that of the model training device 101. The model deployment device 102 can be a device specifically used to perform reasoning tasks, such as question-answering tasks, which can reason based on the input question to generate the corresponding answer. Since the target model consumes fewer computational resources during reasoning, deploying the target model on the less powerful model deployment device 102 allows the model deployment device 102 to utilize the target model to output the correct answer based on its learned reasoning ability with less resource consumption. Optionally, the video playback device 102 may also include input devices, such as a keyboard or a voice acquisition device, to collect questions for the reasoning task.

[0030] The data processing method provided in this application can be executed by the model training device 101. The model training device 101 and the model deployment device 102 can be terminal devices or servers. Terminal devices include, but are not limited to, smartphones (such as Android phones, iOS phones, etc.), tablet computers, portable personal computers, mobile internet devices (MIDs), smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, wearable devices, etc., and this application embodiment does not limit this. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, and this application embodiment does not limit this.

[0031] The general flow of the data processing method provided in this application is as follows: The model training device 101 can acquire a first inference path from a reference model to generate a reference answer based on a training question, and a second inference path from a model to be trained to generate an inference answer based on a training question. Furthermore, the model training device 101 can determine the first attention intensity corresponding to the first inference path based on the first attention information computed by the reference model during the execution of the first inference path. This first attention intensity can represent the degree of influence of each inference step in the first inference path on the reference answer. Similarly, the model training device 101 can determine the second attention intensity corresponding to the second inference path based on the second attention information computed by the model to be trained during the execution of the second inference path. This second attention intensity can represent the degree of influence of each inference step in the second inference path on the inference answer. Therefore, the model training device 101 can train the model to be trained based on the differences between the reference answer and the inference answer, as well as the differences between the first attention intensity and the second attention intensity, to obtain the target model.

[0032] In some embodiments, the model training device 101 can send the target model to the model deployment device 102, such as sending the model parameter file of the target model to the model deployment device 102, so that the model deployment device 102 can deploy the target model.

[0033] In some embodiments, the model deployment device 102 may also receive a question from a reasoning task and input it into a target model so that the target model can output the answer to the question.

[0034] It is understood that the embodiments described in this application are as follows: Figure 1 The implementation environment shown is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0035] Based on the above-described implementation environment, this application provides a data processing method. The data processing method described in this application can be executed by an electronic device, which may be... Figure 1 The model training device 101 in the implementation environment shown can be the same electronic device as the model deployment device 102. Please refer to... Figure 2 , Figure 2 This is a flowchart illustrating a data processing method provided in an embodiment of this application. The data processing method includes the following steps S201-S204: S201. Obtain the first inference path of the reference model generating the reference answer based on the training question, and obtain the second inference path of the model to be trained generating the inference answer based on the training question.

[0036] In this embodiment, the reference model refers to a pre-trained machine learning model with strong reasoning capabilities, such as an LLM (Limited Machine Model). It typically has a large parameter size and consumes significant computational resources during inference operations. In knowledge distillation, the reference model can be understood as a teacher model, a large model, etc., providing reference information regarding inference results or processes. The model to be trained refers to a model whose parameters need adjustment. Its parameter size is smaller (far smaller) than the reference model, consuming fewer computational resources during inference operations. In knowledge distillation, the model to be trained can be understood as a student model, a small model, etc., or it can be an LLM. It can mimic the output and inference process of the reference model to achieve reasoning capabilities close to the reference model while maintaining low computational complexity.

[0037] The reference model and the model to be trained can be models used to perform question-answering tasks. They can reason based on the input question and generate corresponding answers. The training question can be an input question used for model training. It can be a question description in text, image, or other forms, which can trigger the model to perform reasoning and generate corresponding answers. The answers can also be in text, image, or other forms, and this application does not limit this. For example, a text-based training question could be "If a number is even and greater than 10, is it possible that it is a prime number? Please give your reasoning." The reference answer refers to the correct answer output by the reference model for the training question, serving as a target reference during the training process of the model to be trained. For example, for the above training question, the reference model might generate the reference answer as "Impossible. Because all even numbers except 2 are divisible by 2, and even numbers greater than 10 cannot be equal to 2, therefore it cannot be a prime number." The reasoning answer refers to the answer output by the model to be trained for the same training question. Its accuracy is unstable and needs to be optimized.

[0038] Specifically, the reasoning path refers to the reasoning steps and decision-making process that a model undergoes from a question to an answer in a reasoning task. This reasoning path can be likened to a Chain of Thought (CoT) and can include sentences describing the reasoning steps performed by the model during the reasoning process. These reasoning step sentences can be used to illustrate the model's decision-making process. Optionally, the reasoning path may also include each intermediate result obtained by the model in performing the reasoning steps.

[0039] It should be noted that, since the inference steps in the inference path of the reference model may have causal relationships and interdependencies, each step may affect subsequent steps. In the knowledge distillation process of this application, the inference path executed by the reference model is analyzed to identify key and redundant inference steps that play a decisive role. This allows the model to be trained to optimize its inference path, such as focusing more on key inference steps and reducing the imitation of redundant inference steps, thereby simplifying the inference path of the model to be trained, improving the number of inference steps, and fundamentally reducing the computational resources consumed in the inference process.

[0040] It is understood that the first inference path is the inference path of the reference model in generating the reference answer, and the second inference path is the inference path of the model to be trained in generating the inference answer. The terms "first" and "second" are used to distinguish them. For example, the first inference path can be seen in Table 1:

[0041] Table 1 In one possible implementation, the training problem can be obtained through manual construction, internet collection, or generation by a machine learning model, or other methods, depending on the business scenario; this application does not limit this. The first and second inference paths can be generated by a reference model and a model to be trained during a step-by-step inference process based on the training problem. Specifically, in obtaining the first and second inference paths, the electronic device can input the training problem into the reference model to perform inference operations based on the training problem, thereby obtaining a first inference path and a reference answer, including at least two inference steps. Similarly, the electronic device can input the training problem into the model to be trained to perform inference operations based on the training problem, thereby obtaining a second inference path and a reasoning answer, including at least two inference steps.

[0042] In other words, for the same training problem, an electronic device can provide the training problem as input data to both a reference model and a model to be trained, allowing both models to perform inference operations. This enables each model to not only generate an answer but also a reasoning process containing sentences with multiple inference steps. This provides a foundation for subsequent training of the model to be trained based on the differences in their results and the reasoning processes. For example, the first inference path output by the reference model may include a series of inference steps, which can be denoted as... , , ..., , This represents the total number of reasoning steps.

[0043] In some embodiments, the electronic device inputs a training question into two models and provides instructions to the models to explicitly generate inference paths, thereby instructing them to output not only the reference answer but also the inference path. For example, the text instruction input to the model could be, "Please proceed step-by-step based on the question and output the inference answer, with the output format including the inference steps Step 1, Step 2, ... and the result obtained by performing each inference step." Thus, the reference model can generate inference paths as shown in Table 1.

[0044] S202. Based on the first attention information calculated by the reference model during the execution of the first inference path, determine the first attention intensity corresponding to the first inference path.

[0045] In this embodiment, attention information can refer to a set of data obtained by the reference model during the execution of the first inference path, used to characterize the relationship between information units during the inference process. It is expressed in matrix form, which can be called an attention matrix. The attention matrix can also be called an attention weight matrix. Each matrix element in the attention matrix is ​​an attention weight. Each attention weight can refer to the numerical value of the degree of attention (also called dependence or attention intensity) between one inference step and another inference step.

[0046] Specifically, the model can include multiple (at least two) attention modules, also known as attention heads. An attention head is a computational unit in the model that can assign different levels of attention to different inference steps based on the inference context. Each attention head calculates a set of attention weights to form an attention weight matrix. Therefore, the attention information includes the attention matrix formed by the attention weights calculated by each attention head in the reference model, i.e., at least two attention matrices. Each attention matrix can include the attention weights between any two inference steps in the first inference path. For example, the attention matrices can be seen in Table 2:

[0047] Table 2 Table 2 uses a 3×3 attention matrix as an example for explanation. The row and column dimensions of the matrix represent the number of inference steps, i.e., three inference steps S1, S2, and S3. The specific values ​​in Table 2 are for illustrative purposes only and do not limit the values ​​of the attention weights. In this attention matrix, the matrix elements (i.e., attention weights) It can represent the steps of reasoning. To the reasoning steps Attention weights, which are the inference steps Reasoning steps The level of attention paid to a particular factor can reflect the reasoning steps. Reasoning steps The degree of influence. For example, the attention weight in row 3, column 2. The attention weight from reasoning step S2 to reasoning step S3 is 0.25, which means that the degree of attention (dependence) of reasoning step S3 on reasoning step S2 is 0.25, and also reflects that the degree of influence of reasoning step S2 on reasoning step S3 is 0.25.

[0048] It should be noted that during the forward computation of the model, the operations on the inference steps are usually at the character (token) level, and the resulting attention matrix is ​​also at the character (token) level. Electronic devices can aggregate the attention weights in the attention matrix based on the characters contained in each inference step (sentence). For example, they can aggregate all the character-level attention weights associated with a certain inference step by summing, averaging, etc., to obtain the attention weights at the inference step (sentence) level, as shown in Table 2.

[0049] Attention intensity is determined based on attention weights, which can be obtained by summing and statistically analyzing multiple attention weights, such as the sum of the attention a certain reasoning step pays to other reasoning steps. Therefore, attention intensity can represent the total influence of a reasoning step in the reasoning path (such as the total influence on subsequent steps), or it can be understood as the strength of the influence of that reasoning step on the answer. For example, if a certain reasoning step has a significant impact on the reasoning of subsequent steps, its attention weight is high, indicating that the degree of influence of that reasoning step on the answer is also high.

[0050] Understandably, to distinguish the attention information and determined attention intensity calculated by the reference model and the model to be trained, this application defines the attention information calculated by the reference model as the first attention information, and the attention information calculated by the model to be trained as the second attention information. The attention intensity determined based on the first attention information is defined as the first attention intensity, and the attention intensity determined based on the second attention information is defined as the second attention intensity. The attention matrix constructed from the attention weights calculated by the attention head in the reference model is defined as the first attention matrix, and the attention matrix constructed from the attention weights calculated by the attention head in the model to be trained is defined as the second attention matrix.

[0051] Specifically, in determining the first attention intensity corresponding to the first inference path based on the first attention information, partial attention weights between each inference step and subsequent inference steps can be extracted from at least two first attention matrices, and the sum of the extracted partial attention weights is determined as the first attention intensity. Here, each inference step can refer to every inference step in the first inference path, and subsequent inference steps can refer to other inference steps following that inference step in the first inference path. For example, after S1 there are two subsequent inference steps, S2 and S3, and after S2 there is one subsequent inference step, S3. Partial attention weights refer to the attention weights between each inference step and the inference steps following that inference step, and do not include the attention weights between each inference step and the inference steps preceding that inference step.

[0052] Therefore, the extracted attention weights are summed, and the sum is determined as the first attention intensity. For example, the calculation method for this first attention intensity is shown in Formula 1: Formula 1 In Formula 1, Representing the reasoning steps The influence of attention, also known as the overall attention level, can represent the degree of attention each reasoning step receives from subsequent reasoning steps, or the degree of influence each reasoning step has on subsequent reasoning steps, that is, the degree of influence each reasoning step has on the reference answer. This refers to the first in the reference model Each attention module (attention head) This refers to the total number of attention heads in the reference model. The index of the attention head is greater than 1. The value of can indicate the reasoning step. Subsequent reasoning steps . This refers to the number of reasoning steps contained in the first reasoning path, i.e. One reasoning step, Indicating the first in the reference model In each attention head, reasoning steps To the reasoning steps By summing these partial attention weights, we can obtain the degree of influence of this reasoning step on subsequent reasoning steps. For example, in Table 2, the attentional influence of step S1 can be... + =0.5+0.2=0.7.

[0053] It should be noted that conventional attention impact analysis typically only counts a single attention head or the attention head of a single network layer. However, the embodiments in this application not only count all attention heads in the model but also calculate their cumulative impact on subsequent inference steps. This calculation method can identify inference steps that have a significant impact on subsequent inference steps, and these inference steps are the key inference steps in the first inference path.

[0054] In some embodiments, the attention weights in the first attention matrix calculated by multiple attention heads in the reference model can be calculated by the reference model during forward inference. After inference is completed, i.e. after the reference model obtains the reference answer, the attention weights calculated by each network layer and each attention head during the forward inference process can be uniformly obtained and summarized through relevant interfaces.

[0055] In some embodiments, multiple attention heads of the reference model each correspond to a weight parameter, which can be model parameters learned and determined by the reference model during training based on its performance in training data. The weight parameter corresponding to each attention head can represent the contribution of that attention head to the inference task. Therefore, when calculating the influence of attention, a weighted summation can be performed on the partial attention based on the weights corresponding to each attention head, as shown in Equation 2: Formula 2 In Formula 2, unlike Formula 1, when summing the partial attention weights calculated for each attention head, the weight parameter corresponding to that attention head is also added. This calculation method can be understood as a weighted attention head. Therefore, the attention influence calculated based on Formula 2 can be used to further refine the attention analysis.

[0056] It is understandable that the first attention intensity corresponding to the first reasoning path calculated based on Formulas 1 and 2 includes the total attention paid to that reasoning step by subsequent reasoning steps within the first reasoning path. Therefore, this first attention intensity can also represent the degree of influence of each reasoning step in the first reasoning path on the reference answer. Based on the calculated attention influence, the key reasoning steps in the first reasoning path can be identified. For example, by sorting the attention influence values ​​of each reasoning step from high to low, one or more of the highest-ranking reasoning steps can be identified as key reasoning steps in the reasoning process.

[0057] This method of analyzing key reasoning steps in the first reasoning path by extracting attention weights calculated from attention heads in the model can be called "white-box analysis." This refers to a method that directly utilizes observable intermediate computational information within the model during its operation to analyze the model's reasoning process, decision-making basis, or internal behavior. Here, intermediate computational information refers to the attention weights, which, according to Formulas 1 and 2 and the attention weights, can calculate the attentional impact of the reasoning steps, quantifying the importance of each reasoning step in the reasoning path. In this application, key reasoning steps with high attentional impact can also be understood as "reasoning anchors" that significantly influence the final reference answer.

[0058] It should be noted that each attention head in the reference model views the training problem from a single "perspective." For example, some attention heads focus on the grammar of the training problem, while others focus on certain specific inference steps. Therefore, during white-box analysis, we can first identify the attention heads in the reference model that focus on certain inference steps. These attention heads may be the ones driving the inference process, and we can then precisely analyze the inference steps they focus on from these identified attention heads.

[0059] In one possible implementation, the electronic device can select a target attention module from at least two attention modules. Then, the electronic device can extract partial attention weights between each inference step and subsequent inference steps from the first attention matrix obtained from the target attention module, and determine the sum of the extracted partial attention weights as the first attention intensity. Here, the target attention module can refer to an attention head with inference significance from the attention heads of the reference model. This attention head can also be called a receiver head, which refers to the attention head that the reference model focuses on during the inference process for a specific inference step. The process of selecting a receiver head from all attention heads of the reference model can also be understood as the process of identifying the receiver head.

[0060] Understandably, because the receiver head focuses on certain specific inference steps during the inference process, the attention weights for those specific inference steps in the first attention matrix calculated by the receiver head are relatively large. Therefore, a weight threshold can be set to select a target attention head (receiver head) from the attention heads. The target attention head contains attention weights greater than the set weight threshold. Furthermore, partial attention weights between each inference step and its subsequent inference steps are extracted from the first attention matrix calculated by the receiver head, and the sum of the extracted partial attention weights is calculated to obtain the first attention intensity.

[0061] Specifically, when selecting a target attention module, electronic devices can use the kurtosis metric to identify attention weights that are prominent (unevenly distributed). Kurtosis is a statistical measure of whether a distribution has sharp peaks (a few extreme values). The kurtosis of an attention head measures the concentration of its attention weights. It indicates whether an attention head's attention is "highly concentrated on a few inference steps" or evenly distributed across most sentences. A high kurtosis value indicates that a few inference steps are drawing most of the attention of the attention head, and the attention weights associated with these few inference steps are greater than a set weight threshold; it also indicates that the attention head is focused on these inference steps. A low kurtosis value indicates that the attention head's attention is more evenly distributed, with no obvious focus, and all attention weights are less than a set weight threshold.

[0062] Specifically, the electronic device can determine the module kurtosis value corresponding to each attention module based on the established mapping relationship between the attention weights in the attention matrix and the module kurtosis value of the attention module, as well as the attention weights in each first attention matrix. Then, the electronic device can extract the attention modules whose module kurtosis value is greater than a specified kurtosis threshold from at least two attention modules to obtain the target attention module.

[0063] For example, the mapping relationship between the attention weights in the attention matrix and the module kurtosis value of the attention module can be shown in Equation 3: Formula 3 It should be noted that Formula 3 represents the calculation method for one attention head, and the calculation method is the same for each attention head. In Formula 3, This refers to the kurtosis value of the attention head, i.e., the module kurtosis value. and Let be the sequence number of any two reasoning steps. It refers to any two reasoning steps (reasoning steps) To the reasoning steps The mean of the fourth power of the attention weights, It refers to any two reasoning steps (reasoning steps) To the reasoning steps The square of the attention weight. As can be seen from Formula 3, if a certain attention head pays more attention to some reasoning steps, its kurtosis will increase after the fourth power, which will increase the numerator and thus increase the kurtosis value of the attention head, indicating that the attention head focuses on some reasoning steps.

[0064] Therefore, based on Formula 3, the module kurtosis value corresponding to each attention head in the reference model can be calculated. Then, based on a specified kurtosis threshold, target attention modules with kurtosis values ​​greater than that threshold can be selected, thus obtaining the receiver heads. In other words, if the kurtosis value of an attention head is significantly higher than that of other heads, it is considered a receiver head, indicating that this attention head focuses on certain inference steps during the inference process, and these inference steps may guide the subsequent inference results.

[0065] In one possible implementation, the specified kurtosis threshold can be a static threshold, such as a pre-set value, or it can be a dynamic threshold, which can be used to adaptively adjust for different tasks during training, making the receiver head's recognition more flexible. Specifically, the electronic device can calculate the mean kurtosis value and the standard deviation of the kurtosis value based on the module kurtosis values ​​corresponding to at least two attention modules. Then, based on the correlation between the kurtosis mean, the standard deviation of kurtosis, and the kurtosis threshold, as well as the mean kurtosis value and the standard deviation of kurtosis value, a kurtosis threshold associated with the mean kurtosis value and the standard deviation of kurtosis value is determined, thus obtaining the specified kurtosis threshold.

[0066] Here, the mean kurtosis value can refer to the mean of the module kurtosis values ​​corresponding to all attention heads in the reference model, and the standard deviation of the kurtosis value can refer to the standard deviation of the module kurtosis values ​​corresponding to all attention heads in the reference model. The specified kurtosis threshold can be calculated based on the correlation between the mean kurtosis value, the standard deviation of the kurtosis value, and the kurtosis threshold. For example, the method of determining the specified kurtosis threshold based on this correlation can be seen in Formula 4: Formula 4 In formula 4, This indicates a dynamic threshold, specifically a specified kurtosis threshold. This represents the mean kurtosis, which is the calculated average kurtosis value. This represents the standard deviation of kurtosis, which is the calculated standard deviation of the kurtosis value. It is an adjustable parameter. As can be seen, the specified kurtosis threshold can be calculated using Formula 4, and then important attention heads, i.e., receiver heads, that are greater than the specified kurtosis threshold can be selected from at least two attention heads included in the reference model.

[0067] Furthermore, the electronic device can extract only a portion of the attention weights between each inference step and subsequent inference steps from the selected receiver head (target attention head module), and determine the sum of the extracted partial attention weights as the first attention intensity. For example, the calculation of this first attention intensity can be seen in Formula 5: Formula 5 In Formula 5, This refers to the reasoning steps The attentional influence can represent the degree of attention each inference step receives from subsequent inference steps in the receiver head, or the degree of influence each inference step has on subsequent inference steps in the receiver head; it can also be understood as the broadcast influence. The degree to which each reasoning step affects the reference answer, from the receiver's perspective. Unlike Formulas 1 and 2, This refers to the first in the reference model Individual attention modules (receiver heads). This refers to the total number of target attention heads in the reference model. The index of the target attention head is greater than 1. The value of can indicate the reasoning step. Subsequent reasoning steps . This refers to the number of reasoning steps contained in the first reasoning path, i.e. One reasoning step, Indicating the first in the reference model In the receiver header, the reasoning steps To the reasoning steps By summing up the partial attention weights, we can obtain the degree of influence of this inference step on subsequent inference steps in the receiver head, and then use this degree of influence as the first attention intensity.

[0068] Furthermore, after determining the first attention intensity, the key reasoning steps in the first reasoning path can be identified based on the calculated attention influence (broadcast influence). For example, the broadcast influence of each reasoning step can be sorted from high to low, so that one or more of the highest-ranking reasoning steps are identified as key reasoning steps in the reasoning process.

[0069] Understandably, by first identifying attention heads that show high interest in certain reasoning steps—that is, identifying receiver heads—key reasoning steps can be identified by the degree of attention they receive from subsequent reasoning steps. This degree of attention from multiple subsequent reasoning steps can be called the broadcast influence of that reasoning step. The key reasoning steps selected based on Formula 5 can be called "broadcast reasoning steps" or "broadcast sentences."

[0070] Therefore, key reasoning steps identified by the receiver head can have high broadcast influence and receive significant attention from subsequent reasoning steps. These key reasoning steps are typically indispensable in the reasoning path, having a decisive impact on subsequent decisions and guiding and driving the reasoning process. Broadcast reasoning steps can also be understood as "mental anchors" in the reasoning process, that is, reasoning steps that play a crucial role in the core decisions of the reasoning path.

[0071] In some embodiments, in addition to considering the reasoning steps In addition to the degree of attention paid by subsequent inference steps, a time weight can be added, which can represent the change in the strength of the influence of each inference step on subsequent inference steps over time. That is, inference steps that appear earlier (the earlier in the inference path) and are frequently attended to usually have a greater influence on the inference process. For example, an error in an early inference step may lead to all the inference results in subsequent inference steps being wrong; therefore, these inference steps can be given higher weights. In the embodiments of this application, a time decay factor can be added to assign time weights to these steps, as shown in Formula 6: Formula 6 In Formula 6, Representing the reasoning steps Time weighting Representing the reasoning steps The time step of appearance, This represents the total number of reasoning time steps. For sentence-level reasoning, it could refer to the [number of time steps]. Each reasoning step, i.e., reasoning steps , for That is, the total number of reasoning steps is One. Understandably, This is a time decay factor, which decays exponentially; the earlier the time, the greater the time weight assigned. Therefore, the calculation method for the time-weighted first attention intensity can be seen in Formula 7: Formula 7 In Formula 7, Representing the reasoning steps By incorporating time-weighted attention influence (broadcast influence), obtained by multiplying Formula 5 and Formula 6, each inference step can be infused with time-weighted attention influence (broadcast influence) and determined as the first attention intensity.

[0072] S203. Based on the second attention information calculated by the model to be trained during the execution of the second inference path, determine the second attention intensity corresponding to the second inference path.

[0073] In this embodiment, the second attention information, similar to the first attention information, refers to the data set obtained by the model to be trained during the execution of the second inference path, used to characterize the relationship between information units during the inference process. It is also expressed in matrix form, representing the attention matrix calculated by at least two attention modules (attention heads) in the model to be trained. Similar to the first attention matrix, the attention matrix calculated by the model to be trained can be called the second attention matrix. Each matrix element in the second attention matrix is ​​an attention weight, and each attention weight can refer to the numerical value of the degree of attention between one inference step and another inference step in the second inference path.

[0074] In one possible implementation, when determining the second attention intensity corresponding to the second inference path, the electronic device can specifically extract partial attention weights between each inference step and subsequent inference steps from at least two second attention matrices, and determine the second attention intensity by summing the extracted partial attention weights.

[0075] The second attention intensity can include the influence of each reasoning step in the second reasoning path on the reasoning answer. The second attention intensity can be obtained by summing the partial attention weights between each reasoning step and subsequent reasoning steps in each second attention matrix, or by first identifying the receiver heads in the model to be trained, then extracting the partial attention weights between each reasoning step and subsequent reasoning steps from each receiver head, and finally summing them. The specific determination method is the same as that of the first attention intensity, and can be found in the specific description in S202, which will not be repeated here.

[0076] It should be noted that this method, which utilizes the model's attention mechanism to identify and analyze the degree of attention the model pays to specific reasoning steps during the reasoning process, and analyzes the causal dependencies between reasoning steps, is called the causal attribution method. The causal attribution method can not only provide a deeper understanding of the mutual influence between reasoning steps, but also identify key reasoning steps that have a decisive impact on subsequent reasoning steps; these key reasoning steps are the important nodes that drive the direction of reasoning.

[0077] S204. Based on the differences between the reference answer and the inferred answer, as well as the differences between the first attention intensity and the second attention intensity, the model to be trained is trained to obtain the target model.

[0078] In this embodiment, during the training of the model to be trained, the difference between the reference answer generated by the reference model and the inference answer generated by the model to be trained can be used to measure whether the reference answer and the inference answer are close and whether the inference answer is correct. Training the model to be trained based on the difference between the reference answer and the inference answer can ensure the correctness of the inference results of the trained target model during the inference stage. This method of training the model to be trained based on the difference between the reference answer and the inference answer can be called aligned output distribution.

[0079] The first attention intensity refers to the degree of influence of each reasoning step in the reference model on the reference answer within the first reasoning path, and it is also the degree of attention it receives from other reasoning steps. The second attention intensity refers to the degree of influence of each reasoning step in the model to be trained on the reasoning answer within the second reasoning path. Therefore, the difference between the first and second attention intensities can guide the model to be trained to learn the key reasoning steps that have a significant impact on the reference answer in the first reasoning path, as well as redundant reasoning steps. This allows the model to simplify or eliminate redundant reasoning paths, extracting the most concise and efficient reasoning path, thus enabling the model to mimic the behavior of the reference model and achieve approximate reasoning ability with lower computational cost.

[0080] Specifically, the electronic device can generate first loss data based on the difference between the reference answer and the inferred answer, and generate second loss data based on the difference between the first attention intensity and the second attention intensity. Then, the model to be trained is trained based on the sum of the first and second loss data to obtain the target model. The difference between the reference answer and the inferred answer can be the difference between two text answers, for example, it can be measured by the mean squared error (MSE) to obtain the first loss data.

[0081] In some embodiments, if the reference answer and the inferred answer are in other forms, such as images, the first loss data can also be generated based on the MSE between the reference answer image and the inferred answer image. The specific method can be determined based on the actual application scenario, and this application does not limit it.

[0082] The difference between the first attention intensity and the second attention intensity can be calculated by assessing the difference in attention influence per inference step. For example, the calculation method for the second loss data can be found in Formula 8: Formula 8 In Formula 8, This represents the second loss data, which can also be understood as the loss due to the impact of attention. This refers to the attentional impact corresponding to each inference step calculated by the model to be trained (student model). This refers to the attentional impact corresponding to each reasoning step, calculated by the reference model (teacher model). and Both can be represented by vectors, and both are... , It refers to the first The value of the attention impact of each inference step. That is, according to Formula 8, the attention impact of each corresponding step can be subtracted and then the sum of squares can be calculated to obtain the second loss data. This second loss data can be used by the model to be trained to learn which inference steps are important and which are not, thereby simplifying the inference steps.

[0083] Therefore, the electronic device can use the sum of the first loss data and the second loss data to train the model to be trained. The training loss data can be seen in Equation 9: Formula 9 In Formula 9, The first loss data is obtained based on the answers output by the two models. The attention-related loss, i.e., the second loss data, can be obtained based on Equation 8.

[0084] In one possible implementation, in addition to the two alignment objectives mentioned above, the electronic device can also employ other alignment objectives, such as attention weight differences, to ensure that the attention distribution of the model to be trained for inference steps is as close as possible to the reference model. Specifically, the electronic device can generate third loss data based on the differences between the first attention matrix calculated by at least two attention modules in the reference model and the second attention matrix calculated by at least two attention modules in the model to be trained. Then, these three losses are summed—that is, based on the sum of the first, second, and third loss data—to train the model to be trained and obtain the target model.

[0085] In the process of generating third loss data based on the differences between at least two first attention matrices and at least two second attention matrices, the electronic device can average the attention matrices of all attention heads in all layers of the model to obtain an attention mean matrix. Then, the attention mean matrices obtained from the two models are aligned to generate the third loss data.

[0086] In other words, the electronic device can calculate the average weight of each element position in at least two first attention matrices to obtain a first attention mean matrix. Similarly, the electronic device can calculate the average weight of each element position in at least two second attention matrices to obtain a second attention mean matrix. Then, the electronic device can generate third loss data based on the difference between the average weights corresponding to each element position in the first and second attention mean matrices.

[0087] Please refer to the following: Figure 3 , Figure 3 This is a schematic diagram illustrating the principle of aligned attention weights provided in an embodiment of this application, such as... Figure 3 As shown, the upper left corner is a schematic diagram of the attention heads included in the reference model. Each cube represents an attention head, and a first attention matrix can be calculated for each. The upper right corner is a schematic diagram of the attention heads included in the model to be trained. Each cube also represents an attention head, and a second attention matrix can be calculated for each. Each matrix element in each first and second attention matrix can represent any two inference steps. and Attention weights between them.

[0088] Therefore, the electronic device can calculate the average weight corresponding to each element position based on the attention weights at each element position in at least two first attention matrices, and obtain, as shown below. Figure 3The first attention mean matrix shown on the left represents the mean of all attention weights at that position, where each element is the average of all attention weights. Similarly, the electronic device can calculate the mean weight for each element position based on the attention weights at each position in at least two second attention matrices, resulting in... Figure 3 The second attention mean matrix shown on the right represents the mean of all attention weights at that position for each element. Then, third loss data can be generated based on the difference between these two mean matrices. For example, the calculation method for the third loss data can be found in Equation 10: Formula 10 In Formula 10, This represents the attention weight loss, also known as the third loss data. This refers to the value of each element in the second attention mean matrix. , This refers to the value of each element in the first attention mean matrix. . The norm is obtained by subtracting the elements at each position and then summing the squares; it can be equivalent to the expression... .

[0089] In other words, by considering the size of the reasoning steps... The means are aligned to compare the two attention mean matrices element-wise, making the attention distribution of the inference steps of the model to be trained as close as possible to the attention distribution of the reference model. Thus, the electronic device can use the sum of the three losses for the training process of the model to be trained; for example, this sum of the three losses can be seen in Equation 11: Formula 11 In Formula 11, The first loss data is obtained based on the answers output by the two models. The attention-related loss, i.e., the second loss data, can be obtained based on Equation 8. The attention weight loss, i.e. the third loss data, can be obtained based on Equation 10.

[0090] In some embodiments, the electronic device may also align the two mean matrices by element-wise MSE, or by element-wise Kullback-Leibler Divergence (KL) divergence, but this application does not limit this.

[0091] In another possible implementation, besides the two alignment targets mentioned above, the electronic device can also employ other alignment targets, such as sentence-level alignment of inference steps, to make the inference path of the model to be trained as close as possible to the simplified inference path of the reference model. Specifically, the electronic device can, without removing the constraints of any inference step in the first inference path, call the reference model to perform a set number of inference operations based on the training question to obtain the set number of operation answers. Then, based on the similarity between the operation answers of the set number of operations and the reference answer, key inference steps are selected from the first inference path, and based on the differences between the key inference steps and the inference steps in the second inference path, fourth loss data is generated. Afterwards, the electronic device can train the model to be trained based on the sum of the first loss data, the second loss data, and the fourth loss data to obtain the target model.

[0092] The process of calling the reference model to perform a set number of inference operations based on the training problem, under the constraint of removing any inference step in the first inference path, refers to the rollback process of the inference process. The electronic device can sequentially delete each inference step in the first inference path and input an instruction to remove a specific inference step into the reference model, causing the reference model to skip a certain inference step during the inference process based on the training problem and obtain the calculation answer. This process can be understood as rolling back the inference process to before that inference step and generating a new inference result based on the inference path without that inference step. For example, the inference steps in the first inference path can be represented as follows: , The total number of inference steps; a rollback operation refers to rolling back to a point where the inference process only includes... That is, it does not contain reasoning steps. , .

[0093] To ensure the reliability of the new computational results, a reference model can be invoked to perform a set number of inference operations based on the first inference path with a constraint removed. This process can be called resampling. For example, the set number of operations is 100, but the specific number can be set according to actual needs; this application does not limit this. It is understood that each sampling (inference result) starts from the rolled-back inference process to generate a new inference result. After rollback and resampling, the computational answers for the set number of operations can be obtained, such as 100 computational answers. Then, the electronic device can calculate the similarity between the computational answers for the set number of operations and the reference answer, and select inference steps from the first inference path in descending order of similarity to obtain the selected key inference steps.

[0094] Specifically, in the process of selecting key reasoning steps from the first reasoning path based on the similarity between the answer and the reference answer after a set number of calculations, the electronic device can determine the calculation probability distribution corresponding to the answer after a set number of calculations based on the proportion of identical answers to a set number of calculations. Then, based on the similarity between the calculation probability distribution and the reference probability distribution corresponding to the reference answer, reasoning steps that are removed from the first reasoning information when the similarity exceeds a set similarity threshold are identified as key reasoning steps.

[0095] For each resampling operation, the electronic device can obtain the computational answer. The system then calculates the probability distribution of the calculated answer. Taking a set number of iterations (100) as an example, after removing a certain reasoning step, the reference model can perform 100 reasoning iterations and obtain 100 possible answers. For instance, the reference model might output three different answers in these 100 iterations. For example, A1, A2, and A3, A1 outputs 40 times, A2 outputs 50 times, and A3 outputs 10 times. Therefore, a probability distribution can be obtained based on the proportion of each output to a set number of times. For example, the probability distribution corresponding to the answer to this operation is [0.4, 0.5, 0.1], which is obtained by dividing 40, 50, and 10 by 100 respectively. Similarly, by performing the same statistics on the reasoning process that rolls back to remove each reasoning step, the probability distribution corresponding to the operation after removing each reasoning step can be obtained. Furthermore, the electronic device can calculate the similarity between the probability distribution corresponding to each reasoning step and the reference probability distribution corresponding to the reference answer, in order to quantify the impact of each reasoning step on the final result.

[0096] For example, the reference probability distribution can be represented as This inference path contains all inference steps, i.e., the probability distribution of the reference model's output when executing the complete inference path. The operational probability distribution can be represented as... This reasoning path does not contain any reasoning steps. That is, the probability distribution output by the reference model when a certain inference step is missing.

[0097] Furthermore, electronic devices can use the KL divergence between two probability distributions to measure the similarity between the computational probability distribution and the reference probability distribution corresponding to the reference answer. KL divergence is an indicator used to measure the difference between two probability distributions. It can be used to quantify the change in the distribution of the reasoning result after removing a certain reasoning step during the reasoning process, thereby helping to identify key reasoning steps.

[0098] For example, the KL divergence between the operational probability distribution and the reference probability distribution can be calculated as shown in Formula 12: Formula 12 In Formula 12, Denotes KL divergence, This represents all possible answers, such as the answers to the three operations mentioned above. For example, A1, A2 and A3. It includes reasoning steps. The reference probability distribution It is about deleting the reasoning steps. The probability distribution of the operation. It can be understood that the larger the value of the KL divergence, the more steps are involved in the inference process. The greater the influence on the reference answer, the more effective this reasoning step becomes. This could be a key thought process, or conversely, its impact would be smaller; this reasoning step... It may be a redundant reasoning step.

[0099] In some embodiments, the electronic device can roll back to removing a certain inference step. The obtained KL divergence is used as the reasoning step. The impact on the reference answer can be expressed as shown in Formula 13: Formula 13 In Formula 13, It can represent the steps of reasoning. The impact on the reference answer means that the value of the KL divergence can be directly used as the value of this impact.

[0100] Understandably, this method of analyzing outputs based on different outputs without understanding the model's internal structure can be called a "black box analysis" method. This refers to treating the model as a black box during its execution and analyzing it based on its output. In this embodiment, this method can also be called a "reasoning attribution method," which refers to accurately quantifying the impact of each reasoning step on the final answer based on multiple rollback experiments, identifying key reasoning points such as the initial reasoning step, backtracking adjustments, or critical reasoning steps.

[0101] Furthermore, the electronic device can sort the inference steps based on their calculated impact on the reference answer in descending order, and identify the top N inference steps with an impact value greater than a set impact threshold as key inference steps. Understandably, this means the similarity between the computational probability distribution corresponding to the key inference step and the reference probability distribution corresponding to the reference answer is greater than a set similarity threshold.

[0102] After identifying the key inference steps, the electronic device can align the key inference steps with the inference steps in the second inference path, i.e., generate fourth loss data based on the difference between the key inference steps and the inference steps in the second inference path, such as by generating the fourth loss data through the KL / cross-entropy loss function, so as to use the sum of the fourth loss data and other loss data to train the model to be trained.

[0103] In some embodiments, the sum of the first loss data and the fourth loss data can be used to train the model to be trained; that is, the loss data can be represented as... , This represents the fourth loss data, which can also be called the COT loss. That is, the final output distribution is directly aligned, and alignment is performed on the inference path using the COT loss.

[0104] In some embodiments, the electronic device can also generate fourth loss data based on the difference between key inference steps selected by attention influence and inference steps in the second inference path. That is, based on the attention influence calculated as in Formula 1 or Formula 2, N inference steps with influence values ​​greater than a set influence threshold can be selected in descending order, and the difference between them and inference steps in the second inference path can be used to generate fourth loss data.

[0105] In some embodiments, the electronic device can also generate fourth loss data based on the difference between the broadcast inference steps and the inference steps in the second inference path. That is, based on the broadcast impact calculated as in Formula 5, N inference steps with an impact value greater than a set impact threshold can be selected in descending order to obtain a set of broadcast inference steps, which can be understood as a set of broadcast sentences, and the fourth loss data is generated based on the difference between the broadcast inference steps and the inference steps in the second inference path.

[0106] In some embodiments, the electronic device can train the model to be trained based on the sum of the above four parts, and the total loss data can be as shown in Formula 14: Formula 14 In Formula 14, This represents the total loss data. , , and These refer to the first loss data, the second loss data, the third loss data, and the fourth loss data, respectively. The weighting coefficient can be a floating-point number between 0 and 1. As shown in Formula 14, in this embodiment, the attention matrix, attention influence, final output, and inference step sequence can be aligned simultaneously to form a multi-signal distillation loss to achieve inference path distillation.

[0107] In some embodiments, electronic devices can adjust the model parameters of the training model using gradient descent. When updating model parameters using gradient descent, the gradient of the loss data is calculated, and the model parameters are iteratively updated based on this gradient to obtain the adjusted target model. It is understood that the target model is the training model with an optimized second inference path. When performing inference operations, the target model focuses more on key inference steps, reducing or eliminating redundant inference steps, thereby simplifying unnecessary inference processes, improving inference efficiency while maintaining high accuracy, and reducing unnecessary computational costs.

[0108] Please refer to the following: Figure 4 , Figure 4 This is a comparative diagram of the inference process performed by a reference model and a target model, as provided in an embodiment of this application. Figure 4 Taking intelligent question-answering as an example, users can enter a question in the user interface and trigger the generation of a corresponding answer. Figure 4 As shown, when answering the same question, "A positive integer that is even and greater than 10 is called a 'qualified number.' Determine whether the number 14 is a qualified number," both the reference model and the target model can provide the correct answer, such as "Yes, 14 is a qualified number." However, the first inference path executed by the reference model contains 7 inference steps, while the second inference path executed by the target model contains only 4 inference steps. The target model's second inference path is a simplification of the first inference path. Each inference step in the second inference path can be a key inference step identified in the first inference path, or a simplification of its own steps.

[0109] Therefore, by optimizing the inference path and identifying key steps, the distillation process becomes more efficient and can also improve distillation quality. The target model executes simplified inference steps, which improves its inference efficiency and reduces unnecessary computational overhead. In-depth analysis of the inference paths of both models enhances the interpretability of knowledge distillation, helping developers better understand and optimize the model, thus achieving efficient optimization of the inference process. Furthermore, for question-answering scenarios, the target model can improve inference speed while outputting the correct answer, thereby increasing the response speed of the question-answering platform and, to some extent, improving the user experience.

[0110] In some embodiments of this application, the technical solutions provided by the application analyze the attention weights between inference steps calculated by the attention head in the model to conduct an in-depth analysis of the model's inference process. This "white box (causal analysis)" method can accurately locate the causal relationship between inference steps, such as which inference steps depend on which inference steps, and can also identify which inference steps do not affect each other, thereby helping small models to construct the inference process more efficiently.

[0111] Furthermore, by removing a certain reasoning step to roll back the reasoning process, this "black box" (reasoning attribution analysis) method can identify key reasoning steps that significantly impact the result. These steps can enhance the interpretability of the model's final answer and provide valuable support for optimizing the reasoning path of the model to be trained. By optimizing and refining key reasoning steps, the reasoning capabilities of the reference model can be transferred more efficiently during knowledge distillation, reducing redundant computation and thus achieving efficient reasoning distillation.

[0112] Furthermore, by aligning not only the outputs of the two models, but also the causal relationships between inference steps, the distribution of attention to inference steps, and the degree of influence on the final answer, the trained target model can not only retain the core reasoning ability, but also optimize redundant inference steps with a smaller impact. This solves the problem of optimizing the structure of the inference process, improves the simplicity and efficiency of the inference steps, reduces the execution of redundant inference steps, helps reduce computational overhead, and to a certain extent improves the efficiency and quality of knowledge distillation.

[0113] Furthermore, applying the knowledge distillation-derived model to interactive platforms such as online customer service, intelligent assistants, and educational Q&A can improve the platform's reasoning efficiency while ensuring the accuracy of responses, thereby increasing the platform's response speed. Because the target model is relatively small and learns to simplify the reasoning process during knowledge distillation, it can also be deployed on resource-constrained devices, increasing the flexibility of model deployment and enabling high-quality, low-latency intelligent interactive services in low-resource environments.

[0114] To facilitate rapid application of this application by those skilled in the art, the following is combined with... Figure 5 The knowledge distillation architecture diagram shown briefly illustrates a training process provided by an embodiment of this application.

[0115] like Figure 5As shown, the knowledge distillation architecture includes a teacher model and a student model. This application uses an LLM model for both the teacher and student models as an example for explanation, where the teacher model is much larger than the student model. This application's embodiments primarily use two inference analysis methods (black box and white box methods) to identify the key decision points of the teacher model in the inference process, and accordingly optimize the inference process of the student model, ultimately achieving inference distillation of LLM and simplifying the inference process.

[0116] 1. Teacher Model: 1) Input the training question into the teacher model and obtain the reference answer output by the teacher model.

[0117] 2) White-box method: Extract the attention matrix obtained from the attention head operation of all network layers in the teacher model, and aggregate them into a sentence-level (inference step) attention matrix. Calculate the attentional impact of each inference step on subsequent inference steps, as shown in Formulas 1 and 2 above, to obtain the first attention intensity.

[0118] Alternatively, the receiver head in the teacher model can be selected based on the kurtosis value of the attention head, and the attention influence of each inference step on subsequent inference steps can be calculated based on the attention matrix obtained from the receiver head, as shown in Formula 5 above, to obtain the first attention intensity.

[0119] Calculate the mean attention weight corresponding to each element position in all attention matrices to obtain the first attention mean matrix.

[0120] 3) Black-box method: The reasoning path after removing any reasoning steps is input into the teacher model to reason again on the same training problem to obtain the calculated answer. The similarity (KL divergence) between the calculated answer and the reference answer is calculated, and the KL divergence calculated after removing each reasoning step is used as the influence strength of each reasoning step. The reasoning steps are sorted from high to low influence strength, and the top-ranked reasoning steps are selected as key reasoning steps.

[0121] The teacher's model has the following fixed signals: key reasoning steps, attention weight matrix at the reasoning step level, attention influence (first attention intensity), and reference answer.

[0122] 2. Student Model: Generates answers to the same training questions and outputs the student model's inference path and attention matrix. Specifically: 1) Input the training question into the student model and obtain the reasoning answer output by the student model.

[0123] 2) White-box method: Extract the attention matrices obtained from the attention head operations of all network layers and aggregate them into sentence-level (inference step) attention matrices. Calculate the attentional impact of each inference step on subsequent inference steps, as shown in Formulas 1 and 2 above, to obtain the second attention intensity.

[0124] Alternatively, the receiver head in the teacher model can be selected based on the kurtosis value of the attention head, and the attention influence of each inference step on subsequent inference steps can be calculated based on the attention matrix obtained from the receiver head, as shown in Formula 5 above, to obtain the second attention intensity.

[0125] Calculate the mean attention weight corresponding to each element position in all attention matrices to obtain the second attention mean matrix.

[0126] The student model has the following fixed signals: inference path, attention weight matrix at the inference step level, attention influence (secondary attention influence), and inference answer.

[0127] 3. Loss signal design: 1) Output distribution loss The difference between the reference answer and the inferred answer.

[0128] 2) Attention affects loss The difference between the first and second attention intensity.

[0129] 3) Loss of attention weights The difference between the first attention mean matrix at the inference step level of the teacher model and the second attention mean matrix at the inference step level of the student model.

[0130] 4) Loss of reasoning steps The difference between the key reasoning steps and the reasoning steps of the student model.

[0131] Therefore, the loss data of the above four parts are summed (as shown in Formula 14), and the model parameters of the student model are adjusted by multi-signal joint, and the adjusted student model is determined as the target model.

[0132] In summary, on the one hand, the knowledge distillation method of this application can deeply analyze the reasoning process of the teacher model, and can identify redundant or unnecessary reasoning steps by calculating the influence or degree of attention. These reasoning steps do not substantially contribute to the efficiency of reasoning and have little impact on the final answer; they can be completed in a more concise way. Similarly, based on attention intensity and influence, key reasoning steps can also be accurately identified, enabling the student model to reduce the execution of redundant reasoning steps and focus only on executing key reasoning steps. This approach simplifies the reasoning process of the student model, avoids a large amount of irrelevant reasoning computation and memory consumption, and improves reasoning efficiency while reducing the consumption of computing resources.

[0133] On the other hand, training with loss data obtained through four different methods not only mimics the final output of the reference model but also includes attention distribution, attention influence, and influence strength obtained from precise attribution and causal analysis. This allows the student model to better imitate the reasoning path and decision-making logic of the teacher model, rather than simply imitating the final answer. Therefore, the student model can achieve reasoning performance close to that of the teacher model with lower computational overhead, thus achieving high-quality distillation.

[0134] On the other hand, by calculating parameters such as the influence / influence intensity of attention, we can improve the interpretability of model inference while reducing redundant steps, achieve model distillation more efficiently, and provide a more effective tool for future model optimization and debugging.

[0135] On the other hand, the trained student model, due to its more streamlined reasoning steps and reduced computational complexity, can be deployed on resource-constrained terminal or edge devices, improving the flexibility of model deployment and making it applicable to a wider range of human-computer interaction scenarios. The target model obtained through this knowledge distillation can reduce inference latency while maintaining reasoning correctness, enabling it to quickly and accurately answer user input questions, thereby improving the user's interactive experience to some extent.

[0136] The methods of the embodiments of this application have been described in detail above. In order to facilitate better implementation of the above solutions of the embodiments of this application, the apparatus of the embodiments of this application is provided below.

[0137] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Figure 6 The data processing device shown can be mounted in an electronic device, which may specifically be a terminal device or a server. Figure 6 The data processing apparatus shown can be used to perform the above. Figure 2 Some or all of the functionality described in the method embodiments. Please refer to [link / reference]. Figure 6 The data processing device 60 includes: The acquisition unit 601 is used to acquire the first reasoning path of the reference model generating a reference answer based on the training question, and to acquire the second reasoning path of the model to be trained generating a reasoning answer based on the training question. The determining unit 602 is used to determine the first attention intensity corresponding to the first inference path based on the first attention information calculated by the reference model during the execution of the first inference path. The first attention intensity is used to represent the degree of influence of each inference step in the first inference path on the reference answer. The determining unit 602 is further configured to determine the second attention intensity corresponding to the second inference path based on the second attention information calculated by the model to be trained during the execution of the second inference path. The second attention intensity is used to represent the degree of influence of each inference step in the second inference path on the inference answer. Training unit 603 is used to train the model to be trained based on the difference between the reference answer and the reasoned answer, and the difference between the first attention intensity and the second attention intensity, to obtain the target model.

[0138] In one possible implementation, the acquisition unit 601 is specifically used for: The training question is input into the reference model so that the reference model can perform inference operations based on the training question to obtain a first inference path including at least two inference steps and the reference answer. The training question is input into the model to be trained, so that the model to be trained can perform inference operations based on the training question to obtain a second inference path including at least two inference steps and the inference answer.

[0139] In one possible implementation, the first attention information includes first attention matrices computed by at least two attention modules in the reference model, each first attention matrix including attention weights between any two inference steps in the first inference path; the determining unit 602 is specifically used for: Extract partial attention weights between each inference step and subsequent inference steps from the at least two first attention matrices, and determine the sum of the extracted partial attention weights as the first attention intensity.

[0140] In one possible implementation, the first attention information includes first attention matrices computed by at least two attention modules in the reference model, each first attention matrix including attention weights between any two inference steps in the first inference path; the determining unit 602 is specifically used for: Select a target attention module from the at least two attention modules; wherein, the first attention matrix calculated by the target attention module contains attention weights greater than a set weight threshold; Extract partial attention weights between each inference step and subsequent inference steps from the first attention matrix obtained by the target attention module, and determine the sum of the extracted partial attention weights as the first attention intensity.

[0141] In one possible implementation, the determining unit 602 is specifically used for: Based on the established mapping relationship between the attention weights in the attention matrix and the module kurtosis value of the attention module, and the attention weights in each of the first attention matrices, the module kurtosis value corresponding to each attention module is determined. The target attention module is obtained by extracting the attention modules whose kurtosis value is greater than a specified kurtosis threshold from the at least two attention modules.

[0142] In one possible implementation, the device 60 further includes: The calculation unit 604 is used to calculate the mean kurtosis value and the standard deviation of the kurtosis value based on the module kurtosis values ​​corresponding to the at least two attention modules respectively. The determining unit 602 is used to determine the kurtosis threshold associated with the average kurtosis value and the standard deviation of kurtosis value based on the correlation between the kurtosis mean, the kurtosis standard deviation and the kurtosis threshold, as well as the average kurtosis value and the standard deviation of kurtosis value, so as to obtain the specified kurtosis threshold.

[0143] In one possible implementation, the second attention information includes second attention matrices calculated by at least two attention modules in the model to be trained, each second attention matrix including attention weights between any two inference steps in the second inference path; the determining unit 602 is specifically used for: Extract partial attention weights between each inference step and subsequent inference steps from the at least two second attention matrices, and determine the sum of the extracted partial attention weights as the second attention intensity.

[0144] In one possible implementation, the training unit 603 is specifically used for: Based on the difference between the reference answer and the inferred answer, first loss data is generated, and based on the difference between the first attention intensity and the second attention intensity, second loss data is generated. The target model is obtained by training the model to be trained based on the sum of the first loss data and the second loss data.

[0145] In one possible implementation, the first attention information includes a first attention matrix calculated by at least two attention modules in the reference model, and the second attention information includes a second attention matrix calculated by at least two attention modules in the model to be trained; the training unit 603 is specifically used for: Based on the differences between the at least two first attention matrices and the at least two second attention matrices, third loss data is generated, and the model to be trained is trained based on the sum of the first loss data, the second loss data and the third loss data to obtain the target model.

[0146] In one possible implementation, the training unit 603 is specifically used for: Based on the attention weights of each element position in the at least two first attention matrices, the average weights corresponding to each element position are calculated to obtain the first attention mean matrix; Based on the attention weights of each element position in the at least two second attention matrices, the average weights corresponding to each element position are calculated to obtain the second attention mean matrix; The third loss data is generated based on the difference between the weight mean values ​​corresponding to the positions of each element in the first attention mean matrix and the second attention mean matrix.

[0147] In one possible implementation, the training unit 603 is specifically used for: Without removing the constraints of any reasoning step in the first reasoning path, the reference model is invoked to perform a set number of reasoning operations based on the training question to obtain the answer of the set number of operations; Based on the similarity between the calculation answer of the set number of times and the reference answer, key reasoning steps are selected from the first reasoning path, and based on the difference between the key reasoning steps and the reasoning steps in the second reasoning path, fourth loss data is generated; The target model is obtained by training the model to be trained based on the sum of the first loss data, the second loss data, and the fourth loss data.

[0148] In one possible implementation, the training unit 603 is specifically used for: In the set number of calculation answers, the calculation probability distribution corresponding to the set number of calculation answers is determined based on the proportion of the number of identical answers to the set number of calculations. Based on the similarity between the operational probability distribution and the reference probability distribution corresponding to the reference answer, the reasoning steps that are removed from the first reasoning information when the similarity is greater than a set similarity threshold are identified as the key reasoning steps.

[0149] According to one embodiment of this application, Figure 2 The data processing method shown may involve some steps that can be derived from... Figure 6 The data processing apparatus shown is executed by each unit within it. For example, Figure 2 Step S201 shown can be performed by Figure 6 The acquisition unit 601 shown is executed, and steps S202 and S203 can be performed by... Figure 6 The determination unit 602 shown is executed, and step S204 can be performed by... Figure 6 The training unit 603 shown is executed. Figure 6 The data processing apparatus shown can be composed of individual or combined units into one or more other units, or some of the units can be further divided into two or more functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above-mentioned units are based on logical function division. In practical applications, the function of one unit can also be implemented by two or more units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the data processing apparatus may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by two or more units.

[0150] According to another embodiment of this application, a general-purpose computing device, such as a computer device including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can perform operations such as... Figure 2 The computer program (including program code) involved in each step of the corresponding method shown, to construct such... Figure 6 The data processing apparatus shown herein, and the data processing method for implementing the embodiments of this application, are described. A computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the computer-readable recording medium, and executed therein.

[0151] Based on the same inventive concept, the principle and beneficial effects of the data processing device provided in the embodiments of this application in solving the problem are similar to the principle and beneficial effects of the data processing method in the embodiments of this application in solving the problem. For the sake of brevity, the principle and beneficial effects of the method implementation can be referred to.

[0152] Embodiments of this application also provide an electronic device, including: at least one processor; and a storage device for storing at least one computer program, which, when executed by at least one processor, causes the electronic device to implement the data processing methods provided in the above embodiments.

[0153] Figure 7 A schematic diagram of a computer system suitable for implementing an electronic device according to embodiments of this application is shown. The electronic device may be... Figure 1 The device shown.

[0154] It should be noted that, Figure 7 The computer system 700 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0155] like Figure 7 As shown, the computer system 700 includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes based on a computer program stored in Read-Only Memory (ROM) 702 or a computer program loaded from storage portion 708 into Random Access Memory (RAM) 703, such as executing the image generation method described in the above embodiments. Various computer programs and data required for system operation are also stored in RAM 703. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0156] In some embodiments, the following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0157] In particular, according to embodiments of this application, a computer program implementing the image generation method can be carried on a computer-readable medium, which can be downloaded and installed from a network via the communication section 709, and / or installed from a removable medium 711.

[0158] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or at least two wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a computer program that can be used by or in conjunction with an instruction execution system, apparatus, or device. Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer program contained in the computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0159] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or at least two executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and a computer program.

[0160] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0161] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor of an electronic device, causes the electronic device to implement the image generation method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0162] Another aspect of this application provides a computer program product, which includes a computer program that, when executed by a processor, implements the image generation methods provided in the various embodiments described above. The computer program can be stored in a computer-readable storage medium. The computer program product can be a computer program as a product, such as a mobile application (APP), webpage, mini-program, etc.; or, the computer program product can also be a storage medium, device, terminal, virtual machine, etc., containing the computer program.

[0163] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

[0164] It should also be noted that when the above embodiments of this application are applied to specific products or technologies, if it is necessary to obtain user data, the user's permission or consent must be obtained, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

Claims

1. A data processing method, characterized in that, include: Obtain the first inference path of the reference model generating a reference answer based on the training question, and obtain the second inference path of the model to be trained generating an inference answer based on the training question; Based on the first attention information calculated by the reference model during the execution of the first inference path, the first attention intensity corresponding to the first inference path is determined. The first attention intensity is used to represent the degree of influence of each inference step in the first inference path on the reference answer. Based on the second attention information calculated by the model to be trained during the execution of the second inference path, the second attention intensity corresponding to the second inference path is determined. The second attention intensity is used to represent the degree of influence of each inference step in the second inference path on the inference answer. Based on the difference between the reference answer and the inferred answer, and the difference between the first attention intensity and the second attention intensity, the model to be trained is trained to obtain the target model.

2. The method according to claim 1, characterized in that, The process of obtaining a first inference path for the reference model to generate a reference answer based on the training question, and obtaining a second inference path for the model to be trained to generate an inference answer based on the training question, includes: The training question is input into the reference model so that the reference model can perform inference operations based on the training question to obtain a first inference path including at least two inference steps and the reference answer. The training question is input into the model to be trained, so that the model to be trained can perform inference operations based on the training question to obtain a second inference path including at least two inference steps and the inference answer.

3. The method according to claim 1, characterized in that, The first attention information includes first attention matrices calculated by at least two attention modules in the reference model, and each first attention matrix includes attention weights between any two inference steps in the first inference path; The determination of the first attention intensity corresponding to the first inference path based on the first attention information calculated by the reference model during the execution of the first inference path includes: Extract partial attention weights between each inference step and subsequent inference steps from the at least two first attention matrices, and determine the sum of the extracted partial attention weights as the first attention intensity.

4. The method according to claim 1, characterized in that, The first attention information includes first attention matrices calculated by at least two attention modules in the reference model, and each first attention matrix includes attention weights between any two inference steps in the first inference path; The determination of the first attention intensity corresponding to the first inference path based on the first attention information calculated by the reference model during the execution of the first inference path includes: Select a target attention module from the at least two attention modules; wherein, the first attention matrix calculated by the target attention module contains attention weights greater than a set weight threshold; Extract partial attention weights between each inference step and subsequent inference steps from the first attention matrix obtained by the target attention module, and determine the sum of the extracted partial attention weights as the first attention intensity.

5. The method according to claim 4, characterized in that, Selecting a target attention module from the at least two attention modules includes: Based on the established mapping relationship between the attention weights in the attention matrix and the module kurtosis value of the attention module, and the attention weights in each of the first attention matrices, the module kurtosis value corresponding to each attention module is determined. The target attention module is obtained by extracting the attention modules whose kurtosis value is greater than a specified kurtosis threshold from the at least two attention modules.

6. The method according to claim 5, characterized in that, The method further includes: Based on the module kurtosis values ​​corresponding to the at least two attention modules respectively, calculate the mean kurtosis value and the standard deviation of the kurtosis value; Based on the correlation between the mean kurtosis, the standard deviation of kurtosis, and the kurtosis threshold, as well as the average kurtosis and the standard deviation of kurtosis, a kurtosis threshold associated with the average kurtosis and the standard deviation of kurtosis is determined, thus obtaining the specified kurtosis threshold.

7. The method according to claim 1, characterized in that, The second attention information includes the second attention matrix calculated by at least two attention modules in the model to be trained, and each second attention matrix includes the attention weight between any two inference steps in the second inference path; The step of determining the second attention intensity corresponding to the second inference path based on the second attention information computed by the model under training during the execution of the second inference path includes: Extract partial attention weights between each inference step and subsequent inference steps from the at least two second attention matrices, and determine the sum of the extracted partial attention weights as the second attention intensity.

8. The method according to claim 1, characterized in that, The process of training the model to be trained based on the difference between the reference answer and the inferred answer, and the difference between the first attention intensity and the second attention intensity, to obtain the target model, includes: Based on the difference between the reference answer and the inferred answer, first loss data is generated, and based on the difference between the first attention intensity and the second attention intensity, second loss data is generated. The target model is obtained by training the model to be trained based on the sum of the first loss data and the second loss data.

9. The method according to claim 8, characterized in that, The first attention information includes the first attention matrix calculated by at least two attention modules in the reference model, and the second attention information includes the second attention matrix calculated by at least two attention modules in the model to be trained. The step of training the model to be trained based on the sum of the first loss data and the second loss data to obtain the target model includes: Based on the differences between the at least two first attention matrices and the at least two second attention matrices, third loss data is generated, and the model to be trained is trained based on the sum of the first loss data, the second loss data and the third loss data to obtain the target model.

10. The method according to claim 9, characterized in that, The generation of third loss data based on the differences between the at least two first attention matrices and the at least two second attention matrices includes: Based on the attention weights of each element position in the at least two first attention matrices, the average weights corresponding to each element position are calculated to obtain the first attention mean matrix; Based on the attention weights of each element position in the at least two second attention matrices, the average weights corresponding to each element position are calculated to obtain the second attention mean matrix; The third loss data is generated based on the difference between the weight mean values ​​corresponding to the positions of each element in the first attention mean matrix and the second attention mean matrix.

11. The method according to claim 8, characterized in that, The step of training the model to be trained based on the sum of the first loss data and the second loss data to obtain the target model includes: Without removing the constraints of any reasoning step in the first reasoning path, the reference model is invoked to perform a set number of reasoning operations based on the training question to obtain the answer of the set number of operations; Based on the similarity between the calculation answer of the set number of times and the reference answer, key reasoning steps are selected from the first reasoning path, and based on the difference between the key reasoning steps and the reasoning steps in the second reasoning path, fourth loss data is generated; The target model is obtained by training the model to be trained based on the sum of the first loss data, the second loss data, and the fourth loss data.

12. The method according to claim 11, characterized in that, The selection of key reasoning steps from the first reasoning path based on the similarity between the calculated answer and the reference answer, based on the set number of calculations, includes: In the set number of calculation answers, the calculation probability distribution corresponding to the set number of calculation answers is determined based on the proportion of the number of identical answers to the set number of calculations. Based on the similarity between the operational probability distribution and the reference probability distribution corresponding to the reference answer, the reasoning steps that are removed from the first reasoning information when the similarity is greater than a set similarity threshold are identified as the key reasoning steps.

13. A data processing apparatus, characterized in that, include: The acquisition unit is used to acquire a first inference path of the reference model generating a reference answer based on the training question, and to acquire a second inference path of the model to be trained generating an inference answer based on the training question; The determining unit is used to determine the first attention intensity corresponding to the first inference path based on the first attention information calculated by the reference model during the execution of the first inference path. The first attention intensity is used to represent the degree of influence of each inference step in the first inference path on the reference answer. The determining unit is further configured to determine the second attention intensity corresponding to the second inference path based on the second attention information calculated by the model to be trained during the execution of the second inference path. The second attention intensity is used to represent the degree of influence of each inference step in the second inference path on the inference answer. The training unit is used to train the model to be trained based on the difference between the reference answer and the inferred answer, and the difference between the first attention intensity and the second attention intensity, to obtain the target model.

14. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more computer programs that, when executed by one or more processors, cause the electronic device to perform the data processing method according to any one of claims 1-12.

15. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the data processing method according to any one of claims 1-12.

16. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium, wherein a processor of an electronic device reads from and executes the computer program, causing the electronic device to perform the data processing method according to any one of claims 1-12.

Images