Model training method and device based on knowledge distillation, electronic equipment and medium

Abstract

The invention discloses a knowledge distillation-based model training method and device, electronic equipment and a medium. The method comprises the steps that question cue words are input into a student model, and the student model gradually generates student answer marks according to the question cue words; for each student answer mark, calculating the difference between the student answer mark and a teacher answer mark generated by the teacher model; if the difference is equal to or greater than a preset difference threshold, determining the current state of the student model as a target divergence error state; according to an error path generated by the student model based on the target bifurcation error state and a rescue path generated by the teacher model based on the target bifurcation error state, calculating comparison sorting loss; and training the student model according to the comparison sorting loss to obtain a trained student model. According to the method, the key error node can be accurately identified, the student model is effectively taught to recover from the error state, and the reasoning effect of the student model is improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to a model training method, device, electronic device and medium based on knowledge distillation. Background Technology

[0002] Knowledge distillation is a technique for transferring knowledge from a complex model (i.e., the teacher model) to a simpler model (i.e., the student model). The training data for the student model is usually generated by the teacher model (perfect data), while during inference, the student model needs to make predictions based on its own generated (potentially erroneous) historical data. This inconsistency in the distribution of "training-inference" data means that once the student model makes a mistake, it cannot be recovered.

[0003] In related technologies, a probability density-based policy internal distillation method is used to train the student model for error correction. The specific process is as follows: the student model attempts to generate inference steps, and the teacher model calculates the probability value of that step under the teacher's distribution. If the steps generated by the student model deviate from the teacher's distribution, the calculated discrepancy (such as KL divergence) becomes larger, serving as a loss function to penalize the student model and force it to mimic the teacher's probability distribution as closely as possible at each step. However, this approach fails to effectively teach the student model to recover from erroneous states and lacks sufficient ability to identify key error nodes, resulting in poor inference performance for the student model. Summary of the Invention

[0004] The purpose of this application is to provide a model training method, apparatus, electronic device, and storage medium based on knowledge distillation, in order to solve the problems in related technologies that are unable to effectively teach student models to recover from erroneous states and lack sufficient identification of key errors, resulting in poor reasoning performance of student models.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions: In a first aspect, embodiments of this application provide a model training method based on knowledge distillation, comprising: inputting question prompts into a student model, wherein the student model progressively generates student answer tags based on the question prompts, wherein the question prompts include a question and the context of the question; for each student answer tag, calculating the difference between the student answer tag and a teacher answer tag generated by a teacher model; if the difference is equal to or greater than a preset difference threshold, determining the current state of the student model as a target divergence error state; calculating a contrastive ranking loss based on the error path generated by the student model based on the target divergence error state and the rescue path generated by the teacher model based on the target divergence error state; and training the student model based on the contrastive ranking loss to obtain the trained student model.

[0006] Secondly, embodiments of this application provide a model training apparatus based on knowledge distillation, comprising: a generation module, configured to input question prompts into a student model, wherein the student model progressively generates student answer tags based on the question prompts, wherein the question prompts include a question and the context of the question; a first calculation module, configured to calculate, for each student answer tag, the difference between the student answer tag and a teacher answer tag generated by a teacher model; a determination module, configured to determine the current state of the student model as a target divergence error state if the difference is equal to or greater than a preset difference threshold; a second calculation module, configured to calculate a contrastive ranking loss based on the error path generated by the student model based on the target divergence error state and the rescue path generated by the teacher model based on the target divergence error state; and a training module, configured to train the student model based on the contrastive ranking loss to obtain the trained student model.

[0007] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect of this application.

[0008] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect of this application.

[0009] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: In this embodiment of the application, when training and correcting the student model, if the difference between the student's answer marker and the teacher's answer marker generated by the teacher model is equal to or greater than a difference threshold, it is considered that the student model has a serious logical deviation. This can accurately identify key error nodes and improve the reasoning performance of the student model. Based on the error state of the key error node, a comparison relationship between the error path and the recovery path is constructed, allowing the student model to fully differentiate between incorrect and correct decisions in the feature space. This encourages the student model to learn the error correction logic more aggressively, and also informs the student model of the relatively optimal remedial measures in an already erroneous state. Therefore, it effectively teaches the student model to recover from the error state, further improving the reasoning performance of the student model. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a model training method based on knowledge distillation, provided for one embodiment of this application; Figure 2 A schematic diagram of the overall process of a model training method based on knowledge distillation, provided for another embodiment of this application; Figure 3 A schematic diagram of a model training device based on knowledge distillation, provided for one embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, "and / or" in this application indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. It should be noted that all data involved in this application was obtained with the user's authorization.

[0013] In this application, the term "Teacher Model" plays the role of a "guide," equivalent to a knowledgeable teacher whose answers are generally considered standard answers.

[0014] In this application, the term "Student Model" plays the role of "learner," which has a relatively simple structure and requires training to imitate the thinking style of the teacher model.

[0015] In related technologies, using probability density-based intra-policy distillation to train student models for error correction has the following deep-seated technical shortcomings, which limit its effectiveness in complex logical reasoning tasks: 1) Lack of constructive corrective feedback The use of KL divergence or probability scoring in related techniques is essentially a form of "soft penalty." When the student model enters an incorrect state (e.g., an initial calculation error), the probability distribution of the teacher model typically shows an extremely low probability of the current state. However, simple probability fitting does not explicitly tell the student model "what is the relatively optimal remedy in this already incorrect state." The teacher model usually only attempts to forcefully pull the student model back to the original perfect path, but in logical reasoning, if the premise is wrong, forcibly applying the original path often leads to logical breaks.

[0016] 2) Lack of counterfactual path deduction In existing technologies, superiority or inferiority is distinguished solely by probability. However, student models often struggle to differentiate between "suboptimal decisions" and "serious logical biases." The lack of explicit comparison between "incorrect paths" and "correct remedial paths" in current technologies results in student models failing to adequately differentiate between incorrect and correct decisions in the feature space, leading to frequent oscillations around fuzzy boundaries.

[0017] 3) Uniform allocation of computing resources Related technologies involve intensive scoring and distillation of each token along the entire path. However, inference errors typically originate from a few key divergences. Uniformly allocating computational resources across all steps wastes computational resources and dilutes the effectiveness of correcting critical error nodes.

[0018] Therefore, the aforementioned related technologies cannot effectively teach student models to recover from erroneous states and lack sufficient ability to identify key error nodes, resulting in poor reasoning performance of student models. To address this, this application proposes a model training method, apparatus, electronic device, and storage medium based on knowledge distillation to improve the reasoning performance of student models.

[0019] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0020] Figure 1 This is a flowchart illustrating a model training method based on knowledge distillation, provided as an embodiment of this application. Figure 1 As shown, the model training method based on knowledge distillation in this application embodiment may specifically include the following steps: S101, Input the question prompts into the student model. The student model generates student answer tags step by step based on the question prompts. The question prompts include the question and the context of the question.

[0021] In this embodiment of the application, the execution entity of the knowledge distillation-based model training method is a knowledge distillation-based model training device, which can be located in an electronic device. This electronic device can be a terminal device or a server. The terminal device can be a mobile phone, tablet computer, desktop computer, laptop, in-vehicle device, etc.; the server can be a standalone server or a server cluster composed of multiple servers.

[0022] Based on the question and its context Context The system uses prompt word templates to generate question prompt words. These prompt words are then input into the student model for reasoning. The student model progressively generates student answer tokens based on the input prompt words, essentially creating a step-by-step trajectory for each student's answer token, resulting in a student answer trajectory T. student .

[0023] S102, for each student's answer tag, calculate the difference between the student's answer tag and the teacher's answer tag generated by the teacher model.

[0024] In this embodiment, during each student response token generation process, a key deviation detection module is used to calculate in real time the difference between the student response token output by the current student model and the teacher response token output by the teacher model. For example, the KL (Kullback-Leibler) divergence and / or confidence difference between the student response token output by the current student model and the teacher response token output by the teacher model can be calculated in real time. KL divergence, also known as relative entropy, is an indicator that measures the degree of matching between two probability distributions. The larger the KL divergence, the greater the distribution difference and the lower the matching degree.

[0025] S103, if the difference is equal to or greater than the preset difference threshold, then the current state of the student model is determined as the target divergence error state.

[0026] In this embodiment, assuming the correct logical chain is A→B→C, if the student model generates a deviated state A' (i.e., an incorrect intermediate state A', or an incorrect student response token A') at time t, when the detection module identifies that the difference between A' and the teacher's response token A is equal to or greater than a preset difference threshold, time t is marked as a critical divergence point (i.e., a serious logical deviation has occurred), and the current deviated state A' is locked as the target divergence error state S. err .

[0027] If the difference is less than the difference threshold, the student model is controlled to continue generating student response tokens step by step.

[0028] S104, calculate the comparative ranking loss based on the error path generated by the student model based on the target divergence error state and the rescue path generated by the teacher model based on the target divergence error state.

[0029] In this embodiment, when the difference is equal to or greater than a preset difference threshold, it is determined that the student model is currently at a critical divergence point (i.e., the inference trajectory generated by the student model has undergone a serious logical deviation), and a dynamic forking mechanism is immediately triggered to lock the target divergence error state S. err Stop unidirectional generation and construct counterfactual fork paths.

[0030] On the one hand, the negative branch: retains the student model based on the goal divergence error state S. err The subsequent paths that are generated, or allowed to continue generating for several more steps, are denoted as error paths. Path neg This path Path neg It represents the natural cumulative result of erroneous decisions.

[0031] On the other hand, the positive branch: calls the teacher model and forces the teacher model to the current goal divergence error state S. err Takeover is generated based on the initial conditions, resulting in an optimal rescue path. Path pos This path Path pos This represents the "best remedy in this situation," that is, finding the optimal subsequent path to revert to the correct logic given that a deviation has already occurred. Path pos .

[0032] Based on positive and negative sample pairs (negative samples: error paths) Path neg Positive Samples: A Rescue Path Path pos ), calculate the comparison ranking loss L .

[0033] This application's embodiments construct "counterfactual path deduction" samples (i.e., how the outcome will evolve if the optimal strategy is adopted in the current erroneous state, i.e., positive samples) at key nodes of reasoning (i.e., fork nodes), and enhance the student model's ability to correct and recover from erroneous states through comparative learning.

[0034] Furthermore, the step of "calculating the contrastive ranking loss based on the erroneous paths generated by the student model based on the target divergence error state and the rescue paths generated by the teacher model based on the target divergence error state" can be implemented through the following steps: constructing pairs of contrastive teaching samples by using erroneous paths as negative samples and rescue paths as positive samples; inputting the contrastive teaching samples and context into the contrastive ranking loss function to calculate the contrastive ranking loss. L .

[0035] Specifically, paired positive and negative samples (i.e., comparative teaching samples) are placed in context. Context Forming a triple The contrastive ranking loss is calculated by inputting the contrastive loss function (instead of the traditional cross-entropy loss function). L .

[0036] The comparison ranking loss function mentioned above can be:

[0037] in, L To compare the ranking loss; σ The Sigmoid activation function maps the score difference between the positive rescue path and the negative error path to the (0,1) interval, representing the prediction probability that "the positive path is better than the negative path". By maximizing this probability (i.e. minimizing the negative log-likelihood), the student model is forced to widen the decision boundary between positive and negative samples in the feature space. β It is the reciprocal of a preset adjustable contrast scaling factor or temperature parameter, used to control the sensitivity of the loss function to score differences; R () represents the score for the path.

[0038] In the above formula, the larger β A higher value amplifies the difference signal between positive and negative paths, prompting the student model to learn the error correction logic more aggressively; a smaller value... β The value serves to smooth the gradient, preventing training instability caused by individual outliers. In the embodiments of this application, β These are preset, adjustable hyperparameters that can be set according to the convergence difficulty of the specific training task (e.g., β The value range is typically between 0.1 and 1.0.

[0039] The purpose of using this contrastive ranking loss function is to optimize the decision boundary in the feature space, specifically to achieve the following objectives: 1) Positive incentive: improve the student model's selection of the rescue path. Path pos 1) The probability of [something]; 2) Negative inhibition: significantly reduces the student model's continuation of the wrong path. Path neg3) Feature separation: Maximize the feature distance between wrong decision and correct rescue decision in the latent space, that is, maximize the decision boundary between "teacher rescue path" and "student wrong path".

[0040] S105, Train the student model using the contrastive ranking loss to obtain the trained student model.

[0041] In this embodiment of the application, the comparison ranking loss is calculated according to step S104. L The student model's parameters are adjusted, and training is iterated multiple times until the training termination condition is met (e.g., ...). L Training ends when the loss value is less than a preset loss threshold, and the final student model is used as the trained student model.

[0042] During training, minimize the contrastive ranking loss. L With the training objective of maximizing the difference between positive and negative paths in the feature space, the student model is trained iteratively through backpropagation, enabling the student model to internalize the error correction logic of the teacher model, thereby achieving robustness in recovering from local errors during the inference process.

[0043] In summary, the knowledge distillation-based model training method of this application, when training and correcting the student model, considers a serious logical deviation in the student model when the difference between the student's answer label and the teacher's answer label generated by the teacher model is equal to or greater than a difference threshold. This accurately identifies key error nodes and improves the reasoning performance of the student model. Based on the error states of key error nodes, a comparison relationship between error paths and recovery paths is constructed, allowing the student model to fully differentiate between incorrect and correct decisions in the feature space. This encourages the student model to learn error correction logic more aggressively and also informs the student model of the relatively optimal remedial measures in an already erroneous state. Therefore, it effectively teaches the student model to recover from an erroneous state, further improving the reasoning performance of the student model. Training based on the contrastive ranking loss function not only maximizes the likelihood probability of the teacher path but also maximizes the decision boundary between the "teacher recovery path" and the "student error path," forcing the student model to learn error correction capabilities, further improving the reasoning performance of the student model. The teacher model is only triggered to take over generation when the difference between the student model output and the standard distribution is equal to or greater than a preset threshold, rather than generating a teacher model indiscriminately for the entire sequence, thus improving training efficiency.

[0044] To clearly illustrate the knowledge distillation-based model training method of this application's embodiments, the following will be combined with... Figure 2 The overall process of the knowledge distillation-based model training method according to embodiments of this application is described in detail. Figure 2As shown, the process includes: inputting question prompts (including the question and context) into the student model; the student model performing autonomous reasoning based on the question prompts, generating tokens step by step; detecting key divergences at each generation step; if the difference between the tokens output by the student model and the teacher's target distribution is less than a difference threshold, the student model continues autonomous reasoning; if the difference between the tokens output by the student model and the teacher's target distribution is greater than or equal to the difference threshold, a key error state S is detected. err This triggers the dynamic forking mechanism; negative branch. Path neg Obtained by preserving the student's error continuation path; positive branch Path pos Based on the teacher model S err Takeover generation (It should be noted here that the teacher model does not reset inference, but is based on the error state S) err (to perform rescue); the comparison and optimization module constructs pairs of samples { Context, S err , Path neg , Path pos}; Calculate the contrastive loss function The model parameters of the student model are updated through backpropagation, and the next iteration is carried out until the training termination condition is met.

[0045] Figure 3 This is a schematic diagram of a model training device based on knowledge distillation, provided as an embodiment of this application. Figure 3 As shown, the knowledge distillation-based model training device 300 of this application embodiment may specifically include: a generation module 301, a first calculation module 302, a determination module 303, a second calculation module 304, and a training module 305. Wherein: The generation module 301 is used to input question prompts into the student model. The student model generates student answer tags step by step based on the question prompts. The question prompts include the question and the context of the question.

[0046] The first calculation module 302 is used to calculate the difference between the student answer tag and the teacher answer tag generated by the teacher model for each student answer tag.

[0047] The determination module 303 is used to determine the current state of the student model as the target divergence error state if the difference is equal to or greater than a preset difference threshold.

[0048] The second calculation module 304 is used to calculate the comparative ranking loss based on the error path generated by the student model based on the target divergence error state and the rescue path generated by the teacher model based on the target divergence error state.

[0049] Training module 305 is used to train the student model based on the contrastive ranking loss to obtain the trained student model.

[0050] In the embodiments of this application, the specific process by which each module and unit in the knowledge distillation-based model training device implements its function can be found in the relevant descriptions in the above embodiments of the knowledge distillation-based model training method, and will not be repeated here.

[0051] In summary, the knowledge distillation-based model training device of this application, when training and correcting the student model, considers a serious logical deviation in the student model when the difference between the student's answer tag and the teacher's answer tag generated by the teacher model is equal to or greater than a difference threshold. This accurately identifies key error nodes and improves the reasoning performance of the student model. Based on the error state of the key error node, a comparison relationship between the error path and the recovery path is constructed, allowing the student model to fully differentiate between incorrect and correct decisions in the feature space. This encourages the student model to learn error correction logic more aggressively and also informs the student model of the relatively optimal remedial measures in an already erroneous state. Therefore, it effectively teaches the student model to recover from the error state, further improving the reasoning performance of the student model. Training based on the contrastive ranking loss function not only maximizes the likelihood probability of the teacher path but also maximizes the decision boundary between the "teacher recovery path" and the "student error path," forcing the student model to learn error correction capabilities, further improving the reasoning performance of the student model. The teacher model is only triggered to take over generation when the difference between the student model output and the standard distribution is equal to or greater than a preset threshold, rather than generating a teacher model indiscriminately for the entire sequence, thus improving training efficiency.

[0052] This application also provides an electronic device. For example... Figure 4 As shown, the electronic device 400 can vary considerably due to differences in configuration or performance. It may include one or more processors 401 and memory 402, with memory 402 storing one or more programs or instructions. Memory 402 may be temporary or persistent storage. The application program stored in memory 402 may include one or more modules (not shown), each module including a series of computer-executable instructions for the electronic device 400. Furthermore, processor 401 may be configured to communicate with memory 402 and execute the series of computer-executable instructions stored in memory 402 on the electronic device 400. The electronic device 400 may also include one or more power supplies 403, one or more wired or wireless network interfaces 404, one or more input / output interfaces 405, and one or more keyboards 406.

[0053] Specifically, in the embodiments of this application, the electronic device includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of any of the above-described embodiments of the model training method based on knowledge distillation.

[0054] The electronic device in this application, when training and correcting a student model, considers a serious logical deviation in the student model when the difference between the student's answer marker and the teacher's answer marker generated by the teacher model is equal to or greater than a difference threshold. This accurately identifies key error nodes and improves the reasoning performance of the student model. Based on the error states of key error nodes, a comparison relationship between error paths and recovery paths is constructed, allowing the student model to significantly differentiate between incorrect and correct decisions in the feature space. This encourages the student model to learn error correction logic more aggressively and also informs the student model of the relatively optimal remedial measures in an already erroneous state. Therefore, it effectively teaches the student model to recover from an erroneous state, further improving the reasoning performance of the student model. Training based on the contrastive ranking loss function not only maximizes the likelihood probability of the teacher path but also maximizes the decision boundary between the "teacher recovery path" and the "student error path," forcing the student model to learn error correction capabilities and further improving the reasoning performance of the student model. The teacher model is only triggered to take over generation when the difference between the student model output and the standard distribution is equal to or greater than a preset threshold, rather than generating a teacher model indiscriminately for the entire sequence, thus improving training efficiency.

[0055] This application also proposes a readable storage medium storing one or more computer programs or instructions that, when executed by a processor in an electronic device, enable the processor in the electronic device to perform the steps of any of the above-described embodiments of the knowledge distillation-based model training method.

[0056] The readable storage medium of this application, when training and correcting a student model, considers a serious logical deviation in the student model when the difference between the student's answer marker and the teacher's answer marker generated by the teacher model is equal to or greater than a difference threshold. This accurately identifies key error nodes and improves the reasoning performance of the student model. Based on the error states of key error nodes, a comparison relationship between error paths and recovery paths is constructed, allowing the student model to significantly differentiate between incorrect and correct decisions in the feature space. This encourages the student model to learn error correction logic more aggressively and also informs the student model of the relatively optimal remedial measures in an already erroneous state. Therefore, it effectively teaches the student model to recover from an erroneous state, further improving the reasoning performance of the student model. Training based on the contrastive ranking loss function not only maximizes the likelihood probability of the teacher path but also maximizes the decision boundary between the "teacher recovery path" and the "student error path," forcing the student model to learn error correction capabilities and further improving the reasoning performance of the student model. The teacher model is only triggered to take over generation when the difference between the student model output and the standard distribution is equal to or greater than a preset threshold, rather than generating a teacher model indiscriminately for the entire sequence, thus improving training efficiency.

[0057] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0058] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0059] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0060] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0061] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0062] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0063] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0064] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0065] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0066] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0067] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0068] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0069] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A model training method based on knowledge distillation, characterized in that, include: The question prompts are input into the student model, which then generates student answer tags step by step based on the question prompts. The question prompts include the question and the context of the question. For each student answer tag, calculate the difference between the student answer tag and the teacher answer tag generated by the teacher model; If the difference is equal to or greater than a preset difference threshold, then the current state of the student model is determined as a target divergence error state. The comparative ranking loss is calculated based on the error path generated by the student model based on the target divergence error state and the rescue path generated by the teacher model based on the target divergence error state. The student model is trained using the contrastive ranking loss to obtain the trained student model.

2. The method according to claim 1, characterized in that, The calculation of the difference between the student's answer tag and the teacher's answer tag generated by the teacher model includes: Calculate the KL divergence and / or confidence difference between the student's answer label and the teacher's answer label.

3. The method according to claim 1, characterized in that, Also includes: If the difference is less than the difference threshold, the student model is controlled to continue generating student answer tags step by step.

4. The method according to claim 1, characterized in that, The step of calculating the comparative ranking loss based on the error paths generated by the student model based on the target divergence error state and the rescue paths generated by the teacher model based on the target divergence error state includes: The erroneous path is used as a negative sample, and the rescue path is used as a positive sample to construct a pair of comparative teaching samples; The comparative teaching samples and the context are input into the comparative ranking loss function to calculate the comparative ranking loss.

5. The method according to claim 4, characterized in that, The comparison ranking loss function is: Among them, the L For the comparison ranking loss, the σ The Sigmoid activation function, the β The preset adjustable contrast scaling factor, the R () represents the score for the path.

6. The method according to claim 5, characterized in that, Also includes: Based on the convergence difficulty of the training task, the following settings are configured: β .

7. The method according to claim 1, characterized in that, The step of training the student model based on the contrastive ranking loss to obtain the trained student model includes: With minimizing the contrastive ranking loss as the training objective, the student model is backpropagated and iteratively trained to obtain the trained student model.

8. A model training device based on knowledge distillation, characterized in that, include: A generation module is used to input question prompts into a student model, which then generates student answer tags step by step based on the question prompts. The question prompts include the question and the context of the question. The first calculation module is used to calculate the difference between the student answer tag and the teacher answer tag generated by the teacher model for each student answer tag; The determination module is used to determine the current state of the student model as a target divergence error state if the difference is equal to or greater than a preset difference threshold. The second calculation module is used to calculate the comparative ranking loss based on the error path generated by the student model based on the target divergence error state and the rescue path generated by the teacher model based on the target divergence error state. The training module is used to train the student model based on the contrastive ranking loss to obtain the trained student model.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method as described in any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-7.

Images