Teacher model knowledge distillation, student model application method and device, and electronic device

Abstract

This specification provides a method, apparatus, and electronic device for knowledge distillation of a teacher model and application of a student model. The teacher model knowledge distillation method includes: acquiring a teacher model trained on a thought chain text dataset, and a student model corresponding to the teacher model to be knowledge distilled; wherein the student model integrates an implicit reasoning extractor, which includes an extractor based on implicit reasoning; performing implicit reasoning knowledge distillation on the student model based on the teacher model, using the implicit reasoning extractor to perform implicit reasoning calculations on the thought chain semantic vectors learned by the teacher model for step-by-step reasoning logic, obtaining implicit reasoning vectors, and compressing the implicit reasoning vectors into the embedding space of the student model, so that the student model possesses the step-by-step reasoning logic of the teacher model based on the implicit reasoning vectors in the embedding space.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of large language model technology, and in particular to a teacher model knowledge distillation, student model application method and device, and electronic equipment. Background Technology

[0002] A language model (LLM) typically refers to a pre-trained model with a large number of parameters and strong language understanding and generation capabilities.

[0003] Because language models have high deployment requirements and are difficult to deploy in environments with limited computing resources, knowledge distillation (KD) technology has emerged to address this issue. Knowledge distillation is the process of transferring the knowledge and reasoning capabilities of a pre-trained language model (usually called the teacher model) to a smaller model (usually called the student model). It plays a significant role in promoting the popularization of artificial intelligence and optimizing resources. First, through distillation, the student model can retain the core knowledge and complex reasoning capabilities of the teacher model, enabling high-performance intelligent applications even in environments with limited computing resources. This not only reduces the computational and storage costs of the model but also allows AI applications to be deployed more widely to edge devices and mobile terminals, improving the model's versatility in real-world scenarios.

[0004] In some scenarios involving complex reasoning, language models are often combined with Chain-of-Thought (CoT) to enhance the reasoning ability of language models through "multi-step reasoning" using CoT, making the reasoning results output by language models more accurate.

[0005] While teacher models trained using thought chains exhibit accurate judgment in complex scenarios, student models obtained through knowledge distillation of these models lose this accuracy. This is because traditional knowledge distillation uses thought chains only as training aids, completely discarding the reasoning process when deploying the student model. This causes the student model to degenerate into a "black box classifier," unable to replicate the multi-step reasoning of the teacher model, thus losing its accurate judgment in complex scenarios. Summary of the Invention

[0006] This specification provides an embodiment of a teacher model knowledge distillation, a student model application method, apparatus, and electronic device.

[0007] According to a first aspect of the embodiments of this specification, a teacher model knowledge distillation method is provided, the method comprising: Obtain a teacher model trained on a thought chain text dataset, and a student model corresponding to the teacher model to be subjected to knowledge distillation; wherein, the student model integrates an implicit reasoning extractor, and the implicit reasoning extractor includes an extractor based on implicit reasoning. The implicit reasoning knowledge distillation is performed on the student model based on the teacher model. The implicit reasoning extractor performs implicit reasoning calculation on the semantic vector of the thought chain used for step-by-step reasoning logic learned by the teacher model to obtain implicit reasoning vectors. The implicit reasoning vectors are then compressed into the embedding space of the student model so that the student model possesses the step-by-step reasoning logic of the teacher model based on the implicit reasoning vectors in the embedding space.

[0008] Optionally, obtaining the teacher model trained on the thought chain text dataset includes: Obtain the original training dataset and input the original training dataset into the initial language model, and guide the initial language model to generate the thought chain text corresponding to each perspective from at least two predefined different perspectives; The generated thought chain text corresponding to each perspective is used as a new training dataset, and the initial language model is trained under supervision based on the new training dataset to obtain a teacher model with multi-perspective reasoning logic.

[0009] Optionally, the step of using the generated thought chain text corresponding to each perspective as a new training dataset includes: The standardized prompt template is used to convert the thought chain text corresponding to each perspective into standardized thought chain text, and the standardized thought chain text corresponding to each perspective is used as a new training dataset.

[0010] Optional, also includes: The new training dataset is input into the trained teacher model to obtain the inference dataset output by the teacher model; The first negative sample that conflicts under different perspectives and the first positive sample that does not conflict under different perspectives are selected from the inference dataset to construct cross-perspective preference data. From the inference dataset, select the second negative sample that conflicts under the same perspective and the second positive sample that does not conflict under the same perspective to construct the same perspective preference data; The cross-perspective preference data and the same-perspective preference data are used as secondary training datasets, and the teacher model is trained according to the preference optimization algorithm.

[0011] Optionally, the implicit inference vectors are compressed into the embedding space of the student model, so that the student model possesses the step-by-step inference capability of the teacher model based on the implicit inference vectors in the embedding space, including: The encoder of the student model encodes the thought chain text generated by the teacher model into a fixed-dimensional semantic vector. The distance between the implicit inference vector and the fixed-dimensional semantic vector is calculated, and the student model is guided to learn the inference logic of the teacher model in the embedding space with the goal of minimizing the distance.

[0012] Optionally, the student model integrates an implicit inference extractor, including: The implicit inference extractor is embedded in the output layer of the encoder of the student model.

[0013] Optionally, the implicit reasoning extractor may employ a multilayer perceptron, poly-encoder, or graph attention network structure.

[0014] Optionally, the at least two different perspectives include a user intent perspective, a structured analysis perspective, and a scenario rule perspective.

[0015] According to a second aspect of the embodiments of this specification, a method for applying a student model is provided, wherein the student model is obtained through the aforementioned teacher model knowledge distillation method; the method includes: Receive the request text input by the input party; The request text is input into the student model, where the encoder generates a word sequence representation. An implicit inference vector is extracted from the sequence representation by the implicit inference extractor integrated in the student model. The implicit inference vector is concatenated with the sentence vector and then input into the classifier. The classifier outputs the inference result based on the concatenated vector. The reasoning result is returned to the input party.

[0016] According to a third aspect of the embodiments of this specification, a teacher model knowledge distillation apparatus is provided, the apparatus comprising: The acquisition unit acquires a teacher model trained on a thought chain text dataset and a student model corresponding to the teacher model to be subjected to knowledge distillation; wherein, the student model integrates an implicit reasoning extractor, and the implicit reasoning extractor includes an extractor based on implicit reasoning. The distillation unit performs implicit reasoning knowledge distillation on the student model based on the teacher model. The implicit reasoning extractor performs implicit reasoning calculations on the semantic vectors of the thought chain used for step-by-step reasoning logic learned by the teacher model to obtain implicit reasoning vectors. The implicit reasoning vectors are then compressed into the embedding space of the student model so that the student model possesses the step-by-step reasoning logic of the teacher model based on the implicit reasoning vectors in the embedding space.

[0017] According to a fourth aspect of the embodiments of this specification, a student model application device is provided, wherein the student model is obtained through the aforementioned teacher model knowledge distillation method; the method includes: The receiving unit receives the request text input by the input party; The reasoning unit inputs the request text into the student model, where the encoder generates a word sequence representation, and the implicit reasoning extractor integrated in the student model extracts an implicit reasoning vector from the sequence representation. The implicit reasoning vector is then concatenated with the sentence vector and input into the classifier, which outputs the reasoning result based on the concatenated vector. The return unit returns the reasoning result to the input party.

[0018] According to a fifth aspect of the embodiments of this specification, an electronic device is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to perform any of the above-described methods for teacher model knowledge distillation or student model application.

[0019] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, wherein computer-readable instructions are stored on the computer-readable storage medium, and the computer-readable instructions, when executed by a processor, implement any of the above-described methods for teacher model knowledge distillation or student model application.

[0020] This specification provides an embodiment of a knowledge distillation scheme for a teacher model. For a teacher model trained on a thought chain text dataset, it proposes an implicit reasoning knowledge distillation (LRKD) scheme by integrating an implicit reasoning extractor into the student model corresponding to the teacher model. Specifically, when performing implicit reasoning knowledge distillation on the teacher model, the implicit reasoning extractor integrated into the student model performs implicit reasoning calculations on the thought chain semantic vectors learned by the teacher model for step-by-step reasoning logic, obtaining implicit reasoning vectors. These implicit reasoning vectors are then compressed into the embedding space of the student model. Based on the implicit reasoning vectors in the embedding space, the student model can internalize the step-by-step reasoning logic without text generation during reasoning, avoiding the problem of traditional knowledge distillation discarding the reasoning process when generating the student model, resulting in the student model being unable to reproduce the step-by-step reasoning of the teacher model.

[0021] Furthermore, a Multi-Perspective Chain-of-Thought (MPCoT) method is proposed to train teacher models, overcoming the limitations of traditional teacher models' single-perspective reasoning. Specifically, during teacher model training, reasoning paths are generated collaboratively through at least two different perspectives, enabling the teacher model to possess multi-perspective reasoning logic capabilities. This enhances the robustness of both teacher and student models to complex requests (such as long-tail and fuzzy requests), effectively addressing the reasoning bias problem caused by a single perspective.

[0022] On the other hand, an end-to-end training and deployment framework (MPCoT-LRKD) is proposed by combining Multi-Perspective Thinking Chain (MPCoT) with Implicit Reasoning Knowledge Distillation (LRKD). Through multi-perspective thinking chain training and implicit reasoning distillation, the complex reasoning ability learned by the teacher model can be efficiently transferred to the student model of lightweight online service. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of an end-to-end training and deployment framework provided in one embodiment of this specification; Figure 2 This is a flowchart of a teacher model knowledge distillation method provided in one embodiment of this specification; Figure 3 This is a schematic diagram of the generation of a multi-perspective thought chain provided in one embodiment of this specification; Figure 4 This is a schematic diagram of cross-view preference optimization provided in one embodiment of this specification; Figure 5 This is a schematic diagram of implicit reasoning knowledge distillation provided in one embodiment of this specification; Figure 6 This is a flowchart of a student model application method provided in one embodiment of this specification; Figure 7 This is a structural diagram of an electronic device provided in one embodiment of this specification; Figure 8 This is a module of the teacher model knowledge distillation device provided in one embodiment of this specification; Figure 9 This is a module of the student model application device provided in one embodiment of this specification. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0025] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0026] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0027] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0028] Please refer to the following: Figure 1 This document presents a schematic diagram of the end-to-end training and deployment framework (MPCoT-LRKD) provided in this manual. The framework can be divided into a model training phase and a model deployment phase. The model training phase is further divided into Multi-Perspective Mind Chain (MPCoT) training of the teacher model and Implicit Reasoning Knowledge Distillation (LRKD) generation of the student model.

[0029] like Figure 1As shown, the multi-perspective chain of thought differs from the traditional single-perspective chain of thought (SPCoT). When training the teacher model, it collaboratively generates reasoning paths through at least two different perspectives, enabling the teacher model to possess multi-perspective reasoning logic capabilities. This enhances the robustness of both the teacher and student models to reasoning on complex requests (such as long-tail and fuzzy requests), effectively addressing the reasoning bias problem caused by a single perspective.

[0030] After training the teacher model, it is necessary to transform the teacher model into a lightweight student model, a process that requires the use of knowledge distillation techniques.

[0031] This specification proposes an implicit reasoning knowledge distillation method that differs from traditional knowledge distillation. This implicit reasoning knowledge distillation is achieved by designing a lightweight Latent Reasoning Extractor (LRE) and integrating it into the student model. Specifically, when performing implicit reasoning knowledge distillation on the teacher model, the implicit reasoning extractor integrated into the student model performs implicit reasoning calculations on the semantic vectors of the thought chain used for step-by-step reasoning logic learned by the teacher model, obtaining implicit reasoning vectors. These implicit reasoning vectors are then compressed into the embedding space of the student model. Based on the implicit reasoning vectors in the embedding space, the student model can internalize the step-by-step reasoning logic without text generation during reasoning, avoiding the problem of traditional knowledge distillation discarding the reasoning process when generating the student model, resulting in the student model being unable to reproduce the step-by-step reasoning of the teacher model.

[0032] After training the lightweight student model, the implicit inference extractor in the student model is fully preserved during the model deployment phase. This deployed student model, while already having lower inference latency compared to the teacher model and meeting the requirements of high concurrency and low latency online services, ensures the accuracy of inference results by internalizing the step-by-step inference logic.

[0033] The aforementioned end-to-end training and deployment framework, through multi-perspective mind chain training and implicit reasoning distillation, can efficiently transfer the complex reasoning abilities learned by the teacher model to the student model of a lightweight online service.

[0034] Please refer to the following: Figure 2 This document introduces an embodiment of the teacher model knowledge distillation method provided in this specification. The method includes: Step 210: Obtain a teacher model trained on the Mind Chain text dataset, and a student model corresponding to the teacher model to be subjected to knowledge distillation; wherein, the student model integrates an implicit reasoning extractor, and the implicit reasoning extractor includes an extractor based on implicit reasoning.

[0035] This specification addresses the problem that student models lose their accurate judgment in complex scenarios when using a teacher model combined with the MindChain framework for knowledge distillation. Therefore, the teacher model used for knowledge distillation is a teacher model trained on the MindChain text dataset.

[0036] In one exemplary embodiment, obtaining the teacher model trained on the MindChain text dataset in step 210 may include: Step 211: Obtain the original training dataset and input the original training dataset into the initial language model to guide the initial language model to generate the thought chain text corresponding to each perspective from at least two predefined different perspectives.

[0037] Step 213: Use the generated thought chain text corresponding to each perspective as a new training dataset, and perform supervised training on the initial language model based on the new training dataset to obtain a teacher model with multi-perspective reasoning logic.

[0038] In this manual, different training datasets are required for different types of scenarios, and different perspectives need to be predefined according to the characteristics of the actual scenario.

[0039] For example, different perspectives can be divided into user intent perspective, structured analysis perspective, and scenario rule perspective; The user intent perspective is used to measure whether the user intent and the reasoning result meet the functional requirements of the relevant scenario. The structured analysis perspective is used to measure the degree of matching between user intent and the attributes of the reasoning result; The scenario rules perspective is used to measure the degree of ambiguity in scenario specifications.

[0040] The following example uses e-commerce search. The original training dataset can refer to the e-commerce search training dataset, and the training samples in this dataset can be sample pairs of queries and products. At least two predefined perspectives can be used as follows: Figure 3 The user intent perspective (e.g., measuring whether the query for "dog pool" and the product for "puppy bath tub" meet the user's core function), the structured analysis perspective (e.g., the matching degree of the attributes of "puppy bath tub" product such as size, category, and material), and the scenario rule perspective (e.g., the classification rules of "accessories and main products") are shown.

[0041] like Figure 3As shown, after the training dataset is input into the initial language model, it is guided to generate thought chain text from three perspectives—user intent, structured analysis, and scene rules—through predefined viewpoints. The generated thought chain text is then used as a new training dataset, and the initial language model is subjected to supervised training based on this new dataset. This process enables the initial language model to internalize multi-perspective reasoning capabilities, thereby generating a teacher model that can automatically generate reasonable reasoning paths covering multiple perspectives such as user intent, attribute matching, and scene rules based on input. Because the teacher model possesses multi-perspective reasoning logic, it enhances the robustness of the teacher model to reasoning on complex requests (such as long-tail and fuzzy requests), effectively solving the reasoning bias problem caused by a single perspective.

[0042] In an exemplary embodiment, step 213 above, which uses the generated thought chain text corresponding to each perspective as a new training dataset, may include: The standardized prompt template is used to convert the thought chain text corresponding to each perspective into standardized thought chain text, and the standardized thought chain text corresponding to each perspective is used as a new training dataset.

[0043] In practical applications, thought chain texts generated from different perspectives may have structural differences and cannot be uniformly structured. Therefore, by using standardized prompt templates, thought chain texts from different perspectives can be converted into standardized thought chain texts, thereby improving the quality of the training dataset for supervised training of the initial language model.

[0044] In one exemplary embodiment, after training the teacher model, the method may further include: The new training dataset is input into the trained teacher model to obtain the inference dataset output by the teacher model; The first negative sample that conflicts under different perspectives and the first positive sample that does not conflict under different perspectives are selected from the inference dataset to construct cross-perspective preference data. From the inference dataset, select the second negative sample that conflicts under the same perspective and the second positive sample that does not conflict under the same perspective to construct the same perspective preference data; The cross-perspective preference data and the same-perspective preference data are used as secondary training datasets, and the teacher model is trained according to the preference optimization algorithm.

[0045] This specification proposes a training method for a teacher model based on Direct Preference Optimization (DPO), such as... Figure 4As shown, by filtering the first negative samples (i.e., samples whose reasoning conclusions generated by different perspectives contradict each other) and the first positive samples (which do not conflict with each other) in the reasoning dataset output by the teacher model, the first negative samples and the first positive samples are constructed as cross-perspective preference data. At the same time, by filtering the second negative samples (which conflict with each other) and the second positive samples (which do not conflict with each other) in the reasoning dataset, the second negative samples and the second positive samples are constructed as same-perspective preference data. The teacher model is then trained a second time using a preference optimization algorithm, enabling the teacher model to learn to dynamically weigh and select the ideal reasoning perspective based on the specific context of the thought chain text, thereby improving the robustness of judgment in complex scenarios and ultimately outputting a teacher model with multi-perspective collaborative reasoning capabilities.

[0046] Step 220: Perform implicit reasoning knowledge distillation on the student model based on the teacher model. Use the implicit reasoning extractor to perform implicit reasoning calculation on the semantic vector of the thought chain used for step-by-step reasoning logic learned by the teacher model to obtain implicit reasoning vectors. Compress the implicit reasoning vectors into the embedding space of the student model so that the student model has the step-by-step reasoning logic of the teacher model based on the implicit reasoning vectors in the embedding space.

[0047] In this specification, the student model can be based on a Bidirectional Encoder Representations from Transformers (BERT) architecture. An implicit inference extractor is embedded in the output layer of the encoder of the student model. This implicit inference extractor can employ a Multilayer Perceptron (MLP), Poly-Encoder, or Graph Attention Network (GAT) structure.

[0048] Generally, MLPs are better suited for processing structured data than Poly-Encoders and GATs, while GATs are better suited for graph data processing, and Poly-Encoders are better suited for text data matching. Therefore, in practical applications, the structure of the implicit inference extractor can be flexibly selected based on the specific application scenarios of the teacher and student models. For example, the data types that the teacher and / or student models need to process can be identified, and the structure of the implicit inference extractor used in the student model can be determined based on these data types. This allows for the adoption of an implicit inference extractor structure more suitable for processing the data that the model needs to process, thereby improving the data processing performance of the implicit inference extractor.

[0049] In this specification, the embedding space of the student model generally refers to the representation learned by the student model in the feature space during training. In the implicit reasoning knowledge distillation proposed in this application, the high-dimensional semantic vectors of the thought chain learned by the teacher model are compressed into low-dimensional implicit reasoning vectors through implicit reasoning. The student model learns the embedding space containing the step-by-step reasoning logic of the teacher model by minimizing the difference between its output and this implicit reasoning vector. This process is called embedding space internalization of implicit reasoning vectors, so that the student model can possess the step-by-step reasoning logic of the teacher model based on the implicit reasoning vectors internalized in the embedding space.

[0050] In an exemplary embodiment, step 220, which compresses the implicit inference vector into the embedding space of the student model so that the student model possesses the step-by-step inference capability of the teacher model based on the implicit inference vector in the embedding space, may include: The encoder of the student model encodes the thought chain text generated by the teacher model into a fixed-dimensional semantic vector. The distance between the implicit inference vector and the fixed-dimensional semantic vector is calculated, and the student model is guided to learn the inference logic of the teacher model in the embedding space with the goal of minimizing the distance.

[0051] like Figure 5 As shown, in the process of training the student model by performing implicit reasoning knowledge distillation on the teacher model, an encoder (such as BGE-M3) can be used to encode the thought chain text output by the teacher model into a fixed-dimensional semantic vector. The student model simultaneously optimizes two losses: first, the standard relevance classification loss, ensuring accurate basic matching judgments; and second, the implicit inference guidance loss, which is calculated by using the implicit inference vector generated by the implicit inference extractor. With the semantic vector The distance between the two models is calculated, and minimizing this distance is used as the optimization objective to guide the student model to learn the reasoning logic of the teacher model within the embedding space. This process enables the student model to internalize higher-order reasoning semantics without text generation.

[0052] The implicit inference guidance loss can be expressed as Formula 1, and the relevance classification loss can be expressed as Formula 2: Formula 1 Formula 2 The embodiments described above propose an implicit reasoning knowledge distillation (LRKD) method for teacher models trained on thought chain text datasets. This method integrates an implicit reasoning extractor into the student model corresponding to the teacher model, which is then subjected to knowledge distillation. Specifically, when performing implicit reasoning knowledge distillation on the teacher model, the implicit reasoning extractor integrated into the student model performs implicit reasoning calculations on the thought chain semantic vectors learned by the teacher model for step-by-step reasoning logic, obtaining implicit reasoning vectors. These implicit reasoning vectors are then compressed into the embedding space of the student model. Based on the implicit reasoning vectors in the embedding space, the student model can internalize the distributed reasoning logic without text generation during reasoning, avoiding the problem of traditional knowledge distillation discarding the reasoning process when generating the student model, which leads to the student model being unable to reproduce the step-by-step reasoning of the teacher model.

[0053] After introducing the method of knowledge distillation from the teacher model to obtain the student model, such as Figure 1 As shown, the model can then be deployed for student model application.

[0054] Please refer to the following: Figure 7 This document introduces an embodiment of the student model application method provided, wherein the student model is obtained through the aforementioned teacher model knowledge distillation, and the method includes: Step 310: Receive the request text input by the input party; Step 320: Input the request text into the student model, where the encoder generates a word sequence representation, and the implicit reasoning extractor integrated in the student model extracts an implicit reasoning vector from the sequence representation. The implicit reasoning vector is then concatenated with the sentence vector and input into the classifier, which outputs the reasoning result based on the concatenated vector. Step 330: Return the reasoning result to the input party.

[0055] The implicit inference extractor from the student model is fully preserved during the model deployment phase. When the student model performs inference, the encoder generates a token sequence representation based on the input request text. The implicit inference extractor aggregates key semantic information from this token sequence representation to generate an implicit inference vector. This implicit inference vector is then concatenated with the sentence vector (CLS) output by the student model's BERT and input into the classifier to output the final inference result. The entire inference process involves no text generation, and the measured inference latency increment is extremely small (e.g., only about 0.46 milliseconds for the Poly-Encoder architecture and about 16.5 milliseconds for the GAT architecture), meeting the requirements of high concurrency and low latency online services. Furthermore, because the student model internalizes the step-by-step inference logic of the teacher model, the accuracy of the inference results is ensured.

[0056] After introducing the methods of exemplary embodiments of this specification, the apparatus, electronic devices, and computer-readable storage media of exemplary embodiments of this specification will be described next.

[0057] In an exemplary embodiment of this specification, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor implements the method described above by executing the executable instructions.

[0058] Figure 6 This is a schematic structural diagram of an electronic device provided in an exemplary embodiment. Please refer to... Figure 6 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for different scenarios. One or more embodiments of this specification can be implemented in software, such as by the processor reading the corresponding computer program from non-volatile memory into memory and then running it. Of course, besides software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software. That is to say, the execution entity of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0059] Please see Figure 8 This is a block diagram of a teacher model knowledge distillation device provided in one embodiment of this specification. The device corresponds to... Figure 2 The illustrated embodiment includes: The acquisition unit 810 acquires a teacher model trained on the Mind Chain text dataset and a student model corresponding to the teacher model to be subjected to knowledge distillation; wherein, the student model integrates an implicit reasoning extractor, and the implicit reasoning extractor includes an extractor based on implicit reasoning. Distillation unit 820 performs implicit reasoning knowledge distillation on the student model based on the teacher model. The implicit reasoning extractor performs implicit reasoning calculation on the semantic vector of the thought chain used for step-by-step reasoning logic learned by the teacher model to obtain implicit reasoning vectors. The implicit reasoning vectors are then compressed into the embedding space of the student model so that the student model possesses the step-by-step reasoning logic of the teacher model based on the implicit reasoning vectors in the embedding space.

[0060] Optionally, the acquisition unit 810 further includes: Generate sub-units, obtain the original training dataset, and input the original training dataset into the initial language model to guide the initial language model to generate thought chain text corresponding to each perspective from at least two predefined different perspectives; The training subunit uses the generated thought chain text corresponding to each perspective as a new training dataset, and performs supervised training on the initial language model based on the new training dataset to obtain a teacher model with multi-perspective reasoning logic.

[0061] Optionally, the training subunit is further configured to use a standardized prompt template to convert the thought chain text corresponding to each perspective into standardized thought chain text, and use the standardized thought chain text corresponding to each perspective as a new training dataset.

[0062] Optionally, the training subunit may also include: The secondary training subunit inputs the new training dataset into the trained teacher model to obtain the inference dataset output by the teacher model; it then selects first negative samples with conflicts from different perspectives and first positive samples without conflicts from different perspectives from the inference dataset to construct cross-perspective preference data; it further selects second negative samples with conflicts from the same perspective and second positive samples without conflicts from the same perspective from the inference dataset to construct same-perspective preference data; and it uses the cross-perspective preference data and the same-perspective preference data as the secondary training dataset, and trains the teacher model according to the preference optimization algorithm.

[0063] Optionally, the distillation unit 820 further includes compressing the implicit inference vector into the embedding space of the student model, so that the student model possesses the step-by-step inference capability of the teacher model based on the implicit inference vector in the embedding space: The encoder of the student model encodes the thought chain text generated by the teacher model into a fixed-dimensional semantic vector; calculates the distance between the implicit inference vector and the fixed-dimensional semantic vector, and guides the student model to learn the inference logic of the teacher model in the embedding space with the goal of minimizing the distance.

[0064] Optionally, the student model integrates an implicit inference extractor, including: The implicit inference extractor is embedded in the output layer of the encoder of the student model.

[0065] Optionally, the implicit reasoning extractor may employ a multilayer perceptron, poly-encoder, or graph attention network structure.

[0066] Optionally, the at least two different perspectives include a user intent perspective, a structured analysis perspective, and a scenario rule perspective.

[0067] Please see Figure 9 This is a block diagram of a search device provided in one embodiment of this specification, the device corresponding to... Figure 6 The illustrated embodiment includes: The receiving unit 910 receives the request text input by the input party; The reasoning unit 920 inputs the request text into the student model, where the encoder generates a word sequence representation, and the implicit reasoning extractor integrated in the student model extracts an implicit reasoning vector from the sequence representation. The implicit reasoning vector is then concatenated with the sentence vector and input into the classifier, which outputs the reasoning result based on the concatenated vector. Return unit 930 returns the reasoning result to the input party.

[0068] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method. For relevant details, please refer to the description of the method implementation method. It will not be repeated here.

[0069] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the units or modules can be selected to achieve the purpose of the solution described in this specification, depending on actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0070] The systems, devices, modules, or units described in the above embodiments can be implemented by a computer or entity, or by a product with a certain function. A typical implementation device is a computer, which can be a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0071] In exemplary embodiments of this specification, a computer-readable storage medium is also provided, on which computer-readable instructions are stored, which, when executed by a processor, implement the aforementioned method embodiments.

[0072] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0073] Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RE, or any suitable combination thereof.

[0074] Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0075] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0076] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0077] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this specification. The singular forms “a,” “described,” and “the” used in one or more embodiments of this specification and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0078] It should be understood that although the terms first, second, third, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of one or more embodiments of this specification, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to a determination," or "when," or "in the event of a determination."

[0079] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the protection scope of one or more embodiments of this specification.

Claims

1. A teacher model knowledge distillation method, the method comprising: Obtain a teacher model trained on a thought chain text dataset, and a student model corresponding to the teacher model to be subjected to knowledge distillation; wherein, the student model integrates an implicit reasoning extractor, and the implicit reasoning extractor includes an extractor based on implicit reasoning. The implicit reasoning knowledge distillation is performed on the student model based on the teacher model. The implicit reasoning extractor performs implicit reasoning calculation on the semantic vector of the thought chain used for step-by-step reasoning logic learned by the teacher model to obtain implicit reasoning vectors. The implicit reasoning vectors are then compressed into the embedding space of the student model so that the student model possesses the step-by-step reasoning logic of the teacher model based on the implicit reasoning vectors in the embedding space.

2. The method according to claim 1, wherein obtaining the teacher model trained based on the thought chain text dataset comprises: Obtain the original training dataset and input the original training dataset into the initial language model, and guide the initial language model to generate the thought chain text corresponding to each perspective from at least two predefined different perspectives; The generated thought chain text corresponding to each perspective is used as a new training dataset, and the initial language model is trained under supervision based on the new training dataset to obtain a teacher model with multi-perspective reasoning logic.

3. The method according to claim 2, wherein using the generated thought chain text corresponding to each perspective as a new training dataset includes: The standardized prompt template is used to convert the thought chain text corresponding to each perspective into standardized thought chain text, and the standardized thought chain text corresponding to each perspective is used as a new training dataset.

4. The method according to claim 3, further comprising: The new training dataset is input into the trained teacher model to obtain the inference dataset output by the teacher model; The first negative sample that conflicts under different perspectives and the first positive sample that does not conflict under different perspectives are selected from the inference dataset to construct cross-perspective preference data. From the inference dataset, select the second negative sample that conflicts under the same perspective and the second positive sample that does not conflict under the same perspective to construct the same perspective preference data; The cross-perspective preference data and the same-perspective preference data are used as secondary training datasets, and the teacher model is trained according to the preference optimization algorithm.

5. The method according to claim 1, wherein the implicit inference vector is compressed into the embedding space of the student model, so that the student model possesses the step-by-step inference capability of the teacher model based on the implicit inference vector in the embedding space, comprising: The encoder of the student model encodes the thought chain text generated by the teacher model into a fixed-dimensional semantic vector. The distance between the implicit inference vector and the fixed-dimensional semantic vector is calculated, and the student model is guided to learn the inference logic of the teacher model in the embedding space with the goal of minimizing the distance.

6. The method according to claim 5, wherein the student model integrates an implicit inference extractor, comprising: The implicit inference extractor is embedded in the output layer of the encoder of the student model.

7. The method according to claim 1, wherein the implicit reasoning extractor adopts a structure of multilayer perceptron, poly-encoder or graph attention network.

8. The method according to claim 2, wherein the at least two different perspectives include a user intent perspective, a structured analysis perspective, and a scenario rule perspective.

9. A method for applying a student model, wherein the student model is obtained through the teacher model knowledge distillation method of claim 1; the method includes: Receive the request text input by the input party; The request text is input into the student model, where the encoder generates a word sequence representation. An implicit inference vector is extracted from the sequence representation by the implicit inference extractor integrated in the student model. The implicit inference vector is concatenated with the sentence vector and then input into the classifier. The classifier outputs the inference result based on the concatenated vector. The reasoning result is returned to the input party.

10. A teacher model knowledge distillation apparatus, the apparatus comprising: The acquisition unit acquires a teacher model trained on a thought chain text dataset and a student model corresponding to the teacher model to be subjected to knowledge distillation; wherein, the student model integrates an implicit reasoning extractor, and the implicit reasoning extractor includes an extractor based on implicit reasoning. The distillation unit performs implicit reasoning knowledge distillation on the student model based on the teacher model. The implicit reasoning extractor performs implicit reasoning calculations on the semantic vectors of the thought chain used for step-by-step reasoning logic learned by the teacher model to obtain implicit reasoning vectors. The implicit reasoning vectors are then compressed into the embedding space of the student model so that the student model possesses the step-by-step reasoning logic of the teacher model based on the implicit reasoning vectors in the embedding space.

11. A student model application device, the device comprising: The receiving unit receives the request text input by the input party; The reasoning unit inputs the request text into the student model, where the encoder generates a word sequence representation, and the implicit reasoning extractor integrated in the student model extracts an implicit reasoning vector from the sequence representation. The implicit reasoning vector is then concatenated with the sentence vector and input into the classifier, which outputs the reasoning result based on the concatenated vector. The return unit returns the reasoning result to the input party.

12. An electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method according to any one of claims 1-9.

13. A computer-readable storage medium having stored thereon computer-readable instructions that, when executed by a processor, implement the method as described in any one of claims 1-9.

Images