Model distillation method and apparatus

Abstract

Provided are a model distillation method and apparatus. The method comprises: acquiring a fixed training dataset, wherein the fixed training dataset comprises a plurality of training text sequences; guiding, by means of a teacher model, a student model to generate target response text sequences on the basis of the training text sequences, so as to correct semantic differences between response text sequences output by the student model and response text sequences output by the teacher model; on the basis of the plurality of training text sequences and a target response text sequence corresponding to each of the plurality of training text sequences, obtaining a target training dataset; and on the basis of the target training dataset and the teacher model, performing distillation training on the student model. In the present application, semantic differences between outputs of the teacher model and outputs of the student model are marked and evaluated one by one at a sequence level, and errors are identified and corrected, such that the student model is guided to generate more reliable and diversified samples, thereby improving the training effect of the student model.

Description

A model distillation method and apparatus

[0001] This application claims priority to Chinese Patent Application No. 202410947507.1, filed on July 13, 2024, entitled “A Model Distillation Method and Apparatus”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence (AI) technology, and in particular to a model distillation method and apparatus. Background Technology

[0003] In the field of large language models (LLMs), the rapid advancements in autoregressive models have greatly propelled the development of text generation tasks. However, these large models face challenges in deployment and real-time applications. In contrast, smaller models perform poorly in practical generation tasks. To address this issue, knowledge distillation (KD) has been proposed, allowing the development of smaller, more efficient student models that inherit the advantages of large teacher models.

[0004] Most existing knowledge distillation work focuses on distillation of closed-source "black box" models, while "white box" distillation methods that can be used for open-source models may introduce errors by relying too much on the output generated by the student model. The limited capacity and bias of the student model may further exacerbate these errors, leading to a distorted representation of the teacher's knowledge. Summary of the Invention

[0005] The embodiments of this application provide a model distillation method and apparatus, which evaluates the semantic differences between the outputs of the teacher model and the student model at the sequence level by labeling and correcting errors, guiding the student model to generate more reliable and diverse samples, and improving the training effect of the student model.

[0006] In a first aspect, this application provides a model distillation method, including obtaining a fixed training dataset, which includes multiple training text sequences; guiding a student model to generate a target response text sequence based on the training text sequences through a teacher model, so as to correct the semantic differences between the response text sequence output by the student model and the response text sequence output by the teacher model; obtaining a target training dataset based on the multiple training text sequences and the target response text sequences corresponding to each training text sequence; and performing distillation training on the student model based on the target training dataset and the teacher model.

[0007] The model distillation method provided in this application evaluates the semantic differences between the outputs of the teacher model and the student model at the sequence level, identifies and corrects errors, guides the student model to generate more reliable and diverse samples, and improves the training effect of the student model.

[0008] In one possible implementation, the training text sequence includes a prompt text sequence, and the teacher model is a trained large language model. A specific implementation of the teacher model guiding the student model to generate a target response text sequence based on the training text sequence is as follows: the prompt text sequence is used as input to the student model, outputting a first response text sequence; the prompt text sequence and the first response text sequence are used as input to the teacher model, outputting a second response text sequence; semantically different words in the first response text sequence are labeled compared to those in the second response text sequence; the labeled first response text sequence and the prompt text sequence are used as input to the student model, outputting the target response text sequence.

[0009] In this possible implementation, by identifying tokens in the response text sequence output by the student model that are semantically different from those output by the teacher model, the identified tokens are marked, and the marked response text sequence and prompt text sequence are re-inputted into the student model to regenerate the response text sequence. This ensures that the response text sequence output by the student model is semantically close to or the same as that output by the teacher model. This guides the student model to generate more reliable and diverse samples, avoiding the problem of poor distillation results caused by the student model's own generation errors due to its limited capabilities.

[0010] In another possible implementation, a specific method for distilling the student model based on the target training dataset and the teacher model is as follows: The training text sequence is used as input to the student model to obtain a first output probability vector for each word; the prompt text sequence is used as input to the teacher model to obtain a second output probability vector for each word; a first target vector element and a second target vector element are identified in the first output probability vector to obtain a first target output probability vector, where the first target vector element is the vector element distributed in the semantically high-density region of the first output probability vector, and the second target vector element is the largest vector element in the first output probability vector; the semantically high-density region is the region where the vector element distribution probability is greater than a preset threshold; a third target vector element and a fourth target vector element are identified in the second output probability vector to obtain a second target output probability vector, where the third target vector element is the vector element distributed in the semantically high-density region of the second output probability vector, and the fourth target vector element is the largest vector element in the second output probability vector; a first loss value of the loss function is obtained based on the first and second target output probability vectors; and the weight parameters of the student model are adjusted, at least based on the first loss value, to train the student model.

[0011] At the token level, a distribution-adaptive pruning Kullback-Leibler (DAC-KL) loss function is introduced, which focuses on semantically significant regions in the output space, effectively filtering redundant information and retaining the most critical signals. This avoids the problem in related technologies where using KL-Divergence as the objective function of KD makes it difficult to capture the valuable distribution of the teacher model or overfits to high-probability distribution regions.

[0012] In another possible implementation, the distillation method of the model provided in this application further includes: using the first output probability vector and the second output probability vector as inputs to the neural network to truncate the probability vectors, outputting low quantiles and high quantiles, where the low quantiles indicate the minimum probability value boundary of the semantic high-density region and the high quantiles indicate the maximum probability value boundary of the semantic high-density region; and determining the semantic high-density region based on the low quantiles and high quantiles.

[0013] By utilizing learnable subnetworks (i.e., probabilistic vector truncated neural networks) to identify semantically high-density regions in the output space, the student model can focus on semantically significant regions in the output space, effectively filtering redundant information, retaining the most critical signals, and avoiding redundant information interfering with the student model's learning during the distillation process.

[0014] Optionally, the probabilistic vector truncated neural network can be an MLP.

[0015] In one example, the loss function also includes a regularization term, which is used to align the correlation between the output probability vectors of the student model and the teacher model using prior phrases (spans).

[0016] In another possible implementation, the value of the regularization term is determined based on the following steps: dividing the response text sequence output by the student model into word groups to obtain multiple word groups; dividing the output probability vector of the student model into multiple first output probability vector groups based on the multiple word groups; dividing the output probability vector of the teacher model into multiple second output probability vector groups based on the multiple word groups; calculating the similarity distance between each output probability vector in each of the multiple first output probability vector groups; calculating the similarity distance between each output probability vector in each of the multiple second output probability vector groups; and determining the value of the regularization term based on the similarity distances between each output probability vector in each of the first output probability vector groups and the similarity distances between each output probability vector in each of the second output probability vector groups.

[0017] At the span level, this application introduces predefined block priors to align the correlation between the output probability vectors of the student and teacher models, ensuring consistent transmission of semantic information of tokens within the span.

[0018] In another possible implementation, the loss function also includes a second loss value, which is determined based on the response text sequence generated by the student model based on the training text sequence in the target training dataset and the target response text sequence corresponding to the training text sequence.

[0019] In other words, the loss function for distillation learning of the student model in this application includes three parts. The first loss function is the DAC-KL loss function, and the second loss function can be based on the target response text sequence y in the target training dataset and the response text sequence y output by the student model. s The third term is the calculated cross-entropy loss function, and the fourth term is the regularization term (e.g., L2 loss).

[0020] Secondly, this application provides a model distillation apparatus, including an acquisition module, a semantic correction module, a construction module, and a training module. The acquisition module acquires a fixed training dataset, which includes multiple training text sequences. The semantic correction module guides a student model to generate a target response text sequence based on the training text sequences using a teacher model, thereby correcting semantic differences between the response text sequences output by the student model and those output by the teacher model. The construction module obtains a target training dataset based on the multiple training text sequences and the target response text sequences corresponding to each training text sequence. The training module performs distillation training on the student model based on the target training dataset and the teacher model.

[0021] In one possible implementation, the training text sequence includes a cue text sequence, and the teacher model is a trained large language model. The semantic correction module is specifically used to: take the cue text sequence as input to the student model and output a first response text sequence; take the cue text sequence and the first response text sequence as input to the teacher model and output a second response text sequence, wherein the lengths of the first response text sequence and the second response text sequence are equal; label the semantically different words in the first response text sequence and the second response text sequence; and take the labeled first response text sequence and the cue text sequence as input to the student model and output a target response text sequence.

[0022] In another possible implementation, the training module is specifically used for: taking the training text sequence as input to the student model to obtain a first output probability vector for each word; taking the prompt text sequence as input to the teacher model to obtain a second output probability vector for each word; identifying the first target vector element and the second target vector element in the first output probability vector to obtain a first target output probability vector, where the first target vector element is the vector element distributed in the semantically high-density region of the first output probability vector, and the second target vector element is the largest vector element in the first output probability vector, and the semantically high-density region is the region where the vector element distribution probability is greater than a preset threshold; identifying the third target vector element and the fourth target vector element in the second output probability vector to obtain a second target output probability vector, where the third target vector element is the vector element distributed in the semantically high-density region of the second output probability vector, and the fourth target vector element is the largest vector element in the second output probability vector; obtaining the loss value of the loss function based on the first target output probability vector and the second target output probability vector; and adjusting the weight parameters of the student model based on the loss value to train the student model.

[0023] In another possible implementation, the training module is also used to: truncate the input of the neural network with the first output probability vector and the second output probability vector as probability vectors, and output low quantiles and high quantiles, where the low quantiles indicate the minimum probability value boundary of the semantic high-density region and the high quantiles indicate the maximum probability value boundary of the semantic high-density region; and determine the semantic high-density region based on the low quantiles and high quantiles.

[0024] In another possible implementation, the probabilistic vector truncated neural network includes a multilayer perceptron (MLP) neural network.

[0025] In another possible implementation, the loss function also includes a regularization term, which is used to align the correlation between the output probability vectors of the student model and the teacher model using prior phrases.

[0026] In another possible implementation, the value of the regularization term is determined based on the following steps: dividing the response text sequence output by the student model into word groups to obtain multiple word groups; dividing the output probability vector of the student model into multiple first output probability vector groups based on the multiple word groups; dividing the output probability vector of the teacher model into multiple second output probability vector groups based on the multiple word groups; calculating the similarity distance between each output probability vector in each of the multiple first output probability vector groups; calculating the similarity distance between each output probability vector in each of the multiple second output probability vector groups; and determining the value of the regularization term based on the similarity distances between each output probability vector in each of the first output probability vector groups and the similarity distances between each output probability vector in each of the second output probability vector groups.

[0027] In another possible implementation, the loss function also includes a second loss value, which is determined based on the response text sequence generated by the student model based on the training text sequence in the target training dataset and the target response text sequence corresponding to the training text sequence.

[0028] Thirdly, embodiments of this application provide a computing device, including a memory and a processor, wherein the memory stores instructions that, when executed by the processor, cause the method described in the first aspect to be implemented.

[0029] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the method described in the first aspect to be implemented.

[0030] Fifthly, embodiments of this application also provide a computer program or computer program product, the computer program or computer program product including instructions that, when executed, cause a computer to perform the method described in the first aspect.

[0031] In a sixth aspect, embodiments of this application also provide a chip including at least one processor and a communication interface, wherein the processor is used to execute the method described in the first aspect. Attached Figure Description

[0032] Figure 1 shows a schematic diagram of an artificial intelligence main framework;

[0033] Figure 2 is a system architecture diagram of a sample processing system provided in an embodiment of this application;

[0034] Figure 3 is a schematic diagram of the hardware structure of a chip provided in an embodiment of this application;

[0035] Figure 4 shows a schematic diagram of the main flow of the model distillation method provided in the embodiments of this application;

[0036] Figure 5 shows a schematic diagram of the implementation architecture of the model distillation method provided in the embodiments of this application;

[0037] Figure 6 illustrates a schematic diagram of the specific implementation of the sequence correction and regeneration strategy in an embodiment of this application;

[0038] Figure 7 shows a schematic diagram illustrating the implementation of DAC-KL distillation loss calculation in an embodiment of this application;

[0039] Figure 8 shows a schematic diagram of the implementation architecture of the Span prior-guided probability vector correlation regularization calculation in an embodiment of this application.

[0040] Figure 9 shows a schematic diagram of the implementation process of the model distillation method provided in the embodiments of this application;

[0041] Figure 10 is a schematic diagram of a model distillation apparatus provided in an embodiment of this application;

[0042] Figure 11 is a schematic diagram of the structure of the computing device provided in the embodiment of this application. Detailed Implementation

[0043] The term "and / or" used in this article describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0044] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same properties in the description of embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such processes, methods, systems, products, or apparatus.

[0045] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0046] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0047] First, the overall workflow of the artificial intelligence system will be described. Please refer to Figure 1. Figure 1 shows a schematic diagram of an artificial intelligence main framework, which describes the overall workflow of the artificial intelligence system and is applicable to general artificial intelligence field needs.

[0048] The above-mentioned artificial intelligence framework is elaborated below from two dimensions: the "Intelligent Information Chain" (horizontal axis) and the "IT Value Chain" (vertical axis). The "Intelligent Information Chain" reflects a series of processes from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom." The "IT Value Chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence, information (provided and processed by technical means), to the industrial ecosystem of the system. From the perspective of the IT value chain, the main framework of artificial intelligence includes an infrastructure layer, a data processing layer, a general capability layer (achieving general capabilities through data processing results, such as translation, text analysis, computer vision processing, speech recognition, and image recognition), and the topmost application layer, which encapsulates the overall artificial intelligence solution, productizing intelligent information decision-making and realizing its practical application.

[0049] (1) Infrastructure:

[0050] The infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. Communication with the outside world is achieved through sensors; computing power is provided by intelligent chips, including but not limited to hardware acceleration chips such as central processing units (CPUs), neural network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs). The basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0051] (2) Data

[0052] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0053] (3) Data processing

[0054] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0055] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.

[0056] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0057] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0058] (4) General ability

[0059] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0060] (5) Smart Products and Industry Applications

[0061] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They encapsulate overall artificial intelligence solutions, productize intelligent information decision-making, and realize practical applications. Their application areas mainly include: intelligent manufacturing, intelligent transportation, smart home, intelligent healthcare, intelligent security, autonomous driving, safe city, and intelligent terminals.

[0062] The language model training method and apparatus provided in this application are mainly applied in the field of natural language processing, such as intelligent voice scenarios like speech translation, semantic recognition, spoken language understanding, and voice dialogue systems.

[0063] The model distillation method and apparatus provided in this application can be applied to the deployment of large language models. By utilizing the model distillation scheme provided in this application, an LLM is distilled to obtain a smaller model with fewer parameters. The distilled smaller model (i.e., the student model after distillation in this application) inherits the advantages of the large teacher model (i.e., the LLM to be deployed), but has fewer parameters, reducing the computational and storage overhead on the deployment computing device, increasing the model deployment efficiency, and accelerating the model inference speed. For example, the distilled smaller model can be deployed on a terminal device to provide natural language processing (NLP) services. Terminal devices can be, for example, mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc., and this application does not impose any limitations on this.

[0064] For example, the small model obtained from distillation can also be deployed on a server, which can be a cloud server, network server, application server, or management server, or other device or server with data processing capabilities. The server receives query statements / voice / text questions from smart terminals through an interactive interface, then processes the language data by calling the small model deployed on it (for example, by processing data through the student model after distillation in this embodiment), and feeds the processing results back to the user device.

[0065] Understandably, "small model" refers to a natural language processing model with a smaller model size, fewer parameters, shorter training time, and lower computational resource requirements compared to LLM (Local Language Management). For example, a small model has tens of millions to billions of parameters, while an LLM has tens of billions to trillions of parameters.

[0066] Users can also access the small models deployed on the server in various ways to obtain natural language processing services. For example, users can access the server by logging into a webpage on a smart terminal, or by accessing the small models deployed on the server through a client on a smart terminal.

[0067] This application does not limit the application scenarios of the small model obtained by distillation. This small model can handle various natural language processing tasks.

[0068] For example, classification tasks involve small models outputting a classification value for an input text sequence. Examples include sentiment classification, topic classification, or classification based on whether grammar is correct.

[0069] Information extraction task: A small model extracts at least one type of information from the input text.

[0070] Sequence labeling: For each word in the text, a small model is required to classify it based on the context. Examples include Chinese word segmentation, part-of-speech tagging, named entity recognition, and semantic role labeling.

[0071] Sentence relation inference: Small models take two texts as input and infer whether these two texts have a certain nominal relationship. Examples include question answering systems, semantic rewriting, or natural language inference.

[0072] Generative tasks: Input a piece of text and generate another piece of text using a small model. Examples include machine translation, automatic summarization, or poetry and sentence creation.

[0073] It should be noted that the small model obtained by distillation in the embodiments of this application can also be applied to other natural language processing tasks. The above examples are only for the convenience of understanding this solution and are not intended to limit this solution.

[0074] To facilitate understanding of this solution, the sample processing system provided in this application embodiment is first introduced with reference to Figure 2. Referring to Figure 2, the data acquisition device 260 is used to collect training data. The training data in this application embodiment is the data used when distilling the student model.

[0075] After collecting the training data, the data acquisition device 260 stores the training data in the database 230. The training device 220 performs knowledge distillation on the student model based on the teacher model based on the training data maintained in the database 230 to obtain the target model / rule 201.

[0076] In practical applications, the training data maintained in database 230 may not all come from the data acquisition device 260; it may also be received from other devices. Furthermore, it should be noted that training device 220 may not necessarily train the target model / rule 201 entirely based on the training data maintained in database 230; it may also obtain training data from the cloud or other sources for model training. The above description should not be construed as limiting the embodiments of this application.

[0077] The target model / rule 201 trained by the training device 220 can be applied to different systems or devices, such as the execution device 210 in Figure 2. The execution device 210 can be a terminal, such as a mobile phone, tablet computer, laptop computer, augmented reality (AR), virtual reality (VR), wearable device, smart robot, vehicle terminal, etc., or it can be a server or cloud.

[0078] Specifically, training device 220 can deploy the knowledge-distilled student model to execution device 210.

[0079] In Figure 2, the execution device 210 configures the input / output (I / O) interface 212 for data exchange with external devices. The user can input data into the I / O interface 212 through the client device 240. In this embodiment, this data may include text input by the user. It should be noted that this text can be input in any way; for example, the user can input voice data, which the voice recognition model will convert into text data, or the user can directly input text data.

[0080] Preprocessing modules 213 and 214 are used to preprocess the input data (such as text entered by the user) received from I / O interface 212. It should be understood that preprocessing modules 213 and 214 may be omitted, or only one preprocessing module may be used. When preprocessing modules 213 and 214 are absent, the calculation module 211 can directly process the input data.

[0081] The calculation module 211 is used to perform calculations and other related processing on the data input from the preprocessing module 213 and the preprocessing module 214 according to the target model / rule 201 mentioned above.

[0082] During the preprocessing of input data by the execution device 210, or during the calculation and related processing of the calculation module 211 of the execution device 210, the execution device 210 can call the data, code, etc. of the database storage system 250 for corresponding processing, or store the data, instructions, etc. obtained from the corresponding processing into the data storage system 250.

[0083] Finally, I / O interface 212 returns the processing results (such as automatic summarization, translation results, and poetry / sentence construction) to client device 240 for the user. It should be understood that the target model / rule 201 differs for different natural language processing tasks, and the processing results will also differ accordingly.

[0084] It is worth noting that the training device 220 can generate a target model / rule 201 corresponding to different downstream systems. This target model / rule 201 can achieve the above-mentioned goals or complete the above-mentioned tasks, thereby providing the user with the required results. It should also be noted that the training device 220 can generate corresponding preprocessing models for the target model / rule 201 corresponding to different downstream systems, such as the preprocessing models corresponding to preprocessing modules 213 and / or 214.

[0085] In the scenario shown in Figure 2, the user can manually specify data to be input into the execution device 210 (e.g., input prompt text), for example, by operating within the interface provided by the I / O interface 212. Alternatively, the client device 240 can automatically input data into the I / O interface 212 (e.g., input prompt text) and obtain results. If the client device 240 requires user authorization to automatically input data, the user can set appropriate permissions within the client device 240. The user can view the results output by the execution device 210 on the client device 240 (e.g., the output results could be translations, answers to questions, etc.), presented in various forms such as display, sound, or animation. The client device 240 can also act as a data acquisition terminal, collecting the input data (prompt text) from the input I / O interface 212 as shown in Figure 2 and the output results of the target model / rule 201, storing them as new sample data in the database 230.

[0086] It is worth noting that Figure 2 is merely a schematic diagram of a system architecture provided by an embodiment of this application, and the positional relationships between the devices, components, modules, etc. shown in the figure do not constitute any limitation. For example, in Figure 2, the data storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 may also be placed within the execution device 210.

[0087] Figure 3 is a schematic diagram of the hardware structure of a chip provided in an embodiment of this application. The chip includes a Network Processing Unit (NPU) 30. The chip can be installed in the execution device 210 shown in Figure 2 to perform the calculation work of the computing module 211. The chip can also be installed in the training device 220 shown in Figure 2 to perform the training work of the training device 220 and output the target model / rule 201.

[0088] The model distillation method of this application embodiment can be executed in the arithmetic circuit 303 and / or vector calculation unit 307 in the neural network processor 30, thereby obtaining a high-performance student model.

[0089] The following is a brief introduction to the various modules and units in the neural network processor 30.

[0090] The neural network processor 30 is mounted as a coprocessor on the host CPU, and tasks are assigned by the host CPU. The core of the neural network processor 30 is the arithmetic circuit 303. When the neural network processor 30 is working, the controller 304 in the neural network processor 30 can control the arithmetic circuit 303 to retrieve matrix data from the memory and perform multiplication operations.

[0091] In some implementations, the arithmetic circuit 303 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 303 is a two-dimensional pulsating array. The arithmetic circuit 303 can also be a one-dimensional pulsating array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 303 is a general-purpose matrix processor.

[0092] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 302 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 301 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 308.

[0093] The vector computation unit 307 can further process the output of the arithmetic circuit, such as vector multiplication, vector addition, exponentiation, logarithmic operations, size comparisons, etc. For example, the vector computation unit 307 can be used for network computation in non-convolutional / non-fully connected (FC) layers of a neural network, such as pooling, batch normalization, and local response normalization.

[0094] In some implementations, vector computation unit 307 can store the processed output vector into unified buffer 306. For example, vector computation unit 307 can apply a nonlinear function to the output of arithmetic circuit 303, such as a vector of accumulated values, to generate activation values. In some implementations, vector computation unit 307 generates normalized values, merged values, or both. In some implementations, the processed output vector can be used as activation input to arithmetic circuit 303, for example, for use in subsequent layers of a neural network.

[0095] The unified memory 306 is used to store input data and output data.

[0096] The weight data is directly transferred from the external memory to the input memory 301 and / or the unified memory 306 through the Direct Memory Access Controller (DMAC) 305, the weight data in the external memory is stored in the weight memory 302, and the data in the unified memory 306 is stored in the external memory.

[0097] BIU stands for Bus Interface Unit 310, which is used to enable interaction between the main CPU, DMAC and instruction fetch memory 309 via a bus.

[0098] The instruction fetch buffer 309, which is connected to the controller 304, is used to store the instructions used by the controller 304.

[0099] The controller 304 is used to call the instructions cached in the instruction memory 309 to control the operation of the computing accelerator.

[0100] Generally, the unified memory 306, input memory 301, weighted memory 302, and instruction fetch memory 309 can all be on-chip memories. The external memory of the NPU can be memory located outside the NPU itself, such as double data rate synchronous dynamic random access memory (DDR SDRAM), high bandwidth memory (HBM), or other readable and writable memory. It should be understood that the chip hardware structure shown in Figure 3 is merely illustrative and is not limited thereto.

[0101] To facilitate understanding of the solutions in the embodiments of this application, the technical terms involved in this document will be explained first.

[0102] Large language models refer to models trained for natural language processing tasks based on machine learning techniques such as neural networks. They can learn grammatical, semantic, and pragmatic information from natural language and generate, understand, and translate natural language text. Large language models typically require large corpora and computational resources for training. These models have wide applications, including text classification, machine translation, automatic question answering, summarization, and dialogue systems. In this paper, the concept of large language models primarily refers to models with over 10 billion parameters that are difficult to deploy on a single GPU, such as GPT-3, GPT-4, and the Pangu NLP large model.

[0103] Knowledge distillation utilizes the knowledge learned by a large model to guide the training of a smaller model, enabling the smaller model to achieve performance comparable to the large model while significantly reducing the number of parameters. This results in model compression and acceleration, which is the application of knowledge distillation and transfer learning in model optimization. For example, in knowledge distillation, a well-trained large model (such as an LLM with a complex network structure) can be used to train a smaller model (another simpler network), allowing the simpler network to have the same or similar data processing capabilities as the complex network.

[0104] It should be noted that the aforementioned identical or similar data processing capabilities can be understood as the student model and teacher model obtaining the same or similar processing results after knowledge distillation when processing the same data.

[0105] In training a neural network, the loss function is crucial because we want the network's output to be as close as possible to the desired predicted value. This is achieved by comparing the network's current prediction with the target value and updating the weight vector of each layer based on the difference. (Of course, there's usually an initialization process before the first update, where parameters are pre-configured for each layer). For example, if the network's prediction is too high, the weight vector is adjusted to predict a lower value. This adjustment continues until the neural network can predict the target value or a value very close to it. Therefore, it's necessary to predefine "how to compare the difference between the predicted and target values," which is the loss function or objective function. These are important equations used to measure the difference between the predicted and target values. Taking the loss function as an example, a higher output value (loss) indicates a greater difference, so training the neural network becomes a process of minimizing this loss.

[0106] When performing knowledge distillation, it is necessary to construct the loss based on the output of the teacher model and the output of the student model. The model output used to construct the loss can be the output of the output layer of the model, the intermediate feature map output of the intermediate network layer, or the result obtained by processing the output of the output layer and / or the intermediate feature map output of the intermediate network layer.

[0107] In the field of large-scale language models, the rapid advancements in autoregressive models have greatly propelled the development of text generation tasks. However, these large models face challenges in deployment and real-time applications. In contrast, smaller models perform poorly in practical generation tasks. To address this issue, knowledge distillation has been proposed, allowing the development of smaller, more efficient student models that inherit the advantages of large teacher models. Existing knowledge distillation work largely focuses on closed-source, "black-box" model distillation, while "white-box" distillation methods, which can be used with open-source models, may introduce errors due to over-reliance on the output generated by student models. The limited capacity and biases of student models may further exacerbate these errors, leading to distorted representations of teacher knowledge.

[0108] One related approach involves generating and filtering multi-step reasoning explanations from a large teacher model, and then fine-tuning a small student model to improve its ability to solve complex problems. This is a "black-box" distillation method, suitable for non-open-source teacher models such as GPT4. However, it performs poorly in distillation scenarios where open-source teacher models are available, failing to fully explore the guiding value of teacher models.

[0109] In another related approach, kullback leibler divergence (KLD) is used as the distillation target to prevent students from overestimating low-probability regions in the teacher distribution. Simultaneously, student models are not forced to fit all responses sampled from the teacher distribution; instead, students are encouraged to generate teacher-preferred samples within their own capabilities.

[0110] However, this approach relies on student-generated output, which can introduce errors due to poor student performance, leading to poor distillation learning results. Existing KL-Divergence methods (such as FKL and RKL) as the objective function of KD struggle to capture the valuable distribution of the teacher network or overfit to high-probability distribution regions.

[0111] In another related approach, a novel skewed Kullback-Leibler (skewKL) divergence loss is proposed, optimized for stable gradients and minimum approximation error, achieving superior performance. Simultaneously, an adaptive off-policy method is proposed to improve the training efficiency of distillation using student-generated data.

[0112] However, this approach relies on the output generated by the student model, which can introduce errors due to the poor performance of the student model itself, resulting in poor distillation learning performance. SKL introduces additional hyperparameters to adjust KL, but it lacks adaptability to the rich semantic knowledge in the LLM output results under different token variations.

[0113] Therefore, the relevant distillation schemes have the following problems:

[0114] (1) Relying on the output generated by the student model as distillation data to solve the problem of training and inference mismatch during the training of the student model, but due to the limited capabilities of the student model itself, it will introduce its own generation error, resulting in poor distillation effect.

[0115] (2) LLM predicts an ultra-high-dimensional probability vector for each token in the input text sequence. The existing KL-Divergence as the objective function of KD is difficult to capture the valuable distribution of the teacher network or overfit to the high probability distribution region.

[0116] In view of this, this application provides a model distillation method that employs a multi-level semantic revision approach, significantly enhancing the knowledge distillation process of large language models. At the sequence level, this method uses a sequence correction and re-generation (SCRG) strategy to guide the student model to generate more reliable and diverse samples, while seamlessly integrating with both on-policy and off-policy generation methods. At the token level, a distributed adaptive pruning Kullback-Leibler (DAC-KL) loss function is proposed. This function focuses on semantically salient regions in the output space through a learnable subnetwork, effectively filtering redundant information and retaining the most critical signals. Furthermore, at the span level, we use input span priors to align the logits vector relationship between the student and teacher models within the same span.

[0117] The following detailed description, with reference to the accompanying drawings, illustrates the specific implementation of the model distillation method and apparatus provided in this application.

[0118] Figure 4 illustrates the main flow of the model distillation method provided in this embodiment. As shown in Figure 4, the model distillation method provided in this embodiment first processes the fixed dataset (short for fixed training dataset) through sequence correction and regeneration strategies (e.g., sample) to generate a dataset constructed based on the student model output (which can be called the target training dataset, or simply the target dataset). Then, the student model is distilled and trained using the teacher model and the target dataset.

[0119] The sequence correction and regeneration strategy guides the student model to generate more reliable and diverse samples at the sequence level. Then, the training dataset is reconstructed using the semantically correct response sequence y generated by the student model. This overcomes the problem that the student model itself may introduce its own generation errors due to its limited capabilities, resulting in poor distillation performance.

[0120] In another example, to further improve the training performance of the student model and make its prediction accuracy closer to that of the teacher model, a novel DAC-KL distillation loss function is proposed during the student model training process. This function uses a learnable subnetwork to focus on semantically salient regions in the output space, effectively filtering redundant information and retaining the most critical signals. Furthermore, a Span-prior-guided probabilistic vector correlation regularization calculation is provided. At the Span level, we use the input span prior to align the logits vector relationship between the student and teacher models within the same span.

[0121] Figure 5 illustrates the implementation architecture of the model distillation method provided in this application. As shown in Figure 5, the prompt text X in the fixed training dataset is first used as the student model (student q). θ The input is a sequence of response texts. Then, the prompt text "prompt X" and the response text sequence output by the student model are used. As input to the teacher model (teacher p), the output is a sequence of response text. Based on the output probability vector of the student model and the output probability vector of the teacher model The sequence output by the student model is calculated. The sequence output by the teacher model The semantically most different lexical units, for example, can be based on the output probability vector of the student model. and the output probability vector of the teacher model Calculate token-wise KL divergence, then use argmax to identify and find... The biggest semantic difference This process is repeated iteratively until the sequence output by the student model is found. The sequence output by the teacher model The semantically different lexical units are then re-inputted into the student model, so that, under the guidance of the teacher model, the student model outputs a response sequence whose semantics are close to or the same as the output of the teacher model.

[0122] Then, using the y output from the student model... s Replace y in the fixed dataset to construct a new dataset (which can be called the target training dataset, or simply the target dataset) (x, y). sThen, using the target dataset (x, y) s The student model is trained, and the loss value of the DAC-KL distillation loss function is calculated using the output probability vector of the student model and the output probability vector of the teacher model. The weight parameters of the student model are adjusted based on at least this loss value.

[0123] The following sections, with reference to Figures 6-8, detail the specific implementations of the sequence correction and regeneration strategies, DAC-KL distillation loss calculation, and Span prior-guided probability vector correlation regularization calculation provided in the embodiments of this application.

[0124] Figure 6 illustrates a schematic diagram of the specific implementation of the sequence correction and regeneration strategy in this application embodiment. As shown in Figure 6, the fixed dataset includes multiple training sample pairs (x, y). Each training sample (x, y) includes a prompt text sequence x and its corresponding response text sequence y. During model training, the prompt text X from the fixed dataset is first used as the input to the student model, and the output is the response text sequence. Then, the prompt text "prompt X" and the response text sequence output by the student model are used. As input to the teacher model, the output is a sequence of response texts. Based on the output probability vector of the student model and the output probability vector of the teacher model The sequence output by the student model is calculated. The sequence output by the teacher model The semantically most different lexical units, for example, can be based on the output probability vector of the student model. and the output probability vector of the teacher model Calculate token-wise KL divergence, then use argmax to identify and find... The semantically most different word units This process is repeated iteratively until the sequence output by the student model is found. The sequence output by the teacher model The semantically different lexical units are then re-inputted into the student model, so that, under the guidance of the teacher model, the student model outputs a response sequence whose semantics are close to or the same as the output of the teacher model.

[0125] Then, using the y output from the student model... s By replacing y in the fixed dataset, a sampled dataset is constructed, which is the target dataset (x, y) processed using sequence correction and regeneration strategies. sBy using the teacher model to guide the student model to generate more reliable and diverse samples, the problem of poor distillation results caused by the student model's own generation errors due to its limited capabilities is avoided.

[0126] Figure 7 illustrates a schematic diagram of the implementation of DAC-KL distillation loss calculation according to an embodiment of this application. As shown in Figure 7, the target dataset (x, y) is... s In the input model, prompt x is input to both the student model and the teacher model. The student model outputs a probability vector based on the input prompt x. The teacher model derives its output probability vector from the input prompt x. For any student model's output probability vector and its corresponding teacher model's output probability vector (the corresponding vectors refer to the output probability vectors at the same position in the sequence, for example...), ... correspond, Correspondingly, and so on, The corresponding operation is performed to output a probability vector. and output probability vector For example, the output probability vector of the student model Including vector elements The output probability vector of the teacher model Including vector elements The output probability vector of the student model and the output probability vector of the teacher model The input is fed into a probability vector truncated neural network (such as an MLP) to obtain the output probability vector. and output probability vector The low quantile l and high quantile u are identified, and then semantic high-density regions (also known as high-density category regions) are determined using the low quantile l and high quantile u. Finally, the low quantile l and high quantile u are used to determine the output probability vector of the student model. and the output probability vector of the teacher model The vector elements that fall into the semantic high-density region are clipped and then combined with the largest vector element in the probability vector to obtain the target output probability vector. The loss value of DAC-KL distillation loss is then calculated using the target output probability vector of the student model and the target output probability vector of the teacher model. The parameters of the student model and the MLP are adjusted according to the loss value.

[0127] This allows the student model to focus on semantically significant regions in the output space, effectively filtering redundant information and retaining the most critical signals. It avoids the problem in related techniques where using KL-Divergence as the objective function of KD makes it difficult to capture the valuable distribution of the teacher model or overfits to high-probability distribution regions.

[0128] Figure 8 illustrates the implementation architecture of the probability vector correlation regularization calculation guided by prior knowledge of the Span in this application. As shown in Figure 8, the response text sequence output by the student model is divided into multiple phrases. For example, if the response text sequence output by the student model is "a series of art exhibitions", it can be divided into multiple spans with complete meanings through prior knowledge. For example, "a series of" in the response text sequence is divided into one span, and "art exhibitions" is divided into another span. It can be understood that a noun phrase can be a span, a verb phrase can be a span, and a prepositional phrase can be a span, etc.

[0129] The output probability vector of the student model is divided into multiple first output probability vector groups based on multiple word phrases; the output probability vector of the teacher model is also divided into multiple second output probability vector groups based on multiple word phrases. For example, the output probability vector group corresponding to the span "a series of" in the student model is... The corresponding output probability vector set of the teacher model is The other span, "art exhibitions," corresponds to the output probability vector set of the student model. The corresponding output probability vector set of the teacher model is

[0130] Calculate the similarity distance between each output probability vector in each of the multiple first output probability vector groups; calculate the similarity distance between each output probability vector in each of the multiple second output probability vector groups; that is, calculate the similarity distance between each probability vector within a group. For example, the output probability vector of the student model is divided into two probability vector groups, including the probability vector group corresponding to "a series of". The probability vector group corresponding to "art exhibitions" The similarity distances of the probability vectors within the probability vector group corresponding to "a series of" include... The similarity distance a1 between them The similarity distance a2 between them; the similarity distance of each probability vector within the probability vector group corresponding to "art exhibitions" includes and The similarity distance between them is a3. Similarly, the output probability vector of the teacher model is divided into two probability vector groups, including the probability vector group corresponding to "a series of". The probability vector group corresponding to "art exhibitions" The similarity distances of the probability vectors within the probability vector group corresponding to "a series of" include... The similarity distance between them is A1. The similarity distance A2 between them; the similarity distances of each probability vector within the probability vector group corresponding to "art exhibitions" include The similarity distance between them is A3.

[0131] The value of the regularization term is determined based on the similarity distances between the output probability vectors in each of the first output probability vector groups and the output probability vectors in each of the second output probability vector groups. Specifically, using similarity distances a1 and a2, and a3 and a3, the loss is calculated to obtain loss1, loss2, and loss3. Based on loss1, loss2, and loss3, the L2 loss, i.e., the value of the regularization term, is obtained.

[0132] The weight parameters of the student model can be adjusted by at least using the value of the regularization term and the loss value of the DAC-KL distillation loss, so as to further improve the training effect of the student model.

[0133] In another example, the loss function also includes cross-entropy loss, the cross-entropy loss value being calculated by the student model generating the response text sequence y based on the training text sequence prompt x from the target training dataset. s In the target training dataset (x, y) s The value of y is calculated from it.

[0134] In other words, the loss function for distillation learning of the student model in this application includes three parts. The first loss function is the DAC-KL loss function, and the second loss function can be based on the target response text sequence y in the target training dataset and the response text sequence y output by the student model. s The calculation involves the cross-entropy loss function, the third of which is a regularization term (e.g., L2 loss). These three loss values ​​are then weighted and fused to obtain the target loss value of the loss function. The target loss value is used to adjust the weight parameters of the student model, making the prediction accuracy of the student model close to that of the teacher model, which greatly improves the distillation training effect of the student model.

[0135] Figure 9 illustrates a schematic flowchart of the model distillation method provided in this embodiment. This method can be executed on the training device shown in Figure 2. As shown in Figure 9, the model distillation method provided in this embodiment includes at least steps S901 to S904.

[0136] In step S901, a fixed training dataset is obtained.

[0137] The fixed training dataset can be any training dataset, such as the Dolly Evaluation dataset, the Self-Instruct dataset, the Vicuna dataset, the Super-Natural dataset, and the Unnatural dataset. The fixed dataset includes multiple training sample pairs, each of which includes a prompt text sequence prompt x and its corresponding response sequence y, which can be represented as (x, y).

[0138] In step S902, the teacher model guides the student model to generate the target response text sequence based on the training text sequence.

[0139] In step S903, a target training dataset is obtained based on multiple training text sequences and the target response text sequences corresponding to each training text sequence.

[0140] In the two steps above, a sequence correction and regeneration strategy is adopted to guide the student model to generate more reliable and diverse samples at the sequence level. Then, the target training dataset is reconstructed using the semantically correct response sequence y generated by the student model. This overcomes the problem that the student model itself may introduce its own generation errors due to its limited capabilities, resulting in poor distillation effect.

[0141] Compared to traditional knowledge distillation, which uses a fixed dataset and performs on-policy and off-policy distillation using data generated by student networks, the student-generated data may introduce generation errors due to its limited performance. In this application, the embodiment utilizes sequence correction and regeneration strategies to reduce student generation errors and improve the diversity of generated data, and can be adapted to existing on-policy and off-policy generation methods.

[0142] For a detailed description of the sequence correction and regeneration strategies, please refer to Figure 6 above. For the sake of brevity, it will not be repeated here.

[0143] In step S904, the student model is trained by distillation based on the target training dataset and the teacher model.

[0144] This application's embodiments overcome the problems of traditional distillation using FKL, which underfits the teacher model's output distribution; MiniLLM and GKD using RKL, which overfits high-probability distribution regions; and DistiLLM's proposed SKL, which introduces additional hyperparameters to coordinate RKL and SKL but struggles to adapt to the huge variations in the logits distribution of different tokens in LLM outputs. At the same time, it enables the student model to focus on semantically significant regions in the output space, effectively filtering redundant information, retaining the most critical signals, and improving the quality of distillation training.

[0145] By using Span prior-guided probability vector correlation regularization, compared to existing logit correlation distillation which calculates correlation between logit vectors of the same category in the input data for batch-level or CV classification tasks, this scheme combines the NLP task-specific span prior to model semantic correlation within the span. At the span level, this application's embodiment introduces predefined block priors to align the correlation of output probability vectors of student and teacher models, ensuring consistent transmission of semantic information of tokens within the span.

[0146] The specific implementation of the DAC-KL distillation loss calculation and the probability vector correlation regularization calculation guided by the Span prior is described in detail in Figures 7 and 8 above. For the sake of brevity, it will not be repeated here.

[0147] To verify the actual effect of the model distillation method provided in this application, the embodiments of this application are compared with the following benchmark algorithms on a given training and test set, and show significant progress.

[0148] Table 1 shows an objective comparison of the performance indicators of the model distillation method provided in this application embodiment with the state-of-the-art existing distillation methods.

[0149] Table 1

[0150] As shown in Table 1, distillation experiments were conducted on four series of large language model SFT tasks: LLAMA2, OPT, OpenLLAMA2, and GPT2. The results show that the model distillation method provided in this application improves the performance of the Vanilla student model by more than 15% on most benchmarks and exceeds the state-of-the-art distillation method by more than 3%.

[0151] The model distillation method provided in this application ablates the sequence correction and regeneration strategies, DAC-KL distillation loss calculation, and Span prior-guided probability vector correlation regularization calculation, demonstrating that innovations at each level contribute to the overall improvement in model performance, with the comprehensive method producing the greatest performance improvement. SCRG is compared with different student network data generation methods, sampling different distillation datasets. Compared with existing student model generation methods, we observe that SCRG can significantly improve performance. When both off-policy and on-policy sampling methods are used simultaneously, SCRG achieves significantly higher results on all evaluation metrics.

[0152] To verify the effectiveness of the proposed DAC-KL loss, we compared it with different loss functions. The results show that DAC-KL significantly outperforms other loss functions on all evaluation metrics.

[0153] For a detailed comparison of the objective indicators of the specific ablation experiments, please refer to Tables 2 to 5 below.

[0154] Table 2

[0155] Table 3

[0156] Table 4

[0157] Table 5

[0158] Based on the same concept as the aforementioned embodiment of the model distillation method, this application also provides a model distillation apparatus 1000. This apparatus can perform distillation learning on large language models to obtain high-performance, miniaturized student models, which is beneficial for model deployment. The model distillation apparatus 1000 includes units or modules for implementing the various steps in the model distillation method shown in Figures 4-9.

[0159] Figure 10 is a schematic diagram of a model distillation apparatus provided in an embodiment of this application. As shown in Figure 10, the model distillation apparatus 1000 includes an acquisition module 1001, a semantic correction module 1002, a construction module 1003, and a training module 1004. The acquisition module 1001 is used to acquire a fixed training dataset, which includes multiple training text sequences. The semantic correction module 1002 is used to guide a student model to generate a target response text sequence based on the training text sequences through a teacher model, so as to correct the semantic differences between the response text sequence output by the student model and the response text sequence output by the teacher model. The construction module 1003 is used to obtain a target training dataset based on multiple training text sequences and the generated text sequences corresponding to each training text sequence. The training module 1004 is used to perform distillation training on the student model based on the target training dataset and the teacher model.

[0160] In one possible implementation, the training text sequence includes a cue text sequence, and the teacher model is a trained large language model; the semantic correction module 1002 is specifically used to: take the cue text sequence as input to the student model and output a first response text sequence; take the cue text sequence and the first response text sequence as input to the teacher model and output a second response text sequence, wherein the lengths of the first response text sequence and the second response text sequence are equal; label the semantically different words in the first response text sequence and the second response text sequence; take the labeled first response text sequence and the cue text sequence as input to the student model and output a target response text sequence.

[0161] In another possible implementation, the training module 1004 is specifically used for: taking the training text sequence as input to the student model to obtain a first output probability vector for each word; taking the prompt text sequence as input to the teacher model to obtain a second output probability vector for each word; identifying the first target vector element and the second target vector element in the first output probability vector to obtain a first target output probability vector, where the first target vector element is the vector element distributed in the semantically high-density region of the first output probability vector, and the second target vector element is the largest vector element in the first output probability vector, and the semantically high-density region is the region where the vector element distribution probability is greater than a preset threshold; identifying the third target vector element and the fourth target vector element in the second output probability vector to obtain a second target output probability vector, where the third target vector element is the vector element distributed in the semantically high-density region of the second output probability vector, and the fourth target vector element is the largest vector element in the second output probability vector; obtaining the loss value of the loss function based on the first target output probability vector and the second target output probability vector; and adjusting the weight parameters of the student model based on the loss value to train the student model.

[0162] In another possible implementation, the training module 1004 is further configured to: use the first output probability vector and the second output probability vector as inputs to the neural network to truncate the probability vectors, output low quantiles and high quantiles, where the low quantiles indicate the minimum probability value boundary of the semantic high-density region and the high quantiles indicate the maximum probability value boundary of the semantic high-density region; and determine the semantic high-density region based on the low quantiles and high quantiles.

[0163] In another possible implementation, the probabilistic vector truncated neural network includes a multilayer perceptron (MLP) neural network.

[0164] In another possible implementation, the loss function also includes a regularization term, which is used to align the correlation between the output probability vectors of the student model and the teacher model using prior phrases.

[0165] In another possible implementation, the value of the regularization term is determined based on the following steps: dividing the response text sequence output by the student model into word groups to obtain multiple word groups; dividing the output probability vector of the student model into multiple first output probability vector groups based on the multiple word groups; dividing the output probability vector of the teacher model into multiple second output probability vector groups based on the multiple word groups; calculating the similarity distance between each output probability vector in each of the multiple first output probability vector groups; calculating the similarity distance between each output probability vector in each of the multiple second output probability vector groups; and determining the value of the regularization term based on the similarity distances between each output probability vector in each of the first output probability vector groups and the similarity distances between each output probability vector in each of the second output probability vector groups.

[0166] In another possible implementation, the loss function also includes a second loss value, which is determined based on the response text sequence generated by the student model based on the training text sequence in the target training dataset and the target response text sequence corresponding to the training text sequence.

[0167] The model distillation apparatus 1000 according to the embodiments of this application can correspond to the execution of the methods described in the embodiments of this application, and the above and other operations and / or functions of each module in the model distillation apparatus 1000 are respectively for implementing the corresponding processes of each method in FIG4-9. For the sake of brevity, they will not be described again here.

[0168] This application also provides a computing device including at least one processor, a memory, and a communication interface, wherein the processor is used to execute the method described in Figures 4-9.

[0169] Figure 11 is a schematic diagram of the structure of the computing device provided in the embodiment of this application.

[0170] As shown in Figure 11, the computing device 1100 includes at least one processor 1101, a memory 1102, and a communication interface 1103. The processor 1101, memory 1102, and communication interface 1103 are communicatively connected, which can be achieved via a wired (e.g., bus) or wireless connection. The communication interface 1103 is used to send and / or receive data from other devices. The memory 1102 stores computer instructions, which the processor 1101 executes to perform the method described in the preceding method embodiments, thereby enabling the distillation of a high-performance student model using an LLM as the teacher model, facilitating deployment.

[0171] It should be understood that, in the embodiments of this application, the processor 1101 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0172] The memory 1102 may include read-only memory and random access memory, and provides instructions and data to the processor 1101. The memory 1102 may also include non-volatile random access memory. Optionally, the random access memory may be, for example, high bandwidth memory (HBM).

[0173] The memory 1102 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0174] It should be understood that the computing device 1100 according to the embodiments of this application can execute the method shown in Figures 4-9 of the embodiments of this application. For a detailed description of the implementation of the method, please refer to the above text. For the sake of brevity, it will not be repeated here.

[0175] Embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein when the computer instructions are executed by a processor, the aforementioned method is implemented.

[0176] An embodiment of this application provides a chip including at least one processor and an interface, wherein the at least one processor determines program instructions or data through the interface; the at least one processor is used to execute the program instructions to implement the method mentioned above.

[0177] Embodiments of this application provide a computer program or computer program product that includes instructions that, when executed, cause a computer to perform the methods mentioned above.

[0178] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0179] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented using hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0180] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A model distillation method, characterized by, The method comprises: obtaining a fixed training data set, wherein the fixed training data set comprises a plurality of training text sequences; guiding a student model to generate a target response text sequence based on the training text sequence through a teacher model, so as to correct semantic differences in the response text sequence output by the student model and the response text sequence output by the teacher model; obtaining a target training data set based on the plurality of training text sequences and the target response text sequence corresponding to each training text sequence in the plurality of training text sequences; performing distillation training on the student model based on the target training data set and the teacher model.

2. The method of claim 1, wherein, The training text sequence comprises a prompt text sequence, and the teacher model is a large language model trained to completion; The method comprises: inputting the prompt text sequence into the student model to output a first response text sequence; inputting the prompt text sequence and the first response text sequence into the teacher model to output a second response text sequence, wherein the first response text sequence and the second response text sequence are equal in length; labeling word elements in the first response text sequence that are different in semantics from the second response text sequence; inputting the labeled first response text sequence and the prompt text sequence into the student model to output the target response text sequence.

3. The method according to claim 1 or 2, characterized in that, The method comprises: inputting the training text sequence into the student model to obtain a first output probability vector of each word element; inputting the prompt text sequence into the teacher model to obtain a second output probability vector of each word element; identifying a first target vector element and a second target vector element in the first output probability vector to obtain a first target output probability vector, wherein the first target vector element is a vector element distributed in a semantic high-density area in the first output probability vector, and the second target vector element is the largest vector element in the first output probability vector, and the semantic high-density area is an area where the probability of the vector element is greater than a preset threshold; identifying a third target vector element and a fourth target vector element in the second output probability vector to obtain a second target output probability vector, wherein the third target vector element is a vector element distributed in the semantic high-density area in the second output probability vector, and the fourth target vector element is the largest vector element in the second output probability vector; obtaining a first loss value of a loss function based on the first target output probability vector and the second target output probability vector; adjusting the weight parameters of the student model based on at least the first loss value to train the student model.

4. The method of claim 3, wherein, The method further comprises: output a low quantile and a high quantile, the low quantile indicating a minimum probability value boundary of the semantic high-density region, and the high quantile indicating a maximum probability value boundary of the semantic high-density region; determine the semantic high-density region based on the low quantile and the high quantile.

5. The method of claim 4, wherein, The probability vector truncation neural network comprises a multi-layer perception neural network.

6. The method according to any one of claims 3-5, characterized in that, The loss function further comprises a regularization term for utilizing a prior phrase alignment to correlate the output probability vectors of the student model and the teacher model.

7. The method of claim 6, wherein, The value of the regularization term is determined based on the following steps: perform phrase segmentation on the response text sequence output by the student model to obtain a plurality of phrases; segment the output probability vector of the student model based on the plurality of phrases to obtain a plurality of first output probability vector groups; segment the output probability vector of the teacher model based on the plurality of phrases to obtain a plurality of second output probability vector groups; calculate the similarity distance of each output probability vector in each first output probability vector group in the plurality of first output probability vector groups; calculate the similarity distance of each output probability vector in each second output probability vector group in the plurality of second output probability vector groups; determine the value of the regularization term based on the similarity distance of each output probability vector in each first output probability vector group and the similarity distance of each output probability vector in each second output probability vector group.

8. The method according to any one of claims 1 to 7, characterized in that, The loss function further comprises a second loss value determined based on the response text sequence generated by the student model based on the training text sequence in the target training data set and the target response text sequence corresponding to the training text sequence.

9. A model distillation apparatus, characterized by, comprise: an acquisition module configured to acquire a fixed training data set, the fixed training data set comprising a plurality of training text sequences; a semantic correction module configured to guide a student model to generate a target response text sequence based on the training text sequence by a teacher model to correct semantic differences in a response text sequence output by the student model and a response text sequence output by the teacher model; a construction module configured to obtain a target training data set based on the plurality of training text sequences and the target response text sequence corresponding to each training text sequence in the plurality of training text sequences; a training module configured to perform distillation training on the student model based on the target training data set and the teacher model.

10. A computing device comprising a memory and a processor, wherein, The memory stores instructions that, when executed by the processor, cause the method of any one of claims 1-8 to be implemented. The memory stores instructions that, when executed by the processor, cause the method of any one of claims 1-8 to be implemented.

Images