Model training method, training data acquisition method, and related device

Abstract

A model training method, a training data acquisition method, and a related device. The model training method can be used in the field of text processing within the field of artificial intelligence, and comprises: inputting a first text comprising a third text and a first prompt into a machine learning model to obtain a second text outputted by the machine learning model, the third text belonging to a first data set; inputting the second text into a teacher model to obtain a first prediction result outputted by the teacher model; on the basis of the first prediction result, executing a first training operation to update the first prompt, wherein training data used when the teacher model is utilized to perform knowledge distillation on a student model is obtained on the basis of the trained first prompt. In the described solution, the capability of the teacher model is utilized to guide the machine learning model-generated text to approximate private data, such that the training data used when knowledge distillation is performed on the student model is similar to the private data, thus improving the accuracy of the trained student model in executing natural language processing tasks.

Description

A model training method, a method for obtaining training data, and related equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on June 13, 2023, with application number 202310700033.6 and invention name “A model training method, training data acquisition method and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence, and in particular to a model training method, a training data acquisition method, and related equipment. Background Art

[0003] Artificial Intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that seeks to understand the essence of intelligence and develop new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0004] With the development of artificial intelligence technology, the number of machine learning models used to process text is increasing. It is necessary to compress and accelerate machine learning models. Knowledge distillation is an important means of model compression and acceleration.

[0005] However, in some fields, the training data used to train the teacher model is private data. When using the teacher model to perform knowledge distillation on the student model, the training data used by the student model during the training phase cannot be obtained. Therefore, a solution for obtaining training data for the student model is urgently needed.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a model training method, a training data acquisition method, and related equipment. Since the teacher model is obtained by training with private data as training data, the teacher model is familiar with the private data. Therefore, the process of the first training operation is to use the ability of the teacher model to guide the text generated by the machine learning model to approximate the private data, so that the training data used when training the student model using the distillation method can be similar to the private data, which is beneficial to improving the accuracy of the trained student model when performing natural language processing tasks; during the execution of the first training operation, only the first prompt is updated, which reduces the number of parameters that need to be updated during the execution of the first training operation, is beneficial to reducing the execution difficulty of the first training operation, is beneficial to reducing the computer resources consumed during the execution of the first training operation, and is also beneficial to improving the execution efficiency of the first training operation.

[0008] To solve the above technical problems, the embodiments of the present application provide the following technical solutions:

[0009] In a first aspect, an embodiment of the present application provides a model training method that can be used in the field of text processing in the field of artificial intelligence. The method includes: a first training device inputs a first text into a first machine learning model, and obtains a first machine learning model including a third text and a first prompt, and the third text belongs to a first data set. The first training device inputs a second text into a teacher model, processes the second text through the teacher model, and obtains a first prediction result output by the teacher model; wherein the teacher model is a first machine learning model that has performed a training operation. The first training device performs a first training operation based on the first prediction result until the convergence condition of the first loss function is met. During the execution of the first training operation, the parameters of the first machine learning model and the teacher model are kept unchanged, and the expression of the first prompt is iteratively updated. The trained first prompt is used to obtain a second data set. In the process of performing the second training operation on the student model using the teacher model in a distillation manner, the training data in the second data set will be used. "The training data in the second data set" can be understood as the multiple texts included in the second data set.

[0010] In this implementation, when using the distillation method to perform the second training operation on the student model using the teacher model, the training data in the second data set will be used. The training data in the second data set is generated by a special first machine learning model after completing the first training operation, and the first training operation is performed based on the first prediction result output by the teacher model. The input of the teacher model is the output text of the aforementioned first machine learning model. Since the teacher model is obtained after training with the private data as training data, the teacher model is familiar with the private data. Therefore, the process of the first training operation is to use the ability of the teacher model to guide the text generated by the first machine learning model to approach the private data, so that the training data used when training the student model using the distillation method can The first machine learning model is similar enough to the privacy data, which is conducive to improving the accuracy of the trained student model when performing natural language processing tasks; the input of the first machine learning model is the text in the existing first data set, and the first data set includes rich text. Correspondingly, the output text of the first machine learning model is also rich, thereby ensuring the richness of the training data of the student model; in addition, in the process of executing the first training operation, the parameters of the first machine learning model and the teacher model are kept unchanged, and only the first prompt is updated, which reduces the amount of parameters that need to be updated in the process of executing the first training operation, is conducive to reducing the difficulty of executing the first training operation, is conducive to reducing the computer resources consumed in the process of executing the first training operation, and is also conducive to improving the execution efficiency of the first training operation.

[0011] In one possible implementation, the first loss function includes a first loss term, and the first prediction result includes at least one vector corresponding to at least one character. Optionally, if the output of the teacher model is text including at least one character, the first prediction result includes at least one vector corresponding to the at least one character. Training using the first loss term aims to make each vector in the first prediction result closer to a one-hot form.

[0012] In this implementation, the closer each vector included in the first prediction result is to the one-hot form, the more confident the teacher model is about the output first prediction result, which means that the second text input into the teacher model is data familiar to the teacher model, that is, the second text is more similar to the private data used when training the teacher model. The use of the first loss term is beneficial to improving the similarity between the final second text (that is, the training data used when performing knowledge distillation on the student model) and the private data used when training the teacher model. In addition, when the output of the teacher model is text, that is, when the first prediction result includes multiple vectors that correspond one-to-one to at least one word in the text, the purpose of using the first loss term for training includes making each vector in the first prediction result close to the one-hot form, that is, the first loss term is compatible with the case where the output of the teacher model is text, which greatly expands the application scenarios of this solution.

[0013] In one possible implementation, the first loss function includes a second loss term, which indicates the information entropy of multiple first prediction results. The purpose of training using the second loss term includes improving the information entropy of the multiple first prediction results; wherein the information entropy of the multiple first prediction results reflects the degree of chaos of the information system composed of the multiple first prediction results. The greater the information entropy of the multiple first prediction results, the greater the chaos of the information system composed of the multiple first prediction results, that is, the more diverse the multiple first prediction results are.

[0014] In this implementation, if the information entropy of the multiple first prediction results is larger, it means that the multiple first prediction results are more diverse. For the teacher model, the multiple second texts corresponding one-to-one to the aforementioned multiple first prediction results are diverse, which is conducive to improving the diversity of the multiple second texts obtained; in addition, the method of using the second loss term to improve the diversity of the multiple second texts avoids the coupling between the "execution process of the first training operation" and the "generation process of the second text adopted in the training stage of the student model", that is, the first training operation does not need to be executed each time the "generation process of the second text adopted in the training stage of the student model" is executed, which reduces the difficulty of the "generation process of the second text adopted in the training stage of the student model" and improves the flexibility when executing the "generation process of the second text adopted in the training stage of the student model".

[0015] In one possible implementation, the first prompt includes a prompt word and a second prompt. The prompt word means "summarize." During the first training operation, the prompt word may not be updated, and only the expression of the second prompt may be updated. For example, the prompt word may be "summarize" or "conclude." Another example is "generalize," "induct," "summarize," or other words.

[0016] In this implementation, the first prompt is designed to include two parts: a prompt word and a second prompt. The meaning of the prompt word is summary. The function of the prompt word is to prompt the first machine learning model to summarize the information of the second prompt and the third text, that is, the second prompt plus the third text has a certain correlation with the second text output by the first machine learning model. Since the third text comes from an existing data set, the third text is rich. The above-mentioned scheme is conducive to retaining the richness of the third text in the multiple second texts finally obtained.

[0017] In one possible implementation, the first machine learning model includes a normalized exponential (softmax) layer. In the process of the first machine learning model processing the first text, the data that the softmax layer in the first machine learning model needs to process is one or more first data obtained based on the first text, and each first data can correspond to a word in the second text. The process of using the softmax layer to process each first data can include: adding perturbations to each first data to obtain updated first data, and processing each updated first data using the softmax function. Optionally, the softmax layer in the first machine learning model can be the last neural network layer of the first machine learning model, that is, the processing result output after each updated first data is processed by the softmax layer in the first machine learning model can be used to indicate each word in the second text.

[0018] In this implementation, compared with the original first data, the updated first data obtained after adding disturbance to the first data is more diverse. Correspondingly, compared with the processing result of the first data obtained by directly processing the first data through the softmax function, the processing result of the updated first data obtained by processing the updated first data through the softmax function is more diverse. The softmax layer in the first machine learning model is often the last neural network layer of the first machine learning model, that is, the processing result obtained by the softmax layer indicates the second text output by the entire first machine learning model. "The processing result of the updated first data is more diverse" means that the second text output by the entire first machine learning model is more diverse, that is, adding disturbance to the first data to be processed by the softmax layer is conducive to improving the diversity of the second text output by the machine learning model.

[0019] In one possible implementation, the task of the teacher model is to obtain at least one category of information from the second text, the first prediction result includes at least one group of words corresponding one-to-one to the at least one category, the first loss function includes a third loss term, and the third loss term can indicate the similarity between different words included in the same group of words in at least one group of words. The purpose of training with the third loss term includes improving the similarity between different words included in the same group of words in at least one group of words.

[0020] In this implementation, the purpose of using the third loss term for training includes improving the similarity between different words in the same category of information. Since if the task of the teacher model is to obtain at least one category of information from the input second text, different words in the same category of information should be more similar. When different words in the same category of information are more similar, it means that the first prediction result generated by the teacher model when performing the information acquisition task on the input second text is more accurate, which can reflect that the second text input into the teacher model is a text that the teacher model is better at processing, that is, the second text is data that the teacher model is familiar with. Therefore, using the third loss term can improve the similarity between the final second text and the private training data used when training the teacher model.

[0021] In one possible implementation, if the task of the teacher model is to obtain at least two types of information from the second text, that is, the first prediction result includes at least two groups of words corresponding one-to-one to the aforementioned at least two categories, the third loss term can also indicate the similarity between different groups of words in the at least two groups of words included in the first prediction result. The purpose of training using the third loss term also includes reducing the similarity between different groups of words in at least one group of words.

[0022] In this implementation, the purpose of using the third loss term for training also includes reducing information from different categories. If the task of the teacher model is to obtain at least one category of information from the input second text, the words from different categories of information should be more dissimilar. When different words in different categories of information are more dissimilar, it means that the first prediction result generated by the teacher model when performing the information acquisition task on the input second text is more accurate, which can reflect that the second text input to the teacher model is data that the teacher model is familiar with. Therefore, using the third loss term can improve the similarity between the final second text and the private training data used when training the teacher model.

[0023] In one possible implementation, the teacher model and the student model can both be used to perform natural language processing tasks. For example, the task types of the teacher model and the student model can both be any of the following: classifying text, obtaining at least one type of information from text, sequence labeling text, inferring sentence relationships between two texts, or performing generative tasks based on text, etc.

[0024] In the second aspect, the embodiment of the present application provides a method for obtaining training data, which can be used in the field of text processing in the field of artificial intelligence. The method includes: executing a device to obtain a first text, the first text includes a third text and a first prompt after training, the third text belongs to the first data set, and the first prompt after training is obtained through a first training operation; inputting the first text into a first machine learning model to obtain a second text output by the first machine learning model. In which, when using a distillation method, a second data set is used in the process of performing a second training operation on a student model using a teacher model, and the second data set includes multiple second texts; when performing the first training operation using a first loss function, the parameters of the machine learning model and the teacher model are kept unchanged, and the expression of the first prompt is updated. The input of the teacher model is the output text of the machine learning model, and the loss function is obtained based on the first prediction result output by the teacher model.

[0025] The meanings of the various terms in the second aspect of this application and the beneficial effects brought about by the second aspect can all be referred to the first aspect and will not be repeated here.

[0026] On the third aspect, an embodiment of the present application provides a model training method that can be used in the field of text processing in the field of artificial intelligence, and the method includes: a second training device inputs a second text into a teacher model and a student model respectively, and obtains a second prediction result output by the teacher model and a third prediction result output by the student model; the second training device trains the student model by distillation according to the second prediction result and the third prediction result; wherein, the second text is generated by a first machine learning model based on a first prompt after training, the input of the first machine learning model is the first text, the first text includes a third text and the aforementioned first prompt after training, the third text belongs to the first data set, and the first prompt after training is obtained through a first training operation. When the first training operation is performed using the first loss function, the parameters of the first machine learning model and the teacher model are kept unchanged, and the expression of the first prompt is updated. The input of the teacher model is the output text of the first machine learning model, and the first loss function is obtained based on the first prediction result output by the teacher model.

[0027] The meanings of the various terms in the third aspect of this application and the beneficial effects brought about by the third aspect can all be referred to the first aspect and will not be repeated here.

[0028] In a fourth aspect, an embodiment of the present application provides a model training device that can be used in the field of text processing in the field of artificial intelligence. The model training device includes: an input module for inputting a first text into a first machine learning model to obtain a second text output by the first machine learning model, wherein the first text includes a third text and a first prompt, and the third text belongs to a first data set; the input module is also used to input the second text into a teacher model to obtain a first prediction result output by the teacher model; a training module is used to perform a first training operation based on the first prediction result until the convergence condition of the first loss function is met. During the execution of the first training operation, the parameters of the first machine learning model and the teacher model are kept unchanged, and the first prompt is updated; wherein the trained first prompt is used to obtain a second data set, and the training data in the second data set will be used in the process of performing the second training operation on the student model using the teacher model in a distillation manner.

[0029] In the fourth aspect of this application, the model training device is also used to execute the steps performed by the first training device in the first aspect. The specific implementation methods of the steps in the fourth aspect of this application, the meaning of each noun and the beneficial effects brought about can all be referred to the first aspect and will not be repeated here.

[0030] In a fifth aspect, an embodiment of the present application provides a device for acquiring training data, which can be used in the field of text processing in the field of artificial intelligence. The device for acquiring training data includes: an acquisition module for acquiring a first text, the first text including a third text and a first prompt after training, the third text belongs to the first data set, and the first prompt after training is obtained through a first training operation; an input module for inputting the first text into a first machine learning model to obtain a second text output by the first machine learning model, wherein the second data set is used in the process of performing the second training operation on the student model using a teacher model in a distillation manner, and the second data set includes multiple second texts; when performing the first training operation using the first loss function, the parameters of the first machine learning model and the teacher model are kept unchanged, the first prompt is updated, the input of the teacher model is the output text of the first machine learning model, and the first loss function is obtained based on the first prediction result output by the teacher model.

[0031] The meanings of the various terms in the fifth aspect of this application and the beneficial effects brought about by the fifth aspect can be referred to the first aspect and will not be repeated here.

[0032] In a sixth aspect, an embodiment of the present application provides a model training device that can be used in the field of text processing in the field of artificial intelligence. The model training device includes: an input module for inputting a second text into a teacher model and a student model respectively, to obtain a second prediction result output by the teacher model and a third prediction result output by the student model; a training module for training the student model by distillation according to the second prediction result and the third prediction result; wherein the second text is generated by a first machine learning model based on a first prompt after training, the input of the first machine learning model is the first text, the first text includes a third text and the aforementioned first prompt after training, the third text belongs to the first data set, the first prompt after training is obtained through a first training operation, when the first training operation is performed using the first loss function, the parameters of the first machine learning model and the teacher model are kept unchanged, the expression of the first prompt is updated, the input of the teacher model is the output text of the first machine learning model, and the first loss function is obtained based on the first prediction result output by the teacher model.

[0033] The meanings of the various terms in the sixth aspect of this application and the beneficial effects brought about by the sixth aspect can be found in the first aspect and will not be repeated here.

[0034] In the seventh aspect, an embodiment of the present application provides a training device, including a processor and a memory, where the processor is coupled to the memory, the memory is used to store programs; the processor is used to execute the programs in the memory, so that the training device executes the training method of the model described in the first aspect or the third aspect above.

[0035] In an eighth aspect, an embodiment of the present application provides an execution device, including a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store programs, and the processor is used to execute the programs in the memory, so that the execution device executes the method for acquiring training data described in the second aspect above.

[0036] In a ninth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method described in the first, second or third aspect above.

[0037] In a tenth aspect, an embodiment of the present application provides a computer program product, which includes a program. When the program runs on a computer, it enables the computer to execute the method described in the first, second or third aspect above.

[0038] In an eleventh aspect, the present application provides a chip system, which includes a processor for supporting the implementation of the functions involved in the above-mentioned various aspects, such as sending or processing the data and / or information involved in the above-mentioned methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the terminal device or communication device. The chip system can be composed of a chip or can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] FIG1 is a schematic diagram of the structure of an artificial intelligence main framework provided in an embodiment of the present application;

[0040] FIG2 is a system architecture diagram of a data processing system provided in an embodiment of the present application;

[0041] FIG3 is a flow chart of a method for training a model according to an embodiment of the present application;

[0042] FIG4 is a schematic diagram of a first text provided in an embodiment of the present application;

[0043] FIG5 is a schematic diagram of several one-hot vectors provided in an embodiment of the present application;

[0044] FIG6 is another flow chart of a method for training a model according to an embodiment of the present application;

[0045] FIG7 is a flow chart of a method for acquiring training data according to an embodiment of the present application;

[0046] FIG8 is another flow chart of a method for training a model according to an embodiment of the present application;

[0047] FIG9 is a schematic structural diagram of a model training device provided in an embodiment of the present application;

[0048] FIG10 is a schematic diagram of a structure of a device for acquiring training data provided in an embodiment of the present application;

[0049] FIG11 is another structural diagram of a model training device provided in an embodiment of the present application;

[0050] FIG12 is a schematic structural diagram of a device provided in an embodiment of the present application;

[0051] FIG13 is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0053] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0054] First, let's describe the overall workflow of an AI system. See Figure 1, which shows a schematic diagram of the AI ​​framework. This framework will be explained from two perspectives: the "intelligent information chain" (horizontal axis) and the "IT value chain" (vertical axis). The "intelligent information chain" reflects the entire process from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. Throughout this process, data undergoes a condensed journey from "data-information-knowledge-wisdom." The "IT value chain," spanning the underlying infrastructure of human intelligence, information (provided and processed by technology), and the system's industrial ecosystem, reflects the value that AI brings to the information technology industry.

[0055] (1) Infrastructure

[0056] The infrastructure provides computing power for AI systems, enabling communication with the outside world and providing support through a basic platform. Communication with the outside world is achieved through sensors; computing power is provided by smart chips, which can specifically adopt hardware acceleration chips such as central processing units (CPUs), embedded neural network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field programmable gate arrays (FPGAs); the basic platform includes related platform guarantees and support such as distributed computing frameworks and networks, and can include cloud storage and computing, and interconnected networks. For example, sensors communicate with the outside world to obtain data, which is then provided to the smart chips in the distributed computing system provided by the basic platform for calculation.

[0057] (2) Data

[0058] Data above the infrastructure layer represents data sources for AI. This data includes graphics, images, voice, and text, as well as IoT data from traditional devices. This includes business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0059] (3) Data processing

[0060] Data processing generally includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0061] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0062] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0063] Decision-making refers to the process of making decisions after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0064] (4) General ability

[0065] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0066] (5) Smart products and industry applications

[0067] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart terminals, smart manufacturing, smart transportation, smart homes, smart medical care, smart security, autonomous driving, smart cities, etc.

[0068] This application can be applied to various application fields of artificial intelligence technology. For example, it can be used to obtain training data used by machine learning models in various application fields during the training phase. Optionally, the training method used in the aforementioned training phase is knowledge distillation, which refers to a model compression method that transfers the "knowledge" learned by the teacher model to a simpler student model.

[0069] For example, the training data is specifically text, meaning both the teacher model and the student model are used to perform natural language processing (NLP) tasks. Natural language processing is the processing of human language. Natural language processing is the process of systematically analyzing, understanding, and extracting information from text data using a machine learning model. Machine learning models can be used to perform natural language processing tasks in applications such as smart terminals, smart homes, and autonomous driving.

[0070] In the various application areas mentioned above, by using the aforementioned machine learning models, we can manage very large amounts of text data, perform a large number of automated tasks, and solve a variety of problems, such as automatic summarization, machine translation (MT), named entity recognition (NER), relation extraction (RE), information extraction (IE), sentiment analysis, speech recognition, question answering, and topic segmentation.

[0071] For example, natural language processing tasks can be divided into the following categories.

[0072] Classification tasks: The machine learning model outputs a classification value for the entire input text, such as sentiment classification, topic classification, or whether the grammar is correct.

[0073] Information extraction task: obtain at least one type of information from the input text through a machine learning model.

[0074] Sequence labeling: Each word in a text requires the machine learning model to assign a classification category based on the context, such as Chinese word segmentation, part-of-speech tagging, named entity recognition, or semantic role labeling.

[0075] Sentence relationship inference: A machine learning model takes two texts as input and uses them to determine whether the two texts have a certain nominal relationship. Examples include question-answering systems, semantic rewriting, and natural language inference.

[0076] Generative tasks: Input a piece of text and generate another piece of text through a machine learning model. Examples include machine translation, automatic summarization, or writing poetry and sentences.

[0077] It should be noted that the method provided in this application can also be applied to application scenarios where machine learning models are used to perform other natural language processing tasks. The above examples are only for the convenience of understanding this solution and are not used to limit this solution.

[0078] As artificial intelligence technology develops, the machine learning models used to process text are increasing. Therefore, knowledge distillation can be used to compress the teacher model that has performed training operations to obtain a smaller student model. However, in some scenarios, the training data used to train the teacher model is private data, and thus the training data used to train the student model cannot be obtained. In the method provided in this application, the first training operation is performed with the help of the ability of the teacher model. After completing the first training operation, the first machine learning model is used to generate the training data required when using the teacher model to perform knowledge distillation on the aforementioned student model.

[0079] Before giving a detailed introduction to the method provided in this application, please refer to Figure 2, which is a system architecture diagram of the data processing system provided in an embodiment of this application. In Figure 2, the data processing system includes a first training device 210, a database 220, an execution device 230 and a second training device 240.

[0080] Among them, a first data set is stored in the database 220, and the first data set includes a large amount of text. During the execution phase of the first training operation, the first training device 210 performs the first training operation using the first data set. Exemplarily, the first training device 210 inputs the first text into the first machine learning model 201, and obtains the second text output by the first machine learning model 201, and the first text is obtained based on the first data set; the first training device 210 inputs the second text into the teacher model 202, and obtains the first prediction result output by the teacher model 202, and performs the first training operation based on the first prediction result. Among them, the first machine learning model 201 can be specifically expressed as a neural network, or as a non-neural network model. In the embodiment of the present application, only the first machine learning model 201 is expressed as a neural network as an example for explanation.

[0081] During the generation stage of the second data set (i.e., the training data of the student model 203), the first machine learning model 201 is deployed in the execution device 230. After completing the first training operation, the execution device 230 can use the first machine learning model 201 to generate multiple second texts. The second data set used in the training stage of the student model 203 includes the aforementioned multiple second texts.

[0082] During the execution phase of the second training operation, the second training device 240 can obtain a second data set, which includes multiple second texts. The second training device 240 can input the second text distribution into the teacher model 202 and the student model 203 to obtain the second prediction result output by the teacher model 202 and the third prediction result output by the student model 203; the second training device 240 uses distillation to iteratively train the student model 203 based on the second prediction result and the third prediction result.

[0083] It is worth noting that Figure 2 is merely a schematic diagram of the architecture of a data processing system provided by an embodiment of the present invention, and the positional relationships between the devices, components, modules, etc. shown in the figure do not constitute any limitation. For example, in other embodiments of the present application, the first training device 210, the execution device 230, and the second training device 240 may be the same device; or, the first training device 210 may be an independent device, and the execution device 230 and the second training device 240 may be the same device; or, the first training device 210 and the execution device 230 may be the same device, and the second training device 240 may be another independent device, etc., all of which are not limited in the embodiments of the present application.

[0084] In combination with the above description, it can be seen that the method provided in this application may include three stages. The following describes the specific implementation process of these three stages: the execution stage of the first training operation, the generation stage of the second data set, and the execution stage of the second training operation.

[0085] 1. Execution phase of the first training operation

[0086] In the embodiment of the present application, specifically, please refer to FIG3 , which is a flow chart of a model training method provided in the embodiment of the present application. The model training method provided in the embodiment of the present application may include:

[0087] 301. Input a first text into a first machine learning model to obtain a second text output by the first machine learning model, where the first text includes a third text and a first prompt, and the third text belongs to the first data set.

[0088] In an embodiment of the present application, before inputting the first text into the first machine learning model, the first training device must first obtain the first text based on a first dataset, where the first dataset is already available. "The first dataset is already available" can be understood as meaning that the first dataset is a dataset accessible to the first training device. For example, the first dataset can be a publicly available text dataset, or the first dataset can be a text dataset accessible to the first training device but not publicly available.

[0089] The first machine learning model is a machine learning model whose input and output are both text; for example, the first machine learning model can adopt a generator, and for another example, the first machine learning model can include an encoder and a decoder, etc. The first machine learning model can also be expressed in other forms, which can be flexibly determined in combination with the actual application scenario, and are not limited in the embodiments of this application.

[0090] The first training device can implement "obtaining the first text based on the first training data set" in a variety of ways. In one implementation, the first training operation can be performed based on the idea of ​​prompt learning, then each first text includes a third text and a first prompt, and the third text belongs to the first data set, that is, the first data set includes multiple third texts. Then the first training device obtaining the first text may include: the first training device obtains one or more third texts from the first data set, and combines each third text with the first prompt to obtain each first text. Optionally, in the process of the first training device performing the first training operation, the parameters of the first machine learning model and the teacher model are kept unchanged, and the first prompt is iteratively updated.

[0091] For example, since the first prompt is iteratively updated during the first training operation, if the first training device is acquiring the first text for the first time, that is, the first training device is training the first prompt for the first time, the first training device can acquire the initial expression of the first prompt and combine each third text with the initial expression of the first prompt to obtain each first text. If the first training device is not performing the first training on the first prompt for the first time, the first training device can acquire the updated expression of the first prompt obtained in the previous training and combine each third text with the updated expression of the first prompt to obtain each first text.

[0092] For example, the initial expression and updated expression of the first prompt may be specifically expressed as a vector, or other forms of data, etc., which are not limited in the embodiments of the present application.

[0093] The first prompt may be specifically expressed in various forms. In one implementation, the first prompt may be a mixed prompt, which may include a prompt word and a second prompt. During the execution of the first training operation, the prompt word may not be updated, and only the second prompt may be updated.

[0094] Optionally, the prompt word used in the first prompt may mean summary; for example, the prompt word may be "summarize" or "conclude", or for another example, the prompt word may be "generalize", "summarize", "summarize" or other words, etc. The examples of prompt words here are only to prove the feasibility of this solution and are not used to limit this solution.

[0095] For a more intuitive understanding of this solution, please refer to Figure 4, which is a schematic diagram of the first text provided in an embodiment of this application. As shown in Figure 4, the first text may include a prompt word, a first prompt, and a third text. Figure 4 uses the prompt word "summarize" as an example. It should be understood that the example in Figure 4 is only for the convenience of understanding the concept of "first text" and is not intended to limit this solution.

[0096] In an embodiment of the present application, the first prompt is designed to include two parts: a prompt word and a second prompt. The meaning of the prompt word is summary. The function of the prompt word is to prompt the first machine learning model to summarize the information of the second prompt and the third text, that is, the second prompt plus the third text has a certain correlation with the second text output by the first machine learning model. Since the third text comes from an existing data set, the third text is rich. The above-mentioned scheme is conducive to retaining the richness of the third text in the multiple second texts finally obtained.

[0097] Optionally, in another implementation, the first data set includes a plurality of first texts, and the first training device obtaining the first texts may also include: the first training device directly obtaining each first text from the first data set.

[0098] In an embodiment of the present application, the first training device may input each first text into a first machine learning model, process the first text using the first machine learning model, and obtain a second text output by the first machine learning model. For example, the second text may include multiple characters, and the first machine learning model may output multiple first vectors representing the second text, each of the multiple first vectors indicating a character in the second text.

[0099] The first machine learning model includes multiple neural network layers, and the multiple neural network layers may include at least one normalized index (softmax) layer. In the process of the first machine learning model processing the first text, the data that the softmax layer in the first machine learning model needs to process is at least one first data obtained based on the first text, that is, the aforementioned at least one first data can be the data generated by the previous neural network layer of the aforementioned softmax layer in the process of the first machine learning model processing the first text. Optionally, each first data can correspond to a word in the second text, and the previous neural network layer of the softmax layer in the first machine learning model can generate one first data at a time, or multiple first data corresponding one-to-one to multiple words included in the entire second text.

[0100] Optionally, the softmax layer in the first machine learning model can be the last neural network layer of the first machine learning model, that is, the processing result output after processing each first data through the softmax layer in the first machine learning model can be used to indicate each word in the second text.

[0101] In one implementation, the process of the first training device using the softmax layer in the first machine learning model to process each first data may include: the first training device obtains the softmax layer in the first machine learning model, adds disturbance to each first data to obtain the updated first data, and processes the updated first data using the softmax function. Exemplarily, the disturbance added to each first data may be random, or may be obtained based on a preset distribution space. For example, the aforementioned disturbance may be obtained based on the Gumbel distribution space, Gaussian distribution space or other distribution space, etc. The specific method can be flexibly determined in combination with the actual application scenario, and is not limited in the embodiments of the present application. To further understand this solution, an example of the formula used when processing the first data through the aforementioned softmax layer is disclosed below:

[0102] Wherein, y represents the processing result obtained by processing each first data using the softmax layer in the first machine learning model, h represents a first data, Represents the disturbance added to the first data, g represents a vector sampled from the Gumbel distribution space; σ is a hyperparameter, optionally, the value of σ is around 1, that is, the difference between σ and 1 is within a preset range; τ is also a hyperparameter, τ can take a lower value in this solution, illustratively, the value of τ can be 0.001 or other values, etc. It should be noted that formula (1) is only an example for facilitating the understanding of this solution and is not used to limit this solution.

[0103] In an embodiment of the present application, compared with the original first data, the updated first data obtained after adding disturbance to the first data is more diverse. Correspondingly, compared with the processing result of the first data obtained by directly processing the first data through the softmax function, the processing result of the updated first data obtained by processing the updated first data through the softmax function is more diverse; and the softmax layer in the first machine learning model is often the last neural network layer of the first machine learning model, that is, the processing result obtained by the softmax layer indicates the second text output by the entire first machine learning model, and "the processing result of the updated first data is more diverse" means that the second text output by the entire first machine learning model is more diverse, that is, adding disturbance to the first data to be processed by the softmax layer is beneficial to improving the diversity of the second text output by the machine learning model.

[0104] In another implementation, the process of the first training device processing each first data using the softmax layer in the first machine learning model may include: the first training device directly processes each first data through the softmax layer in the first machine learning model.

[0105] Optionally, if the last neural network layer of the first machine learning model does not adopt the softmax layer, after obtaining at least one second data generated by the second-to-last neural network layer of the first machine learning model, disturbance can be added to each of the aforementioned second data to obtain updated second data, and each updated second data is processed by the last neural network layer of the first machine learning model to obtain the processing result of each updated second data output by the last neural network layer of the first machine learning model (that is, the output of the entire first machine learning model); the processing result of each updated second data is used to indicate a word in the second text.

[0106] 302. Input the second text into the teacher model to obtain a first prediction result output by the teacher model.

[0107] In an embodiment of the present application, after obtaining the second text output by the first machine learning model, the first training device can input the second text into the teacher model, process the second text through the teacher model, and obtain the first prediction result output by the teacher model, wherein the teacher model is a machine learning model that has performed training operations.

[0108] The teacher model can be used to perform any one or more natural language processing tasks. For examples of various natural language processing tasks, please refer to the above description, which will not be repeated here. The type of task performed by the specific teacher model can be determined in combination with the actual application scenario, and is not limited in the embodiments of this application.

[0109] The content indicated by the first prediction result is related to the category of the task that the teacher model is used to perform. For example, if the teacher model is used to perform sentiment classification on the input second text, the first prediction result may indicate whether the sentiment category of the second text is positive, negative, or neutral. For another example, if the teacher model is used to perform part-of-speech tagging on the input second text, the first prediction result may indicate the part-of-speech category of each word in the second text. The part-of-speech category may be verb, noun, adverb, or other categories, etc., which are not exhaustive here.

[0110] For another example, if the teacher model is used to extract information from the input second text to extract time and event type from the input second text, the first prediction result may include two groups of words, one group of the two groups of words includes the time extracted from the second text, and the other group of the two groups of words includes the time type extracted from the second text.

[0111] For another example, if the teacher model is used to translate the input second text, the first prediction result can be the translation result of the second text, and so on. It should be noted that the example of the first prediction result here is only for the convenience of understanding this solution and does not limit this solution.

[0112] The first prediction result may include one or more second vectors. In one application scenario, if the teacher model outputs a text including at least one word, the first prediction result includes at least one second vector corresponding to the at least one word, each second vector being used to indicate a word in the text. In another application scenario, if the teacher model outputs a predicted category for the entire second text input, the first prediction result may also be represented by a second vector, which is used to indicate the predicted category of the second text.

[0113] 303. Based on the first prediction result, a first training operation is performed using a first loss function. During the execution of the first training operation, the parameters of the first machine learning model and the teacher model are kept unchanged, and the first prompt is updated, wherein the trained first prompt is used to obtain a second data set, and the training data in the second data set is used during the process of performing the second training operation on the student model using the teacher model using a distillation method.

[0114] In an embodiment of the present application, after obtaining the first prediction result output by the teacher model, the first training device can perform a first training operation based on the first prediction result. Specifically, in one implementation method, during the execution of the first training operation, the parameters of the first machine learning model and the teacher model are kept unchanged, and the first prompt is updated. Exemplarily, the first training device can calculate the function value of the first loss function based on the first prediction result, perform gradient differentiation on the function value of the first loss function, and use the backpropagation algorithm to update the expression of the first prompt to complete one training of the first prompt. The first training device repeats steps 301 to 303 multiple times to implement iterative training of the first prompt until the convergence condition is met, and determines that the first training operation is completed. The aforementioned convergence condition can be to meet the convergence condition of the first loss function, or it can be that the number of iterative trainings for the first prompt reaches a preset number.

[0115] Optionally, in another implementation, the parameters of the first machine learning model may be updated during the execution of the first training operation. For example, the first training device may calculate the function value of the first loss function based on the first prediction result, perform gradient differentiation on the function value of the first loss function, and use a backpropagation algorithm to update the parameters of the first machine learning model to complete one training of the first machine learning model. The first training device repeats steps 301 to 303 multiple times to implement iterative training of the first machine learning model until the convergence condition is met, thereby determining that the first training operation is completed.

[0116] "The training data in the second data set" can be understood as the multiple texts included in the second data set. The specific implementation process of "generating the training data in the second data set by the first machine learning model after completing the first training operation" and "using the teacher model to perform the second training operation on the student model" will be described in subsequent embodiments and will not be repeated here.

[0117] Optionally, the first loss function may include a first loss term, the first prediction result includes at least one second vector, and the purpose of training using the first loss term includes making each of the aforementioned at least one second vector closer to a one-hot form; optionally, if the output of the teacher model is a text including at least one word, the first prediction result includes at least one second vector corresponding one-to-one to the aforementioned at least one word.

[0118] Exemplarily, a one-hot vector includes multiple values ​​in which one 1 exists, and all other values ​​in the one-hot vector are 0. For a more intuitive understanding of this solution, please refer to FIG5 , which is a schematic diagram of several one-hot vectors provided in an embodiment of the present application. FIG5 shows multiple one-hot vectors, including three vectors of length 5, two vectors of length 7, and one vector of length 10. As shown in FIG5 , each one-hot vector includes only one 1, and the remaining elements are all 0. It should be understood that the example in FIG5 is only for the convenience of understanding the concept of "one-hot vector" and is not used to limit this solution.

[0119] Optionally, the first loss term may indicate the similarity between each second vector and the third vector, and the purpose of training using the first loss term includes enabling each second vector to approximate the form of the third vector; the third vector is obtained based on the second vector, and the third vector is in the form of one-hot. Exemplarily, each second vector may include multiple values, and the first training device obtaining the third vector based on the second vector may include: the first training device determines the largest value in the second vector as 1, and determines all other values ​​except the maximum value in the second vector as 0, thereby obtaining the third vector. For example, if the second vector is expressed as (0.1, 0.1, 0.2, 0.6), since 0.6 is the largest value among the 4 values ​​included in the aforementioned second vector, 0.6 in the second vector is determined to be 1, and the other values ​​in the second vector are determined to be 0 to obtain the third vector, then the third vector obtained based on the aforementioned second vector can be expressed as (0, 0, 0, 1).

[0120] For another example, if the second vector is expressed as (0.01, 0.08, 0.01, 0.2, 0, 0, 0.7), since 0.7 has the largest value among the 7 values ​​included in the second vector, 0.7 in the second vector is determined to be 1, and the other values ​​in the second vector are determined to be 0 to obtain the third vector. The third vector obtained based on the second vector can be expressed as (0, 0, 0, 0, 0, 1) and so on. It should be noted that the examples here are only for the convenience of understanding “obtaining a one-hot third vector based on the second vector”, and the specific expressions of the “second vector” and the “third vector” can be determined in combination with actual conditions, and are not limited in the embodiments of the present application.

[0121] To further understand this solution, let's take the output of the teacher model as an example, where the teacher model outputs multiple vectors. Here is an example of the function used for the first loss term:

[0122] Among them, L SPECRepresents the first loss term, N represents the second text in a batch (batch) including N second texts, represents the jth second vector included in the first prediction result output by the teacher model after the i-th second text among the N second texts is input into the teacher model, that is, a second vector corresponding to the j-th word among the multiple words included in the first prediction result, Representative based on The resulting third vector is Is a one-hot vector, "based on get The specific implementation process of " can refer to the above description and will not be repeated here; Representative Calculation and The similarity between The calculation is and Take the cross-entropy (CE) between as an example, and The cross entropy between and The degree of difference between and The greater the difference between and The larger the cross entropy between them, the better the L SPEC The purpose of training is to make L SPEC Getting smaller and smaller, that is, and It should be noted that the examples in formula (2) are only for the convenience of understanding this solution. For example, “calculating and The method of "similarity between" can also be replaced by calculating and The cosine similarity, L1 distance between them or other similarity calculation methods may be used; for example, the N second texts may not include all the second texts in a batch, and the value of N may also be 2, 3, 4, 5 or other values, etc. The examples in formula (2) are not used to limit this solution.

[0123] In an embodiment of the present application, if each vector included in the first prediction result is closer to the one-hot form, it means that the teacher model is more confident about the output first prediction result, which means that the second text input into the teacher model is data familiar to the teacher model, that is, it means that the second text is more similar to the private data used when training the teacher model. The use of the first loss term is beneficial to improving the similarity between the final second text (that is, the training data used when performing knowledge distillation on the student model) and the private data used when training the teacher model; in addition, when the output of the teacher model is text, that is, when the first prediction result includes multiple vectors that correspond one-to-one to at least one word in the text, the purpose of using the first loss term for training includes making each vector in the first prediction result close to the one-hot form, that is, the first loss term is compatible with the case where the output of the teacher model is text, which greatly expands the application scenarios of this solution.

[0124] Optionally, the first loss function may include a second loss term, which indicates the information entropy of the multiple first prediction results. The purpose of training with the second loss term includes improving the information entropy of the multiple first prediction results; wherein the information entropy of the multiple first prediction results reflects the degree of chaos of the information system composed of the multiple first prediction results. The greater the information entropy of the multiple first prediction results, the greater the chaos of the information system composed of the multiple first prediction results, that is, the more diverse the multiple first prediction results are.

[0125] Exemplarily, the at least one first prediction result mentioned above can be a first prediction result corresponding to a preset number of second texts. The preset number of second texts can be the second texts in a batch, or can include second texts in different batches. The preset number can be 2, 3, 4, 5 or other values, etc., or, the at least one first prediction result mentioned above can include the first prediction result corresponding to all second texts in a batch, or, the at least one first prediction result mentioned above can include the first prediction result corresponding to all second texts in half a batch, etc. The specific number of first prediction results after which the second loss item is calculated can be flexibly determined in combination with the actual application scenario, and is not limited in the embodiments of the present application.

[0126] To further understand this solution, an example of the function used by the second loss term is disclosed below:

[0127] Among them, L divs Represents the second loss term, N represents the second text in a batch (batch) including N second texts, P i represents a first prediction result generated by the teacher model corresponding to the i-th second text in N second texts, represents the N first prediction results generated by the teacher model corresponding to the N second texts, Represents the calculation of the information entropy of N first prediction results, Represents the information entropy of the N first prediction results divided by N; using L divs The purpose of training includes making L divs The value of L is getting smaller and smaller, that is, using L divs The purpose of training includes Getting bigger and bigger, that is, using L divs The purpose of training includes increasing the information entropy of the plurality of first prediction results. It should be understood that the example of formula (2) is only for facilitating the understanding of this solution and is not used to limit this solution.

[0128] In an embodiment of the present application, if the information entropy of the multiple first prediction results is larger, it means that the multiple first prediction results are more diverse. For the teacher model, the multiple second texts corresponding one-to-one to the aforementioned multiple first prediction results are diverse, which is conducive to improving the diversity of the multiple second texts obtained; in addition, the method of using the second loss term to improve the diversity of the multiple second texts avoids the coupling between the "execution process of the first training operation" and the "generation process of the second text adopted in the training stage of the student model", that is, the first training operation does not need to be executed each time the "generation process of the second text adopted in the training stage of the student model" is executed, which reduces the difficulty of the "generation process of the second text adopted in the training stage of the student model" and improves the flexibility when executing the "generation process of the second text adopted in the training stage of the student model".

[0129] Optionally, if the task of the teacher model is to obtain at least one category of information from the second text, that is, the first prediction result includes at least one group of words corresponding one-to-one to the aforementioned at least one category, the first loss function may include a third loss term, and the third loss term may indicate the similarity between different words included in the same group of words in at least one group of words. The purpose of training using the third loss term includes improving the similarity between different words included in the same group of words in at least one group of words.

[0130] Exemplarily, if the task of the teacher model is to obtain time, place and event type from the input second text, the first prediction result may include three groups of words, the group of words corresponding to "time" includes "April 20" and "April 21", the group of words corresponding to "place" includes "XX Hot Spring Town" and "XX Square", and the group of words corresponding to "event type" includes "tourism". The third loss term can indicate the similarity between different words in the group of words corresponding to "time", the similarity between different words in the group of words corresponding to "place", and / or the similarity between different words in the group of words corresponding to "event type", etc.; in addition, since the group of words corresponding to "event type" includes only one word, the similarity between "tourism" and "tourism" can be calculated using the third loss term. It should be noted that the examples here are only for the convenience of understanding this solution and are not used to limit this solution.

[0131] Optionally, if the task of the teacher model is to obtain at least two types of information from the second text, that is, the first prediction result includes at least two groups of words corresponding one-to-one to the aforementioned at least two categories, the third loss term can also indicate the similarity between different groups of words in the at least two groups of words included in the first prediction result. The purpose of training using the third loss term also includes reducing the similarity between different groups of words in the at least two groups of words.

[0132] Exemplarily, if the task of the teacher model is to obtain time, place and event type from the input second text, the first prediction result may include three groups of words, the group of words corresponding to "time" includes "April 20" and "April 21", the group of words corresponding to "place" includes "XX Hot Spring Town" and "XX Square", and the group of words corresponding to "event type" includes "tourism". The third loss item can also indicate the similarity between a word in the group of words corresponding to "time" and a word in the group of words corresponding to "place", the similarity between a word in the group of words corresponding to "place" and a word in the group of words corresponding to "event type", or the similarity between a word in the group of words corresponding to "time" and a word in the group of words corresponding to "event type", etc. It should be noted that the examples here are only for the convenience of understanding this scheme and are not used to limit this scheme.

[0133] To further understand this solution, an example of the function used by the third loss term is disclosed below:

[0134] Among them, L scl represents the third loss term, R represents the first prediction result including R words, D rRepresents the rth word in the Rth word group. The rth word group output by the teacher model can specifically include a second vector corresponding to each character in the rth word group. i,j ∈D r Representatives i,j Including a word in the rth group of words output by the teacher model, including the i-th vector to the j-th vector in the first prediction result, that is, s i,j It can represent a word in the rth group of words; represent Including another word in the rth group of words output by the teacher model, including the first word in the first prediction result vector to the vector, that is can represent another word in the rth group of words, then s i,j and is a different word in the rth group; N r represents the number of words included in the rth word group; The expansion of can be found in formula (4).

[0135] Instructions i,j and The similarity between The expansion of can be found in formula (5), where s is calculated i,j and For example, the cosine similarity between i,j and The Euclidean distance, L1 distance or other methods between them are used to obtain s i,j and The similarity between them, etc., are not exhaustive here; β is a hyperparameter; The first prediction result includes the first Word formation, Representatives m,n Including the R word group output by the teacher model From the mth vector to the nth vector in the word group, then s i,j and s m,n For different groups of words in the R group, Instructions i,j With s m,n The similarity between The calculation method can be found in The calculation method of is understood, which will not be elaborated here; from formula (4), we can know that It indicates not only the similarity between different words included in the same word group in the R word groups included in the first prediction result, but also the similarity between words in different groups in the R word groups included in the first prediction result.

[0136] Using L scl The purpose of training is to make L scl Getting smaller and smaller, that is, Getting bigger and bigger, thus using L scl The purpose of training includes improving the similarity between different words included in the same word group in R, and also includes reducing the similarity between words in different groups in R. It should be understood that the examples in formulas (3) to (5) are only for the convenience of understanding this solution and are not used to limit this solution.

[0137] In an embodiment of the present application, the purpose of using the third loss term for training includes improving the similarity between different words in the same category of information. Since if the task of the teacher model is to obtain at least one category of information from the input second text, different words in the same category of information should be more similar. When different words in the same category of information are more similar, it means that the first prediction result generated by the teacher model when performing the information acquisition task on the input second text is more accurate, which can reflect that the second text input into the teacher model is a text that the teacher model is better at processing, that is, the second text is data that the teacher model is familiar with. Therefore, using the third loss term can improve the similarity between the final second text and the private training data used when training the teacher model.

[0138] The purpose of using the third loss term for training also includes reducing information from different categories. If the task of the teacher model is to obtain at least one category of information from the input second text, the words from different categories of information should be more dissimilar. When different words in different categories of information are less similar, it means that the first prediction result generated by the teacher model when performing the information acquisition task on the input second text is more accurate, which can reflect that the second text input to the teacher model is data that the teacher model is familiar with. Therefore, using the third loss term can improve the similarity between the final second text and the private training data used when training the teacher model.

[0139] Optionally, the first loss function may include a first loss term and a second loss term.

[0140] Optionally, if the task of the teacher model is to obtain at least one type of information from the input second text, the first loss function may include a first loss term and a third loss term, or the first loss function may include a second loss term and a third loss term, or the first loss function may include a first loss term, a second loss term, and a third loss term.

[0141] For a more intuitive understanding of this solution, please refer to Figure 6, which is another flow chart of the model training method provided in an embodiment of the present application. As shown in Figure 6, after obtaining the third text from the first data set, the first training device concatenates the prompt word (i.e., Summarize in Figure 6), the expression of the first prompt, and the third text to obtain the first text; the first text is input into the first machine learning model to obtain the second text output by the first machine learning model.

[0142] The first training device inputs the second text into the teacher model to obtain a first prediction result output by the teacher model, and calculates the value of the first loss term and the value of the second loss term based on the first prediction result. According to the value of the first loss term and the value of the second loss term, the function value of the first loss function is generated. The meanings of the "first loss term" and the "second loss term" can be found in the above description and will not be repeated here.

[0143] The first training device performs gradient differentiation on the function value of the first loss function and adopts the back-propagation algorithm to update the expression of the first prompt, thereby completing a training of the first prompt. It should be understood that the example in FIG6 is only for the convenience of understanding this scheme and is not used to limit this scheme.

[0144] In an embodiment of the present application, in the process of performing a second training operation on the student model using a teacher model by means of distillation, the training data in the second data set will be used. The training data in the second data set is generated by a special first machine learning model after completing the first training operation, and the first training operation is performed based on the first prediction result output by the teacher model, and the input of the teacher model is the output text of the aforementioned first machine learning model; since the teacher model is obtained after training with the private data as training data, the teacher model is familiar with the private data, and therefore, the process of the first training operation is to guide the text generated by the first machine learning model to approach the private data with the help of the ability of the teacher model, so that the training data used when training the student model by means of distillation can The first machine learning model is similar enough to the privacy data, which is conducive to improving the accuracy of the trained student model when performing natural language processing tasks; the input of the first machine learning model is the text in the existing first data set, and the first data set includes rich text. Correspondingly, the output text of the first machine learning model is also rich, thereby ensuring the richness of the training data of the student model; in addition, in the process of executing the first training operation, the parameters of the first machine learning model and the teacher model are kept unchanged, and only the first prompt is updated, which reduces the amount of parameters that need to be updated in the process of executing the first training operation, is conducive to reducing the difficulty of executing the first training operation, is conducive to reducing the computer resources consumed in the process of executing the first training operation, and is also conducive to improving the execution efficiency of the first training operation.

[0145] 2. Generation of the Second Dataset

[0146] In the embodiment of the present application, the "second data set generation stage" can also be understood as the stage of generating multiple second texts (i.e., the second data set) using the first machine learning model after the first training operation is performed; or, it can also be understood as the stage of generating training data used when performing knowledge distillation on the student model using the teacher model. Specifically, please refer to Figure 7, which is a flow chart of a method for obtaining training data provided in the embodiment of the present application. The method for obtaining training data provided in the embodiment of the present application may include:

[0147] 701. Obtain a first text, where the first text includes a third text and a trained first prompt, the third text belongs to a first data set, and the trained first prompt is obtained through a first training operation.

[0148] 702. Input the first text into the first machine learning model to obtain the second text output by the first machine learning model, wherein a second data set is used in the process of performing a second training operation on the student model using the teacher model in a distillation manner, and the second data set includes multiple second texts. When performing the first training operation using the first loss function, the parameters of the first machine learning model and the teacher model are kept unchanged, the first prompt is updated, the input of the teacher model is the output text of the machine learning model, and the first loss function is obtained based on the first prediction result output by the teacher model.

[0149] In the embodiment of the present application, the specific implementation of steps 701 and 702, the beneficial effects brought about, and the meanings of the nouns in steps 701 and 702 can all be found in the description of the embodiment corresponding to Figure 3, and will not be repeated here. The specific implementation process of the above-mentioned "first training operation" can be found in the description of the embodiment corresponding to Figure 3. It should be noted that, in one implementation, as described in the above-mentioned steps 701 and 702, the first data set includes multiple third texts, each first text includes a third text and a first prompt, and the process of executing the first training operation is to iteratively update the first prompt (which can also be understood as iteratively updating the expression of the first prompt). Then, the first text used in steps 701 and 702 includes the first prompt that has executed the first training operation and the third text obtained from the first data set.

[0150] Optionally, in another implementation, the first data set directly includes multiple first texts, and the parameters of the first machine learning model are iteratively updated during the execution of the first training operation. The first machine learning model used in steps 701 and 702 is the first machine learning model that has executed the first training operation.

[0151] 3. Execution Phase of the Second Training Operation

[0152] In the embodiments of the present application, the "second training operation execution phase" can also be understood as the process of using a teacher model to perform the second training operation on the student model using a distillation method; it can also be understood as the process of using the teacher model to perform knowledge distillation on the student model. Specifically, please refer to Figure 8, which is another flow chart of the model training method provided in the embodiments of the present application. The model training method provided in the embodiments of the present application may include:

[0153] 801. Input the second text into the teacher model and the student model respectively to obtain a second prediction result output by the teacher model and a third prediction result output by the student model.

[0154] In an embodiment of the present application, the second training device can obtain one or more second texts from the second data set, input each second text into the teacher model and the student model respectively, and obtain the second prediction result output by the teacher model after processing each second text by the teacher model, and obtain the third prediction result output by the student model after processing each second text by the student model. For the type of task performed by the "teacher model", please refer to the description in the embodiment corresponding to Figure 3 above, which will not be repeated here. The meaning of "second prediction result" is the same as that of "first prediction result", the difference is that "second prediction result" is generated by the teacher model during the execution stage of the second training operation, and "first prediction result" is generated by the teacher model during the execution stage of the first training operation. The meaning of "second prediction result" can refer to the description in the embodiment corresponding to Figure 3 above, which will not be repeated here.

[0155] The student model is also used to perform one or more natural language processing tasks. The student model and the teacher model are used to perform the same tasks. The "content indicated by the third prediction result" is the same as the "content indicated by the first prediction result". Please refer to the description in the corresponding embodiment of Figure 3 above, which will not be repeated here.

[0156] The difference between the student model and the teacher model is that the scale of the student model is smaller than that of the teacher model, that is, the storage space occupied by the student model is smaller than that of the teacher model, and the computer resources consumed when using the student model for data processing are smaller than the computer resources consumed when using the teacher model for data processing.

[0157] 802. Based on the second prediction result and the third prediction result, the student model is trained by distillation, wherein the second text is generated by the first machine learning model based on the trained first prompt, the input of the first machine learning model is the first text, the first text includes the third text and the aforementioned trained first prompt, the third text belongs to the first data set, the trained first prompt is obtained through the first training operation, when the first training operation is performed using the first loss function, the parameters of the first machine learning model and the teacher model are kept unchanged, the expression of the first prompt is updated, the input of the teacher model is the output text of the first machine learning model, and the first loss function is obtained based on the first prediction result output by the teacher model.

[0158] In the embodiment of the present application, the meanings of the multiple terms in steps 801 and 802 can be referred to the description of the embodiment corresponding to FIG3 and will not be repeated here. The specific implementation process of the above-mentioned "first training operation" can be referred to the description of the embodiment corresponding to FIG3 and will not be repeated here.

[0159] The second training device can train the student model using a distillation method based on the second prediction result, the third prediction result, and the function value of the second loss function. The second training device repeatedly performs steps 801 and 802 to iteratively train the student model until a convergence condition is met, thereby obtaining a student model that has undergone the second training operation. The convergence condition can be that the convergence condition of the second loss function is met, or that the student model has been iteratively trained a predetermined number of times.

[0160] Exemplarily, the second training device can generate a function value of a second loss function based on the second prediction result and the third prediction result, perform gradient differentiation on the function value of the second loss function, and use the backpropagation algorithm to update the parameters of the student model, thereby completing one training of the student model.

[0161] Optionally, the second loss function may include a fourth loss term, which indicates the similarity between the second prediction result output by the teacher model and the third prediction result output by the student model. The purpose of training with the fourth loss term includes improving the similarity between the second prediction result and the third prediction result, that is, the purpose of training with the fourth loss term includes reducing the degree of difference between the second prediction result and the third prediction result.

[0162] For example, the “similarity between the second prediction result and the third prediction result” can be obtained by calculating the Jensen–Shannon divergence (JSD) or relative entropy (Kullback–Leibler divergence, KLD) between the second prediction result and the third prediction result, or calculating other information between the second prediction result and the third prediction result, etc., which is not limited here.

[0163] To further understand this solution, the following example shows an example of the function used by the fourth loss term, taking the output of both the teacher model and the student model as multiple vectors:

[0164] Among them, L divg Represents the fourth loss term, N represents the second text in a batch including N second texts, represents the jth second vector included in the third prediction result output by the student model after the i-th second text among the N second texts is input into the student model, represents the jth second vector included in the second prediction result output by the teacher model after the i-th second text among the N second texts is input into the teacher model, Representative Calculation and It should be understood that the example in formula (6) is only for the convenience of understanding this solution and is not used to limit this solution.

[0165] Optionally, the second loss function may include a fifth loss term; the fifth loss term indicates the similarity between the first feature information and the second feature information, the first feature information is the feature information of the second text generated in the process of the teacher model processing the second text, and the second feature information is the feature information of the second text generated in the process of the student model processing the second text, and the purpose of training using the fifth loss term includes improving the similarity between the first feature information and the second feature information.

[0166] To further understand the present solution, an example of the function adopted by the fifth loss term is disclosed as follows: L hidd =‖(h S W h , h T )‖2; (7)

[0167] Among them, L hiidd represents the fifth loss term, ‖(h S W h , h T )‖2 represents the calculation of h S W h With hT The L2 distance between S Represents the second feature information generated by the student model, h S W h Represents a linear change of the second feature information, h T represents the first feature information generated by the teacher model. It should be understood that the example in formula (7) is only for the convenience of understanding this solution. For example, in actual situations, the calculation of h S W h With h T The L2 distance between them is replaced by calculating h S W h With h T The similarity between the first feature information and the second feature information can be calculated by L1 distance, Euclidean distance or other methods, and the example in formula (7) is not used to limit this solution.

[0168] Optionally, the second loss function may include a fourth loss term and a fifth loss term. To further understand this solution, an example of the second loss function is disclosed as follows: KD =λ1L divg +λ2L hidd ; (8)

[0169] Among them, L KD Represents the second loss function, λ1 represents L divg The weight of L hidd The weight of , that is, the weighted sum of the fourth loss term and the fifth loss term is used to obtain the second loss function. It should be understood that the example in formula (8) is only for the convenience of understanding this solution and is not used to limit this solution.

[0170] To further understand the beneficial effects of the method provided by this application, the following is an introduction to the beneficial effects of this application with experimental data. We tested the classification task and information extraction task performed by the student model, and the test results are shown in Table 1 below.

[0171] Table 1

[0172] Among them, Out-of-domain text (OOD) is a current method for performing knowledge distillation on the student model without original training data. When the task performed by the student model is the SST-2 task, that is, the student model outputs a predicted sentiment classification for the entire input text, the accuracy (ACC) is used to measure the accuracy of the sentiment classification output by the student model. The higher the accuracy of the classification value output by the student model, the higher the ACC value.

[0173] When the task performed by the student model is a CoLA task, that is, the student model outputs a predicted classification value for the entire input text. The aforementioned predicted classification value indicates whether the input text meets the grammatical requirements. The Matthews correlation coefficient (MCC) is used to measure the accuracy of the classification value output by the student model. The higher the accuracy of the classification value output by the student model, the higher the MCC value.

[0174] When the task performed by the student model is information extraction, that is, the student model identifies at least one type of information with specific meaning from the entire input text, the F1 score is used to measure the accuracy of the prediction results output by the student model. The higher the accuracy of the prediction results output by the student model, the higher the F1 score value.

[0175] It can be seen from the data shown in Table 1 that compared with the prediction results output by the trained student model obtained by using OOD, the prediction results output by the trained student model obtained by the method provided in this application are more accurate.

[0176] We also conducted more comparative experiments on the SST-2 task performed by the student model. The following Table 2 shows the experimental results.

[0177] Table 2

[0178] Unlabeled distillation (Unlabel KD) and unlabeled distillation with adversarial training (Unlabel KD+Adv) are two existing methods for performing knowledge distillation on student models without training data. When the task performed by the student model is the SST-2 task, ACC is still used to measure the accuracy of the sentiment classification output by the student model. The higher the accuracy of the classification value output by the student model, the higher the value of ACC. It can be seen from the data shown in Table 2 that the prediction results output by the trained student model obtained by the method provided in this application have the highest accuracy.

[0179] On the basis of the embodiments corresponding to Figures 1 to 8, in order to better implement the above-mentioned scheme of the embodiment of the present application, the following also provides related equipment for implementing the above-mentioned scheme. Please refer to Figure 9 in detail, which is a structural diagram of the training device of the model provided in the embodiment of the present application. The training device 900 of the model includes: an input module 901, which is used to input the first text into the machine learning model to obtain the second text output by the machine learning model, the first text includes a third text and a first prompt (prompt), and the third text belongs to the first data set; the input module 901 is also used to input the second text into the teacher model to obtain the first prediction result output by the teacher model; the training module 902 is used to perform a first training operation based on the first prediction result until the convergence condition of the loss function is met. In the process of performing the first training operation, the parameters of the machine learning model and the teacher model are kept unchanged, and the first prompt is updated; wherein, the first prompt after training is used to obtain the second data set, and the training data in the second data set will be used in the process of performing the second training operation on the student model using the teacher model in a distillation manner.

[0180] Optionally, the loss function includes a first loss term, the first prediction result includes at least one vector corresponding one-to-one to at least one word, and the purpose of training using the first loss term includes making each vector closer to a one-hot form.

[0181] Optionally, the loss function includes a second loss term, which indicates the information entropy of the multiple first prediction results.

[0182] Optionally, the first prompt includes a prompt word and a second prompt, the prompt word means a summary, and the second prompt is updated during the execution of the first training operation.

[0183] Optionally, the machine learning model includes a normalized exponential (softmax) layer. In the process of the machine learning model processing the first text, the data that the softmax layer in the machine learning model needs to process is the first data obtained based on the first text; wherein, the process of using the softmax layer to process the first data includes: adding disturbance to the first data to obtain updated first data, and processing the updated first data using the softmax function.

[0184] Optionally, the task of the teacher model is to obtain at least one category of information from the second text, the first prediction result includes at least one group of words corresponding one-to-one to at least one category, the loss function includes a third loss term, and the purpose of training using the third loss term includes improving the similarity between different words included in the same group of words in at least one group of words.

[0185] Optionally, the purpose of using the third loss term for training also includes reducing the similarity between words in different groups in at least one group of words.

[0186] It should be noted that the information interaction, execution process, etc. between the modules / units in the model training device 900 are based on the same concept as the various method embodiments corresponding to Figures 3 to 6 in this application. For specific contents, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0187] Please continue to refer to Figure 10. Figure 10 is a structural diagram of a training data acquisition device provided in an embodiment of the present application. The training data acquisition device 1000 includes: an acquisition module 1001, used to acquire a first text, the first text includes a third text and a first prompt after training, the third text belongs to the first data set, and the first prompt after training is obtained through a first training operation; an input module 1002, used to input the first text into the machine learning model to obtain a second text output by the machine learning model, wherein, when using a distillation method, the second data set will be used in the process of performing the second training operation on the student model using the teacher model, and the second data set includes multiple second texts; when using the loss function to perform the first training operation, the parameters of the machine learning model and the teacher model are kept unchanged, the first prompt is updated, the input of the teacher model is the output text of the machine learning model, and the loss function is obtained based on the first prediction result output by the teacher model.

[0188] Optionally, the first prediction result includes at least one vector corresponding to at least one word, the loss function includes a first loss term, and the purpose of training using the first loss term includes making each vector closer to a one-hot form.

[0189] Optionally, the loss function includes a second loss term, which indicates the information entropy of the multiple first prediction results.

[0190] It should be noted that the information interaction, execution process, etc. between the modules / units in the training data acquisition device 1000 are based on the same concept as the various method embodiments corresponding to Figure 7 in this application. For specific contents, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0191] Please continue to refer to FIG11 , which is another structural diagram of a model training device provided in an embodiment of the present application. The model training device 1100 includes: an input module 1101 for inputting a second text into a teacher model and a student model, respectively, to obtain a second prediction result output by the teacher model and a third prediction result output by the student model;

[0192] The training module 1102 is used to train the student model by distillation according to the second prediction result and the third prediction result, wherein the second text is generated by the first machine learning model based on the first prompt after training, the input of the first machine learning model is the first text, the first text includes the third text and the aforementioned first prompt after training, the third text belongs to the first data set, the first prompt after training is obtained through the first training operation, when the first training operation is performed using the first loss function, the parameters of the first machine learning model and the teacher model are kept unchanged, and the expression of the first prompt is updated, the input of the teacher model is the output text of the first machine learning model, and the first loss function is obtained based on the first prediction result output by the teacher model.

[0193] It should be noted that the information interaction, execution process, etc. between the modules / units in the model training device 1100 are based on the same concept as the various method embodiments corresponding to Figure 8 in this application. For specific contents, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0194] Next, a device provided in an embodiment of the present application is introduced. Please refer to Figure 12. Figure 12 is a structural diagram of the device provided in an embodiment of the present application. The device 1200 can be used to execute the steps performed by the first training device in Figures 3 to 6, or the device 1200 can be used to execute the steps performed by the execution device in Figure 7, or the device 1200 can be used to execute the steps performed by the second training device in Figure 8.

[0195] Specifically, the device 1200 is implemented by one or more servers. The device 1200 may vary greatly due to different configurations or performance. It may include one or more central processing units (CPUs) 1222 (for example, one or more processors) and memory 1232, and one or more storage media 1230 (for example, one or more mass storage devices) storing applications 1242 or data 1244. Among them, the memory 1232 and the storage medium 1230 can be temporary storage or permanent storage. The program stored in the storage medium 1230 may include one or more modules (not shown in the figure), each module may include a series of instruction operations in the training device. Furthermore, the central processing unit 1222 can be configured to communicate with the storage medium 1230 to execute a series of instruction operations in the storage medium 1230 on the device 1200.

[0196] The device 1200 may also include one or more power supplies 1226, one or more wired or wireless network interfaces 1250, one or more input and output interfaces 1258, and / or one or more operating systems 1241, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0197] In one embodiment of the present application, the central processing unit 1222 is configured to execute the model training method executed by the first training device in the embodiments corresponding to Figures 3 to 6. The specific manner in which the central processing unit 1222 executes the aforementioned steps is based on the same concept as the various method embodiments corresponding to Figures 3 to 6 of the present application, and the technical effects achieved are the same as the various method embodiments corresponding to Figures 3 to 6 of the present application. For details, please refer to the description of the method embodiments shown above in the present application, and will not be repeated here.

[0198] In another embodiment, the central processing unit 1222 is configured to execute the method for acquiring training data executed by the execution device in the embodiment corresponding to FIG7 . The specific manner in which the central processing unit 1222 executes the aforementioned steps is based on the same concept as the various method embodiments corresponding to FIG7 in this application, and the technical effects brought about are the same as the various method embodiments corresponding to FIG7 in this application. For details, please refer to the description of the method embodiments shown above in this application, and will not be repeated here.

[0199] In another embodiment, the central processing unit 1222 is configured to execute the model training method executed by the second training device in the embodiment corresponding to FIG8 . The specific manner in which the central processing unit 1222 executes the aforementioned steps is based on the same concept as the various method embodiments corresponding to FIG8 in this application, and the technical effects achieved are the same as the various method embodiments corresponding to FIG8 in this application. For details, please refer to the description of the method embodiments shown above in this application, and will not be repeated here.

[0200] A computer-readable storage medium is also provided in an embodiment of the present application, which stores a program for signal processing. When the computer-readable storage medium is run on a computer, it enables the computer to execute the steps executed by the first training device in the method described in the embodiments shown in Figures 3 to 6 above, or enables the computer to execute the steps executed by the execution device in the method described in the embodiment shown in Figure 7 above, or enables the computer to execute the steps executed by the second training device in the method described in the embodiment shown in Figure 8 above.

[0201] Also provided in an embodiment of the present application is a computer program product, which, when executed on a computer, enables the computer to execute the steps executed by the first training device in the method described in the embodiments shown in Figures 3 to 6 above, or enables the computer to execute the steps executed by the execution device in the method described in the embodiment shown in Figure 7 above, or enables the computer to execute the steps executed by the second training device in the method described in the embodiment shown in Figure 8 above.

[0202] The execution device, the first training device or the second training device provided in the embodiments of the present application can specifically be a chip, and the chip includes: a processing unit and a communication unit, the processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, a pin or a circuit, etc. The processing unit can execute the computer execution instructions stored in the storage unit, so that the chip executes the training method of the model described in the embodiments shown in Figures 3 to 6 above, or so that the chip executes the method for obtaining training data described in the embodiment shown in Figure 7 above, or so that the chip executes the training method of the model described in the embodiment shown in Figure 8 above. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit can also be a storage unit located outside the chip in the wireless access device end, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0203] Specifically, see Figure 13 , which is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip may be a neural network processor (NPU) 130. NPU 130 is mounted on a host CPU (host CPU) as a coprocessor, with tasks assigned by the host CPU. The core of the NPU is arithmetic circuit 1303, which is controlled by controller 1304 to extract matrix data from memory and perform multiplication operations.

[0204] In some implementations, the arithmetic circuit 1303 includes multiple processing units (PEs). In some implementations, the arithmetic circuit 1303 is a two-dimensional systolic array. The arithmetic circuit 1303 can also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1303 is a general-purpose matrix processor.

[0205] For example, assume there are input matrix A, weight matrix B, and output matrix C. The computation circuit retrieves the corresponding data of matrix B from weight memory 1302 and caches it on each PE in the computation circuit. The computation circuit then retrieves the data of matrix A from input memory 1301 and performs a matrix operation on it with matrix B. The partial or final matrix result is stored in accumulator 1308.

[0206] Unified memory 1306 is used to store input and output data. Weight data is directly transferred to weight memory 1302 through the Direct Memory Access Controller (DMAC) 1305. Input data is also transferred to unified memory 1306 through the DMAC.

[0207] BIU stands for Bus Interface Unit 1310 , which is used for interaction between the AXI bus, DMAC, and instruction fetch buffer (IFB) 1309 .

[0208] The bus interface unit 1310 (BIU) is used for the instruction fetch memory 1309 to obtain instructions from the external memory, and is also used for the storage unit access controller 1305 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0209] DMAC is mainly used to move input data in the external memory DDR to the unified memory 1306 or to move weight data to the weight memory 1302 or to move input data to the input memory 1301.

[0210] The vector calculation unit 1307 includes multiple operation processing units. When necessary, it further processes the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as batch normalization, pixel-level summation, and upsampling of feature planes.

[0211] In some implementations, the vector calculation unit 1307 can store the processed output vector to the unified memory 1306. For example, the vector calculation unit 1307 can apply a linear function and / or a nonlinear function to the output of the operation circuit 1303, such as linear interpolation of the feature plane extracted by the convolution layer, or accumulate a vector of values ​​to generate an activation value. In some implementations, the vector calculation unit 1307 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1303, such as for use in subsequent layers in a neural network.

[0212] An instruction fetch buffer 1309 connected to the controller 1304 is used to store instructions used by the controller 1304;

[0213] Unified memory 1306, input memory 1301, weight memory 1302, and instruction fetch memory 1309 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0214] Among them, the operations of each layer in the above-mentioned first machine learning model, teacher model or student model can be performed by the operation circuit 1303 or the vector calculation unit 1307.

[0215] The processor mentioned in any of the above places can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned first aspect method.

[0216] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0217] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0218] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0219] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, computer, training device or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive (SSD)).

Claims

1. A model training method, characterized in that: The method comprises: Inputting a first text into a machine learning model to obtain a second text output by the machine learning model, wherein the first text includes a third text and a first prompt, and the third text belongs to a first data set; Inputting the second text into the teacher model to obtain a first prediction result output by the teacher model; Performing a first training operation based on the first prediction result until a convergence condition of the loss function is satisfied, and in the process of performing the first training operation, keeping parameters of the machine learning model and the teacher model unchanged, and updating the first prompt; The first prompt after training is used to obtain a second data set, and the training data in the second data set is used in the process of performing a second training operation on the student model using the teacher model in a distillation manner.

2. The method according to claim 1, characterized in that The loss function includes a first loss term, the first prediction result includes at least one vector corresponding to at least one word, and the purpose of training using the first loss term includes making each of the vectors closer to a one-hot form.

3. The method according to claim 1 or 2, characterized in that: The loss function includes a second loss term, which indicates the information entropy of multiple first prediction results.

4. The method according to claim 1 or 2, characterized in that: The first prompt includes a prompt word and a second prompt, the prompt word means a summary, and the second prompt is updated during the execution of the first training operation.

5. The method according to claim 1 or 2, characterized in that: The machine learning model includes a normalized exponential softmax layer, and in the process of the machine learning model processing the first text, the data that needs to be processed by the softmax layer in the machine learning model is the first data obtained based on the first text; The process of processing the first data using the softmax layer includes: adding disturbance to the first data to obtain updated first data, and processing the updated first data using a softmax function.

6. The method according to claim 1 or 2, characterized in that: The task of the teacher model is to obtain at least one category of information from the second text, the first prediction result includes at least one group of words corresponding one-to-one to the at least one category, the loss function includes a third loss term, and the purpose of training with the third loss term includes improving the similarity between different words included in the same group of words in the at least one group of words.

7. The method according to claim 6, characterized in that The purpose of using the third loss term for training also includes reducing the similarity between words in different groups of the at least one group of words.

8. A method for obtaining training data, characterized in that: The method comprises: Acquire a first text, the first text including a third text and a trained first prompt, the third text belongs to a first data set, and the trained first prompt is obtained through a first training operation; Inputting the first text into a machine learning model to obtain a second text output by the machine learning model, wherein a second data set is used in the process of performing a second training operation on a student model using a teacher model in a distillation manner, and the second data set includes a plurality of the second texts; When using the loss function to perform the first training operation, the parameters of the machine learning model and the teacher model are kept unchanged, the first prompt is updated, the input of the teacher model is the output text of the machine learning model, and the loss function is obtained based on the first prediction result output by the teacher model.

9. The method according to claim 8, characterized in that The first prediction result includes at least one vector corresponding to at least one word, the loss function includes a first loss term, and the purpose of training using the first loss term includes making each of the vectors closer to a one-hot form.

10. The method according to claim 8 or 9, characterized in that: The loss function includes a second loss term, which indicates the information entropy of multiple first prediction results.

11. A model training device, characterized in that: The device comprises: An input module, configured to input a first text into a machine learning model to obtain a second text output by the machine learning model, wherein the first text includes a third text and a first prompt, and the third text belongs to a first data set; The input module is further used to input the second text into the teacher model to obtain a first prediction result output by the teacher model; A training module, configured to perform a first training operation based on the first prediction result until a convergence condition of a loss function is satisfied, and in the process of performing the first training operation, keep parameters of the machine learning model and the teacher model unchanged, and update the first prompt; The first prompt after training is used to obtain a second data set, and the training data in the second data set is used in the process of performing a second training operation on the student model using the teacher model in a distillation manner.

12. The device according to claim 11, characterized in that The loss function includes a first loss term, the first prediction result includes at least one vector corresponding to at least one word, and the purpose of training using the first loss term includes making each of the vectors closer to a one-hot form.

13. The device according to claim 11 or 12, characterized in that The loss function includes a second loss term, which indicates the information entropy of multiple first prediction results.

14. The device according to claim 11 or 12, characterized in that The first prompt includes a prompt word and a second prompt, the prompt word means a summary, and the second prompt is updated during the execution of the first training operation.

15. The device according to claim 11 or 12, characterized in that The machine learning model includes a normalized exponential softmax layer, and in the process of the machine learning model processing the first text, the data that needs to be processed by the softmax layer in the machine learning model is the first data obtained based on the first text; The process of processing the first data using the softmax layer includes: adding disturbance to the first data to obtain updated first data, and processing the updated first data using a softmax function.

16. The device according to claim 11 or 12, characterized in that The task of the teacher model is to obtain at least one category of information from the second text, the first prediction result includes at least one group of words corresponding one-to-one to the at least one category, the loss function includes a third loss term, and the purpose of training with the third loss term includes improving the similarity between different words included in the same group of words in the at least one group of words.

17. The device according to claim 16, characterized in that The purpose of using the third loss term for training also includes reducing the similarity between words in different groups of the at least one group of words.

18. A device for acquiring training data, characterized in that: The device comprises: An acquisition module, configured to acquire a first text, wherein the first text includes a third text and a trained first prompt, wherein the third text belongs to a first data set, and the trained first prompt is obtained through a first training operation; an input module, configured to input the first text into a machine learning model to obtain a second text output by the machine learning model, wherein a second data set is used in the process of performing a second training operation on a student model using a teacher model in a distillation manner, and the second data set includes a plurality of the second texts; When using the loss function to perform the first training operation, the parameters of the machine learning model and the teacher model are kept unchanged, the first prompt is updated, the input of the teacher model is the output text of the machine learning model, and the loss function is obtained based on the first prediction result output by the teacher model.

19. The device according to claim 18, characterized in that The first prediction result includes at least one vector corresponding to at least one word, the loss function includes a first loss term, and the purpose of training using the first loss term includes making each of the vectors closer to a one-hot form.

20. The device according to claim 18 or 19, characterized in that The loss function includes a second loss term, which indicates the information entropy of multiple first prediction results.

21. A training device, characterized in that comprising a processor and a memory, the processor being coupled to the memory, The memory is used to store programs; The processor is used to execute the program in the memory so that the training device performs the method as claimed in any one of claims 1 to 7.

22. An execution device, characterized in that: comprising a processor and a memory, the processor being coupled to the memory, The memory is used to store programs; The processor is configured to execute the program in the memory so that the execution device executes the method according to any one of claims 8 to 10.

23. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 10.

24. A computer program product, characterized in that The computer program product comprises a program, and when the program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 10 .

Images