A model training method, a training data acquisition method, and related devices

Abstract

This application discloses a model training method, a training data acquisition method, and related equipment. The method can be used in the text processing field of artificial intelligence. The method includes: inputting a first text, including a third text and a first prompt, into a machine learning model to obtain a second text output by the machine learning model, wherein the third text belongs to a first dataset; inputting the second text into a teacher model to obtain a first prediction result output by the teacher model; performing a first training operation based on the first prediction result to update the first prompt; using the teacher model to perform knowledge distillation on a student model, the training data is obtained based on the trained first prompt; through the aforementioned scheme, by leveraging the capabilities of the teacher model to guide the text generated by the machine learning model to approximate the privacy data, the training data used for knowledge distillation on the student model is similar to the privacy data, thereby improving the accuracy of the trained student model in performing natural language processing tasks.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method for training a model, a method for acquiring training data, and related equipment. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0003] With the development of artificial intelligence technology, the number of machine learning models used to process text is increasing. Compression and acceleration of machine learning models are necessary, and knowledge distillation is an important means of model compression and acceleration.

[0004] However, in some fields, the training data used to train the teacher model is private data. Therefore, when using the teacher model to perform knowledge distillation on the student model, it is impossible to obtain the training data used by the student model during the training phase. Thus, a solution for obtaining the training data of the student model is urgently needed. Summary of the Invention

[0005] This application provides a model training method, a training data acquisition method, and related equipment. Since the teacher model is obtained by training on private data, it is familiar with the private data. Therefore, the first training operation guides the text generated by the machine learning model to approximate the private data by leveraging the teacher model's capabilities. This allows the training data used when training the student model using distillation to be similar to the private data, which is beneficial for improving the accuracy of the trained student model in performing natural language processing tasks. During the execution of the first training operation, only the first prompt is updated, reducing the number of parameters that need to be updated during the execution of the first training operation. This reduces the execution difficulty of the first training operation, reduces the computer resources consumed during the execution of the first training operation, and also improves the execution efficiency of the first training operation.

[0006] To address the aforementioned technical problems, the embodiments of this application provide the following technical solutions:

[0007] In a first aspect, embodiments of this application provide a model training method applicable to text processing within the field of artificial intelligence. The method includes: a first training device inputting first text into a first machine learning model to obtain a first machine learning model comprising a third text and a first prompt, the third text belonging to a first dataset. The first training device inputting second text into a teacher model, processing the second text through the teacher model to obtain a first prediction result output by the teacher model; wherein the teacher model is a first machine learning model that has undergone training operations. The first training device performs a first training operation based on the first prediction result until the convergence condition of a first loss function is met. During the execution of the first training operation, the parameters of the first machine learning model and the teacher model remain unchanged, and the expression of the first prompt is iteratively updated. The trained first prompt is used to obtain a second dataset. When using a distillation method, the training data in the second dataset is used during the second training operation performed by the teacher model on the student model. "Training data in the second dataset" can be understood as multiple texts included in the second dataset.

[0008] In this implementation, during the second training operation of the student model using the distillation method, training data from the second dataset is used. This second dataset is generated by a dedicated first machine learning model after the first training operation, which is based on the first prediction result output by the teacher model. The input to the teacher model is the output text of the aforementioned first machine learning model. Since the teacher model is trained using private data, it is familiar with the private data. Therefore, the first training operation leverages the teacher model's capabilities to guide the text generated by the first machine learning model to approximate the private data. Thus, the training data used when training the student model using the distillation method can... Similar to privacy data, this approach helps improve the accuracy of the trained student model when performing natural language processing tasks. The input to the first machine learning model is text from an existing first dataset, which contains rich text. Correspondingly, the output text of the first machine learning model is also rich, thus ensuring the richness of the training data for the student model. Furthermore, during the execution of the first training operation, the parameters of the first machine learning model and the teacher model remain unchanged, and only the first prompt is updated. This reduces the number of parameters that need to be updated during the execution of the first training operation, which helps to reduce the execution difficulty of the first training operation, reduce the computer resources consumed during the execution of the first training operation, and also helps to improve the execution efficiency of the first training operation.

[0009] In one possible implementation, the first loss function includes a first loss term, and the first prediction result includes at least one vector corresponding one-to-one with at least one character; optionally, if the output of the teacher model is text containing at least one character, then the first prediction result includes at least one vector corresponding one-to-one with the aforementioned at least one character. The purpose of training using the first loss term includes making each vector in the first prediction result closer to a one-hot form.

[0010] In this implementation, the closer each vector in the first prediction result is to the one-hot form, the more confident the teacher model is in the output of the first prediction result. This means that the second text input into the teacher model is data that the teacher model is familiar with, and that the second text is more similar to the privacy data used when training the teacher model. Using the first loss term helps to improve the similarity between the final second text (i.e., the training data used when performing knowledge distillation on the student model) and the privacy data used when training the teacher model. In addition, when the output of the teacher model is text, that is, when the first prediction result includes multiple vectors that correspond one-to-one with at least one character in the text, the purpose of training with the first loss term is to make each vector in the first prediction result close to the one-hot form. In other words, the first loss term can be compatible with the case where the output of the teacher model is text, which greatly expands the application scenarios of this scheme.

[0011] In one possible implementation, the first loss function includes a second loss term, which indicates the information entropy of multiple first prediction results. The purpose of training using the second loss term includes improving the information entropy of the multiple first prediction results. The information entropy of the multiple first prediction results reflects the degree of disorder of the information system composed of the multiple first prediction results. The greater the information entropy of the multiple first prediction results, the greater the disorder of the information system composed of the multiple first prediction results, that is, the more diverse the multiple first prediction results are.

[0012] In this implementation, the greater the information entropy of the multiple first prediction results, the more diverse the multiple first prediction results are. Therefore, for the teacher model, the multiple second texts corresponding one-to-one with the aforementioned multiple first prediction results are diverse, which is beneficial to improving the diversity of the obtained multiple second texts. In addition, the method of using a second loss term to improve the diversity of multiple second texts avoids the coupling between the "execution process of the first training operation" and the "generation process of the second text used in the training phase of the student model". That is, the first training operation does not need to be executed each time the "generation process of the second text used in the training phase of the student model" is executed, which reduces the difficulty of the "generation process of the second text used in the training phase of the student model" and also improves the flexibility when executing the "generation process of the second text used in the training phase of the student model".

[0013] In one possible implementation, the first prompt includes a prompt word and a second prompt. The prompt word is defined as a summary. During the first training operation, the prompt word may not be updated; only the expression of the second prompt may be updated. For example, the prompt word could be "summarize," "conclude," or other words such as "summarize," "inductively," "summarize," or other similar terms.

[0014] In this implementation, the first prompt is designed to include two parts: a prompt word and a second prompt. The prompt word is meant to summarize, and its function is to prompt the first machine learning model to summarize the information of the second prompt and the third text. This means that there is a certain degree of correlation between the second prompt plus the third text and the second text output by the first machine learning model. Since the third text comes from an existing dataset, it is rich in content. By adopting the aforementioned scheme, it is beneficial to retain the richness of the third text among the multiple second texts obtained in the end.

[0015] In one possible implementation, the first machine learning model includes a normalized exponent (softmax) layer. During the processing of the first text by the first machine learning model, the softmax layer processes one or more pieces of first data derived from the first text, each piece of first data corresponding to a character in the second text. The process of processing each piece of first data using the softmax layer may include: adding a perturbation to each piece of first data to obtain updated first data, and then processing each updated piece of first data using the softmax function. Optionally, the softmax layer in the first machine learning model can be the last neural network layer of the first machine learning model; that is, the processing result output by the softmax layer after processing each updated piece of first data can be used to indicate each character in the second text.

[0016] In this implementation, compared to the original first data, the updated first data obtained by adding perturbation to the first data is more diverse. Correspondingly, compared to the processing result of the first data obtained by directly processing the first data through the softmax function, the processing result of the updated first data obtained by processing the updated first data through the softmax function is more diverse. The softmax layer in the first machine learning model is often the last neural network layer of the first machine learning model. That is, the processing result obtained through the softmax layer indicates the second text that can be output by the entire first machine learning model. "The processing result of the updated first data is more diverse" means that the second text output by the entire first machine learning model is more diverse. In other words, adding perturbation to the first data that needs to be processed by the softmax layer helps to improve the diversity of the second text output by the machine learning model.

[0017] In one possible implementation, the teacher model's task is to extract at least one class of information from the second text. The first prediction result includes at least one set of words that correspond one-to-one with the at least one class. The first loss function includes a third loss term, which can indicate the similarity between different words included in the same set of words in the at least one set of words. The purpose of training using the third loss term includes improving the similarity between different words included in the same set of words in the at least one set of words.

[0018] In this implementation, the purpose of using the third loss term for training includes improving the similarity between different words in the same type of information. Since the task of the teacher model is to obtain at least one type of information from the input second text, different words in the same type of information should be more similar. When different words in the same type of information are more similar, it means that the first prediction result generated by the teacher model when performing the information acquisition task on the input second text is more accurate. It can reflect that the second text input into the teacher model is the text that the teacher model is better at processing, that is, the second text is data that the teacher model is familiar with. Therefore, using the third loss term can improve the similarity between the final second text and the privacy training data used when training the teacher model.

[0019] In one possible implementation, if the task of the teacher model is to obtain at least two types of information from the second text, that is, the first prediction result includes at least two sets of words that correspond one-to-one with the aforementioned at least two types, the third loss term can also indicate the similarity between different sets of words in the at least two sets of words included in the first prediction result. The purpose of training with the third loss term also includes reducing the similarity between different sets of words in at least one set of words.

[0020] In this implementation, the purpose of using the third loss term for training also includes reducing information from different categories. Since the task of the teacher model is to obtain at least one type of information from the input second text, the words from different categories of information should be less similar. When the words from different categories of information are less similar, it means that the first prediction result generated by the teacher model when performing the information acquisition task on the input second text is more accurate, and it can reflect that the second text input to the teacher model is data that the teacher model is familiar with. Therefore, using the third loss term can improve the similarity between the final second text and the privacy-preserving training data used when training the teacher model.

[0021] In one possible implementation, both the teacher model and the student model can be used to perform natural language processing tasks. For example, the task types of both the teacher model and the student model can be any of the following: classifying text, extracting at least one type of information from text, performing sequence labeling on text, inferring sentence relationships between two texts, or performing generative tasks based on text, etc.

[0022] Secondly, embodiments of this application provide a method for acquiring training data, applicable to text processing within the field of artificial intelligence. The method includes: an execution device acquiring first text, which includes a third text and a trained first prompt. The third text belongs to a first dataset, and the trained first prompt is obtained through a first training operation. The first text is then input into a first machine learning model to obtain a second text output by the first machine learning model. Specifically, a second dataset is used during the second training operation on the student model using a teacher model, and this second dataset includes multiple second texts. When performing the first training operation using a first loss function, the parameters of the machine learning model and the teacher model remain unchanged, while the expression of the first prompt is updated. The input to the teacher model is the output text of the machine learning model, and the loss function is obtained based on the first prediction result output by the teacher model.

[0023] The meanings of the terms used in the second aspect of this application, as well as the beneficial effects brought about by the second aspect, can be found in the first aspect, and will not be repeated here.

[0024] Thirdly, embodiments of this application provide a model training method that can be used in the text processing field of artificial intelligence. The method includes: a second training device inputting a second text into a teacher model and a student model respectively, obtaining a second prediction result output by the teacher model and a third prediction result output by the student model; the second training device training the student model using a distillation method based on the second and third prediction results; wherein the second text is generated by a first machine learning model based on a trained first prompt, the input of the first machine learning model is the first text, the first text includes the third text and the aforementioned trained first prompt, the third text belongs to a first dataset, the trained first prompt is obtained through a first training operation, when performing the first training operation using a first loss function, the parameters of the first machine learning model and the teacher model remain unchanged, the expression of the first prompt is updated, the input of the teacher model is the output text of the first machine learning model, and the first loss function is obtained based on the first prediction result output by the teacher model.

[0025] The meanings of the terms in the third aspect of this application and the beneficial effects brought about by the third aspect can be found in the first aspect, and will not be repeated here.

[0026] Fourthly, embodiments of this application provide a model training apparatus that can be used in the text processing field of artificial intelligence. The model training apparatus includes: an input module for inputting first text into a first machine learning model to obtain second text output by the first machine learning model, the first text including a third text and a first prompt, the third text belonging to a first dataset; the input module is also used for inputting the second text into a teacher model to obtain a first prediction result output by the teacher model; a training module for performing a first training operation based on the first prediction result until the convergence condition of a first loss function is met, during the execution of the first training operation, keeping the parameters of the first machine learning model and the teacher model unchanged, and updating the first prompt; wherein, the trained first prompt is used to obtain a second dataset, and the training data in the second dataset is used during the second training operation performed by the teacher model on the student model using a distillation method.

[0027] In the fourth aspect of this application, the training device for the model is also used to perform the steps executed by the first training device in the first aspect. The specific implementation of the steps in the fourth aspect of this application, the meaning of each term, and the beneficial effects thereon can all be found in the first aspect, and will not be repeated here.

[0028] Fifthly, embodiments of this application provide a training data acquisition device, which can be used in the text processing field of artificial intelligence. The training data acquisition device includes: an acquisition module for acquiring first text, the first text including third text and a trained first prompt, the third text belonging to a first dataset, and the trained first prompt obtained through a first training operation; an input module for inputting the first text into a first machine learning model to obtain a second text output by the first machine learning model, wherein a second dataset is used during the second training operation of the student model using a distillation method, the second dataset including multiple second texts; when performing the first training operation using a first loss function, the parameters of the first machine learning model and the teacher model remain unchanged, the first prompt is updated, the input of the teacher model is the output text of the first machine learning model, and the first loss function is obtained based on the first prediction result output by the teacher model.

[0029] The meanings of the terms in the fifth aspect of this application and the beneficial effects brought about by the fifth aspect can be found in the first aspect, and will not be repeated here.

[0030] Sixthly, embodiments of this application provide a model training apparatus that can be used in the text processing field of artificial intelligence. The model training apparatus includes: an input module for inputting a second text into a teacher model and a student model respectively, to obtain a second prediction result output by the teacher model and a third prediction result output by the student model; and a training module for training the student model using a distillation method based on the second and third prediction results. The second text is generated by a first machine learning model based on a trained first prompt. The input to the first machine learning model is the first text, which includes the third text and the aforementioned trained first prompt. The third text belongs to a first dataset. The trained first prompt is obtained through a first training operation. When performing the first training operation using a first loss function, the parameters of the first machine learning model and the teacher model remain unchanged, and the expression of the first prompt is updated. The input to the teacher model is the output text of the first machine learning model, and the first loss function is obtained based on the first prediction result output by the teacher model.

[0031] The meanings of the terms in the sixth aspect of this application and the beneficial effects brought about by the sixth aspect can be found in the first aspect, and will not be repeated here.

[0032] In a seventh aspect, embodiments of this application provide a training device, including a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store a program, and the processor is used to execute the program in the memory, causing the training device to perform the training method of the model described in the first or third aspect above.

[0033] Eighthly, embodiments of this application provide an execution device, including a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store a program, and the processor is used to execute the program in the memory, causing the execution device to perform the training data acquisition method described in the second aspect above.

[0034] Ninthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the methods described in the first, second, or third aspects above.

[0035] In a tenth aspect, embodiments of this application provide a computer program product, which includes a program that, when run on a computer, causes the computer to perform the methods described in the first, second, or third aspects above.

[0036] Eleventhly, this application provides a chip system including a processor for supporting the implementation of the functions involved in the foregoing aspects, such as transmitting or processing data and / or information involved in the foregoing methods. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for the terminal device or communication device. This chip system may be composed of chips or may include chips and other discrete devices. Attached Figure Description

[0037] Figure 1 A schematic diagram of the main framework of artificial intelligence provided in the embodiments of this application;

[0038] Figure 2 A system architecture diagram of the data processing system provided in this application embodiment;

[0039] Figure 3 A schematic flowchart illustrating a training method for a model provided in an embodiment of this application;

[0040] Figure 4 A schematic diagram of the first text provided in the embodiments of this application;

[0041] Figure 5 Several schematic diagrams of one-hot vectors provided for embodiments of this application;

[0042] Figure 6 Another flowchart illustrating the training method for the model provided in this application embodiment;

[0043] Figure 7 A flowchart illustrating a method for acquiring training data provided in an embodiment of this application;

[0044] Figure 8 Another flowchart illustrating the training method for the model provided in this application embodiment;

[0045] Figure 9 A schematic diagram of a training device for a model provided in an embodiment of this application;

[0046] Figure 10 A schematic diagram of a training data acquisition device provided in an embodiment of this application;

[0047] Figure 11 Another schematic diagram of the training device for the model provided in the embodiments of this application;

[0048] Figure 12 A schematic diagram of the device provided in an embodiment of this application;

[0049] Figure 13 This is a schematic diagram of a chip structure provided in an embodiment of this application. Detailed Implementation

[0050] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0051] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0052] First, the overall workflow of the artificial intelligence system is described; please refer to [link / reference]. Figure 1 , Figure 1The diagram illustrates a structural framework for artificial intelligence (AI). The framework is further elaborated below along two dimensions: the "Intelligent Information Chain" (horizontal axis) and the "IT Value Chain" (vertical axis). The "Intelligent Information Chain" reflects a series of processes from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom." The "IT Value Chain" reflects the value that AI brings to the information technology industry, from the underlying infrastructure of human intelligence and information (provided and processed through technological means) to the industrial ecosystem of the system.

[0053] (1) Infrastructure

[0054] The infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. Communication with the outside world is achieved through sensors; computing power is provided by intelligent chips, which can specifically employ hardware acceleration chips such as central processing units (CPUs), embedded neural network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). The basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0055] (2) Data

[0056] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0057] (3) Data processing

[0058] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0059] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.

[0060] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0061] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0062] (4) General ability

[0063] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0064] (5) Smart Products and Industry Applications

[0065] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They encapsulate overall artificial intelligence solutions, productize intelligent information decision-making, and realize practical applications. Their application areas mainly include: intelligent terminals, intelligent manufacturing, intelligent transportation, smart homes, intelligent healthcare, intelligent security, autonomous driving, and smart cities.

[0066] This application can be applied to various application areas of artificial intelligence technology. For example, it can be used to obtain the training data used by machine learning models in various application areas during the training phase. Optionally, the training method used in the aforementioned training phase is knowledge distillation, which refers to a model compression method that transfers the "knowledge" learned by the teacher model to a simple student model.

[0067] For example, the training data described above is specifically text, meaning that both the teacher model and the student model are used to perform natural language processing (NLP) tasks. Natural language processing is the processing of human language; it is the process of systematically analyzing, understanding, and extracting information from text data using a machine learning model. Machine learning models can be used to perform NLP tasks in application areas such as smart terminals, smart homes, and autonomous driving.

[0068] In all the aforementioned application areas, by using the machine learning models described above, we can manage very large amounts of text data, or perform a large number of automated tasks, and solve a wide variety of problems, such as automatic summarization, machine translation (MT), name entity recognition (NER), relation extraction (RE), information extraction (IE), sentiment analysis, speech recognition, question answering systems, and topic segmentation, etc.

[0069] For example, natural language processing tasks can be categorized as follows.

[0070] Classification tasks: Machine learning models output a classification value for the entire input text. Examples include sentiment classification, topic classification, or whether the grammar is correct.

[0071] Information extraction task: Extract at least one type of information from the input text using a machine learning model.

[0072] Sequence labeling: For each word in the text, a machine learning model is required to assign a classification category based on the context. Examples include Chinese word segmentation, part-of-speech tagging, named entity recognition, and semantic role labeling.

[0073] Sentence relation inference: Machine learning models take two texts as input and determine whether these two texts have a certain nominal relationship. Examples include question answering systems, semantic rewriting, and natural language inference.

[0074] Generative tasks involve taking one piece of text as input and generating another piece of text using a machine learning model. Examples include machine translation, automatic summarization, or poetry and sentence creation.

[0075] It should be noted that the method provided in this application can also be applied to other natural language processing tasks using machine learning models. The examples above are only for the convenience of understanding this solution and are not intended to limit this solution.

[0076] As artificial intelligence technology develops, the machine learning models used to process text are becoming larger. Therefore, knowledge distillation can be used to compress the teacher model that has undergone training operations to obtain a smaller student model. However, in some scenarios, the training data used to train the teacher model is private data, making it impossible to obtain the training data used to train the student model. In the method provided in this application, the first training operation is performed with the help of the teacher model. After the first training operation is completed, the first machine learning model is used to generate the training data required when performing knowledge distillation on the aforementioned student model using the teacher model.

[0077] Before providing a detailed description of the methods provided in this application, please refer to [the relevant documentation / reference]. Figure 2 , Figure 2 This application provides a system architecture diagram for a data processing system, in which... Figure 2 In the data processing system, there are a first training device 210, a database 220, an execution device 230, and a second training device 240.

[0078] The database 220 stores a first dataset, which includes a large amount of text. During the execution phase of the first training operation, the first training device 210 performs the first training operation using the first dataset. For example, the first training device 210 inputs the first text into the first machine learning model 201 to obtain the second text output by the first machine learning model 201, the first text being derived from the first dataset; the first training device 210 then inputs the second text into the teacher model 202 to obtain the first prediction result output by the teacher model 202, and performs the first training operation based on the first prediction result. The first machine learning model 201 can be specifically represented as a neural network or as a non-neural network model; in this embodiment, only the example of the first machine learning model 201 being a neural network is used for illustration.

[0079] During the generation phase of the second dataset (i.e., the training data of student model 203), the execution device 230 deploys the first machine learning model 201. After completing the first training operation, the execution device 230 can use the first machine learning model 201 to generate multiple second texts. The second dataset used in the training phase of student model 203 includes the aforementioned multiple second texts.

[0080] During the execution phase of the second training operation, the second training device 240 can acquire a second dataset, which includes multiple second texts. The second training device 240 can input the second text distribution into the teacher model 202 and the student model 203 to obtain the second prediction result output by the teacher model 202 and the third prediction result output by the student model 203. Based on the second prediction result and the third prediction result, the second training device 240 iteratively trains the student model 203 using a distillation method.

[0081] It is worth noting that Figure 2 This is merely a schematic diagram of the architecture of a data processing system provided in an embodiment of the present invention, and the positional relationships between the devices, components, modules, etc. shown in the figure do not constitute any limitation. For example, in some other embodiments of this application, the first training device 210, the execution device 230, and the second training device 240 may be the same device; or, the first training device 210 may be an independent device, and the execution device 230 and the second training device 240 may be the same device; or, the first training device 210 and the execution device 230 may be the same device, and the second training device 240 may be another independent device, etc., and no limitations are made in the embodiments of this application.

[0082] As can be seen from the above description, the method provided in this application may include three stages. The specific implementation process of these three stages, namely the execution stage of the first training operation, the generation stage of the second dataset, and the execution stage of the second training operation, will be described below.

[0083] I. Execution Phase of the First Training Operation

[0084] For specific details in the embodiments described in this application, please refer to [link / reference]. Figure 3 , Figure 3 This is a flowchart illustrating a training method for a model provided in an embodiment of this application. The training method for a model provided in an embodiment of this application may include:

[0085] 301. Input the first text into the first machine learning model to obtain the second text output by the first machine learning model. The first text includes the third text and the first prompt. The third text belongs to the first dataset.

[0086] In this embodiment of the application, before inputting the first text into the first machine learning model, the first training device needs to obtain the first text based on the first dataset, which is an existing dataset. "The first dataset is an existing dataset" can be understood as the first dataset being a dataset that the first training device can obtain. For example, the first dataset can be a publicly available text dataset, or the first dataset can be a text dataset that the first training device can obtain but is not publicly available.

[0087] The first machine learning model is a machine learning model whose input and output are both text. For example, the first machine learning model can be a generator, or it can include an encoder and a decoder, etc. The first machine learning model can also be represented in other forms, which can be flexibly determined in combination with the actual application scenario. No limitation is made in the embodiments of this application.

[0088] The first training device can achieve "obtaining the first text based on the first training dataset" in several ways. In one implementation, the first training operation can be performed based on the idea of ​​prompt learning. In this case, each first text includes a third text and a first prompt. The third texts belong to the first dataset, meaning the first dataset contains multiple third texts. Therefore, obtaining the first text by the first training device can include: the first training device obtaining one or more third texts from the first dataset and combining each third text with the first prompt to obtain each first text. Optionally, during the first training operation, the parameters of the first machine learning model and the teacher model remain unchanged, while the first prompt is iteratively updated.

[0089] For example, since the first prompt is iteratively updated during the first training operation, if the first training device is acquiring the first text for the first time, that is, if the first training device is training the first prompt for the first time, then the first training device can obtain the initial expression of the first prompt and combine each third text with the initial expression of the first prompt to obtain each first text. If the first training device is performing training on the first prompt for the first time, the first training device can obtain the updated expression of the first prompt obtained in the previous training and combine each third text with the updated expression of the first prompt to obtain each first text.

[0090] For example, the initial expression and the updated expression of the first prompt can be represented as vectors or other forms of data, etc., which are not limited in the embodiments of this application.

[0091] The first prompt can take many forms. In one implementation, the first prompt can be a hybrid prompt, which can include both the prompt word and the second prompt. During the first training operation, the prompt word may not be updated, and only the second prompt may be updated.

[0092] Optionally, the prompt word used in the first prompt can mean a summary; for example, the prompt word can be "summarize", "conclude", "summarize", "summarize" or other words, etc. The examples of prompt words here are only to prove the feasibility of this solution and are not intended to limit this solution.

[0093] For a more intuitive understanding of this solution, please refer to [link / reference]. Figure 4 , Figure 4 This is a schematic diagram of the first text provided for an embodiment of this application. For example... Figure 4 As shown, the first text may include a prompt word, a first prompt, and a third text. Figure 4 For example, if the prompt word is "summarize", it should be understood that... Figure 4 The examples in the text are only for the purpose of understanding the concept of "first text" and are not intended to limit this scheme.

[0094] In this embodiment, the first prompt is designed to include two parts: a prompt word and a second prompt. The prompt word means a summary, and its function is to prompt the first machine learning model to summarize the information of the second prompt and the third text. This means that there is a certain degree of correlation between the second prompt plus the third text and the second text output by the first machine learning model. Since the third text comes from an existing dataset, it is rich in content. By adopting the aforementioned scheme, it is beneficial to retain the richness of the third text in the final multiple second texts.

[0095] Alternatively, in another implementation, if the first dataset includes multiple first texts, then the first training device acquiring the first texts may also include: the first training device directly acquiring each first text from the first dataset.

[0096] In this embodiment, the first training device can input each first text into a first machine learning model, process the first text through the first machine learning model, and obtain the second text output by the first machine learning model. For example, the second text may include multiple characters, and the output of the first machine learning model may be multiple first vectors representing the second text, where each of the multiple first vectors may indicate a character in the second text.

[0097] The first machine learning model includes multiple neural network layers, which may include at least one softmax layer. During the processing of the first text by the first machine learning model, the softmax layer processes at least one set of first data derived from the first text. That is, the aforementioned at least one set of first data may be data generated by the neural network layer preceding the softmax layer during the processing of the first text. Optionally, each set of first data may correspond to a single character in the second text. The neural network layer preceding the softmax layer in the first machine learning model may generate one set of first data at a time, or it may generate multiple sets of first data corresponding one-to-one with the multiple characters included in the entire second text.

[0098] Optionally, the softmax layer in the first machine learning model can be the last neural network layer of the first machine learning model. That is, the processing result output by the softmax layer in the first machine learning model after processing each first data can be used to indicate each character in the second text.

[0099] In one implementation, the process by which the first training device processes each piece of first data using the softmax layer in the first machine learning model may include: the first training device adding a perturbation to each piece of first data through the softmax layer obtained in the first machine learning model to obtain updated first data, and then processing the updated first data using the softmax function. For example, the perturbation added to each piece of first data may be random, or it may be based on a preset distribution space. For instance, the perturbation may be based on a Gumbel distribution space, a Gaussian distribution space, or other distribution spaces, etc., and can be flexibly determined according to the actual application scenario. This application embodiment does not impose any limitations. To further understand this solution, the following discloses an example of the formula used when processing the first data through the aforementioned softmax layer:

[0100]

[0101] Where y represents the processing result obtained by processing each first data point using the softmax layer in the first machine learning model, and h represents a single first data point. σ represents the perturbation added to the first data, g represents a vector sampled from the Gumbel distribution space; σ is a hyperparameter, optionally, the value of σ is around 1, that is, the difference between σ and 1 is within a preset range; τ is also a hyperparameter, τ can take a lower value in this scheme, for example, the value of τ can be 0.001 or other values, etc. It should be noted that Equation (1) is only an example for the convenience of understanding this scheme and is not used to limit this scheme.

[0102] In this embodiment, compared to the original first data, the updated first data obtained by adding perturbation to the first data is more diverse. Correspondingly, compared to the processing result of the first data obtained by directly processing the first data through the softmax function, the processing result of the updated first data obtained by processing the updated first data through the softmax function is more diverse. The softmax layer in the first machine learning model is often the last neural network layer of the first machine learning model. That is, the processing result obtained through the softmax layer indicates the second text that can be output by the entire first machine learning model. "The processing result of the updated first data is more diverse" means that the second text output by the entire first machine learning model is more diverse. In other words, adding perturbation to the first data that needs to be processed by the softmax layer helps to improve the diversity of the second text output by the machine learning model.

[0103] In another implementation, the process by which the first training device processes each first data point using the softmax layer in the first machine learning model may include: the first training device directly processes each first data point through the softmax layer in the first machine learning model.

[0104] Optionally, if the last neural network layer of the first machine learning model is not a softmax layer, after obtaining at least one second data point generated by the penultimate neural network layer of the first machine learning model, a perturbation can be added to each of the aforementioned second data points to obtain updated second data points. Each updated second data point is then processed by the last neural network layer of the first machine learning model to obtain the processing result of each updated second data point output by the last neural network layer of the first machine learning model (i.e., the output of the entire first machine learning model). The processing result of each updated second data point is used to indicate a character in the second text.

[0105] 302. Input the second text into the teacher model to obtain the first prediction result output by the teacher model.

[0106] In this embodiment of the application, after the first training device obtains the second text output by the first machine learning model, it can input the second text into the teacher model, process the second text through the teacher model, and obtain the first prediction result output by the teacher model. The teacher model is a machine learning model that has performed training operations.

[0107] This teacher model can be used to perform any one or more natural language processing tasks. Examples of various natural language processing tasks can be found in the above description, and will not be repeated here. The specific type of task performed by the teacher model can be determined in combination with the actual application scenario, and is not limited in the embodiments of this application.

[0108] The content indicated by the first prediction result is related to the category of the task performed by the teacher model. For example, if the teacher model is used to classify the sentiment of the input second text, the first prediction result can indicate that the sentiment category of the second text is positive, negative, or neutral. As another example, if the teacher model is used to perform part-of-speech tagging on the input second text, the first prediction result can indicate the part-of-speech category of each word in the second text. The part-of-speech category can be verb, noun, adverb, or other categories, etc., which are not exhaustively listed here.

[0109] For example, if the teacher model is used to extract information from the input second text to extract time and event type from the input second text, the first prediction result may include two sets of words, one set of words including the time extracted from the second text, and the other set of words including the time type extracted from the second text.

[0110] For example, if the teacher model is used to translate the second input text, the first prediction result can be the translation result of the second text, etc. It should be noted that the example of the first prediction result here is only for the convenience of understanding this scheme and does not limit this scheme.

[0111] The first prediction result may include one or more second vectors. In one application scenario, if the teacher model outputs text containing at least one character, the first prediction result includes at least one second vector corresponding one-to-one with the aforementioned at least one character, each second vector indicating a character in the aforementioned text. In another application scenario, if the teacher model outputs a predicted category for the entire input second text, the first prediction result can also be represented as a second vector indicating the predicted category of the second text.

[0112] 303. Based on the first prediction result, the first training operation is performed using the first loss function. During the first training operation, the parameters of the first machine learning model and the teacher model remain unchanged, and the first prompt is updated. The first prompt after training is used to obtain the second dataset. The training data in the second dataset will be used during the second training operation of the student model using the teacher model in the distillation method.

[0113] In this embodiment, after obtaining the first prediction result output by the teacher model, the first training device can perform a first training operation based on the first prediction result. Specifically, in one implementation, during the execution of the first training operation, the parameters of the first machine learning model and the teacher model remain unchanged, and the first prompt is updated. For example, the first training device can calculate the function value of the first loss function based on the first prediction result, perform gradient differentiation on the function value of the first loss function, and use the backpropagation algorithm to update the expression of the first prompt to complete one training iteration of the first prompt. The first training device repeats steps 301 to 303 multiple times to achieve iterative training of the first prompt until a convergence condition is met, at which point the first training operation is considered complete. The aforementioned convergence condition can be either satisfying the convergence condition of the first loss function or reaching a preset number of iterative training iterations on the first prompt.

[0114] Alternatively, in another implementation, the parameters of the first machine learning model can be updated during the execution of the first training operation. For example, the first training device can calculate the function value of the first loss function based on the first prediction result, perform gradient differentiation on the function value of the first loss function, and use the backpropagation algorithm to update the parameters of the first machine learning model to complete one training iteration of the first machine learning model. The first training device repeats steps 301 to 303 multiple times to achieve iterative training of the first machine learning model until the convergence condition is met, at which point the first training operation is considered complete.

[0115] "Training data in the second dataset" can be understood as multiple texts included in the second dataset. The specific implementation process of "generating training data in the second dataset by the first machine learning model after completing the first training operation" and "using the teacher model to perform the second training operation on the student model" will be described in subsequent embodiments and will not be elaborated here.

[0116] Optionally, the first loss function may include a first loss term, and the first prediction result includes at least one second vector. The purpose of training with the first loss term is to make each of the aforementioned at least one second vector closer to a one-hot form. Optionally, if the output of the teacher model is text containing at least one character, the first prediction result includes at least one second vector corresponding one-to-one with the aforementioned at least one character.

[0117] For example, a one-hot vector contains one 1 among its multiple values, and all other values ​​are 0. For a more intuitive understanding of this scheme, please refer to [link to relevant documentation]. Figure 5 , Figure 5 Several schematic diagrams of one-hot vectors provided in the embodiments of this application are shown. Figure 5 The text shows several one-hot vectors, including three vectors of length 5, two vectors of length 7, and one vector of length 10; for example... Figure 5 As shown, each one-hot vector contains only one 1, and the rest are 0. This should be understood. Figure 5 The examples in the text are only for the purpose of understanding the concept of "one-hot vectors" and are not intended to limit this scheme.

[0118] Optionally, the first loss term may indicate the similarity between each second vector and the third vector. The purpose of training using the first loss term includes making each second vector approximate the form of the third vector; the third vector is obtained based on the second vectors, and the third vector is in one-hot form. For example, each second vector may include multiple values. The first training device obtaining the third vector based on the second vectors may include: the first training device setting the largest value in the second vector as 1, and setting all other values ​​in the second vector as 0, thereby obtaining the third vector. For example, if the second vector is (0.1, 0.1, 0.2, 0.6), since 0.6 is the largest among the four values ​​included in the second vector, 0.6 in the second vector is set as 1, and the other values ​​in the second vector are set as 0 to obtain the third vector. Therefore, the third vector obtained based on the second vector can be represented as (0, 0, 0, 1).

[0119] For example, if the second vector is represented as (0.01, 0.08, 0.01, 0.2, 0, 0, 0.7), since 0.7 is the largest among the seven values ​​included in the second vector, we set 0.7 in the second vector to 1 and set the other values ​​in the second vector to 0 to obtain the third vector. The third vector obtained based on the second vector can be represented as (0, 0, 0, 0, 0, 0, 1), etc. It should be noted that the examples here are only for the convenience of understanding "obtaining a one-hot form of the third vector based on the second vector". The specific representation of the "second vector" and the "third vector" can be determined according to the actual situation, and no limitation is made in the embodiments of this application.

[0120] To further understand this scheme, the following example illustrates the function used in the first loss term, assuming the teacher model outputs multiple vectors:

[0121]

[0122] Among them, L SPEC Represents the first loss term, and N represents the number of second texts within a batch. This represents the j-th second vector included in the first prediction result output by the teacher model after inputting the i-th second text out of N second texts. In other words, it's the second vector corresponding to the j-th character among the multiple characters included in the first prediction result. Representative based on The resulting third vector It is a vector in one-hot form, "based on get The specific implementation process can be found in the above description, and will not be repeated here; Representative calculation and The similarity between them, in equation (2) is... The calculation is and Taking cross-entropy (CE) as an example, and The cross-entropy between them is calculated as follows: and The degree of difference between them and The greater the difference between them, the better. and The greater the cross-entropy between them, the better the use of L SPEC The purpose of training is to make L SPEC Getting smaller and smaller, which is why... and The differences between them become smaller and smaller. It should be noted that the examples in equation (2) are only for the convenience of understanding this scheme. For example, "calculate..." and The method of "similarity between" can also be replaced by calculating and The cosine similarity, L1 distance, or other similarity calculation methods can be used; for example, the N second texts may not include all the second texts in a batch, and the value of N can be 2, 3, 4, 5 or other values, etc. The examples in equation (2) are not intended to limit this scheme.

[0123] In this embodiment, if each vector in the first prediction result is closer to the one-hot form, it means that the teacher model is more confident in the output of the first prediction result. This means that the second text input into the teacher model is data that the teacher model is familiar with, that is, it means that the second text is more similar to the privacy data used when training the teacher model. Using the first loss term is beneficial to improve the similarity between the final second text (that is, the training data used when performing knowledge distillation on the student model) and the privacy data used when training the teacher model. In addition, when the output of the teacher model is text, that is, when the first prediction result includes multiple vectors that correspond one-to-one with at least one character in the text, the purpose of training with the first loss term is to make each vector in the first prediction result close to the one-hot form. That is, the first loss term can be compatible with the case where the output of the teacher model is text, which greatly expands the application scenarios of this solution.

[0124] Optionally, the first loss function may include a second loss term, which indicates the entropy of multiple first prediction results. The purpose of training with the second loss term includes improving the entropy of multiple first prediction results. The entropy of multiple first prediction results reflects the degree of disorder of the information system composed of multiple first prediction results. The greater the entropy of multiple first prediction results, the greater the disorder of the information system composed of multiple first prediction results, that is, the more diverse the multiple first prediction results are.

[0125] For example, the aforementioned at least one first prediction result can be a first prediction result corresponding to a preset number of second texts. The preset number of second texts can be second texts within a batch, or it can include second texts within different batches. The preset number can be 2, 3, 4, 5, or other values, etc. Alternatively, the aforementioned at least one first prediction result can include first prediction results corresponding to all second texts within a batch, or the aforementioned at least one first prediction result can include first prediction results corresponding to all second texts within half a batch, etc. The specific number of first prediction results obtained before starting to calculate the second loss term can be flexibly determined in combination with the actual application scenario, and no limitation is made in the embodiments of this application.

[0126] To further understand this scheme, an example of the function used for the second loss term is disclosed below:

[0127]

[0128] Among them, L divs P represents the second loss term, where N represents the number of second texts within a batch, and P represents the second loss term. i This represents a first prediction result generated by the teacher model corresponding to the i-th second text among N second texts. This represents the N first prediction results generated by the teacher model, each corresponding to one of the N second texts. This represents the information entropy of calculating N first prediction results. This represents the information entropy of the N first prediction results divided by N; using L divs The purpose of training includes enabling L divs The value of L is getting smaller and smaller, that is, using L divs The purpose of training includes enabling Getting bigger and bigger, that is, using L divs The purpose of training is to increase the information entropy of multiple first prediction results. It should be understood that the example in Equation (2) is only for the convenience of understanding this scheme and is not intended to limit this scheme.

[0129] In this embodiment, the greater the information entropy of the multiple first prediction results, the more diverse the multiple first prediction results are. For the teacher model, the multiple second texts corresponding one-to-one with the aforementioned multiple first prediction results are diverse, which is beneficial to improving the diversity of the obtained multiple second texts. In addition, the method of using a second loss term to improve the diversity of multiple second texts avoids the coupling between the "execution process of the first training operation" and the "generation process of the second text used in the training phase of the student model". That is, the first training operation does not need to be executed each time the "generation process of the second text used in the training phase of the student model" is executed, which reduces the difficulty of the "generation process of the second text used in the training phase of the student model" and also improves the flexibility when executing the "generation process of the second text used in the training phase of the student model".

[0130] Optionally, if the task of the teacher model is to obtain at least one type of information from the second text, that is, the first prediction result includes at least one set of words that correspond one-to-one with the aforementioned at least one type, the first loss function may include a third loss term. The third loss term may indicate the similarity between different words included in the same set of words in the at least one set of words. The purpose of training with the third loss term includes improving the similarity between different words included in the same set of words in the at least one set of words.

[0131] For example, if the teacher model's task is to obtain the time, location, and event type from the input second text, the first prediction result can include three sets of words: the set of words corresponding to "time" includes "April 20th" and "April 21st", the set of words corresponding to "location" includes "XX Hot Spring Town" and "XX Square", and the set of words corresponding to "event type" includes "tourism". The third loss term can indicate the similarity between different words in the set of words corresponding to "time", the similarity between different words in the set of words corresponding to "location", and / or the similarity between different words in the set of words corresponding to "event type", etc. In addition, since the set of words corresponding to "event type" includes only one word, the similarity calculated using the third loss term can be the similarity between "tourism" and "tourism". It should be noted that the example here is only for the convenience of understanding this scheme and is not intended to limit this scheme.

[0132] Optionally, if the task of the teacher model is to obtain at least two types of information from the second text, that is, the first prediction result includes at least two sets of words that correspond one-to-one with the aforementioned at least two types, the third loss term can also indicate the similarity between different sets of words in the at least two sets of words included in the first prediction result. The purpose of training with the third loss term also includes reducing the similarity between different sets of words in the at least two sets of words.

[0133] For example, if the teacher model's task is to obtain the time, location, and event type from the input second text, the first prediction result can include three sets of words: the set of words corresponding to "time" includes "April 20th" and "April 21st", the set of words corresponding to "location" includes "XX Hot Spring Town" and "XX Square", and the set of words corresponding to "event type" includes "tourism". The third loss term can also indicate the similarity between a word in the set of words corresponding to "time" and a word in the set of words corresponding to "location", the similarity between a word in the set of words corresponding to "location" and a word in the set of words corresponding to "event type", or the similarity between a word in the set of words corresponding to "time" and a word in the set of words corresponding to "event type", etc. It should be noted that the example here is only for the convenience of understanding this scheme and is not intended to limit this scheme.

[0134] To further understand this scheme, an example of the function used for the third loss term is disclosed below:

[0135]

[0136]

[0137]

[0138] Among them, L scl R represents the third loss term, and R represents the first prediction result including the word R. r Representing the r-th word in the R-group of words, the r-th word output by the teacher model can specifically include a second vector corresponding one-to-one with each character in the r-th word, s i,j ∈D r Represents s i,j This includes a word from the r-th word group output by the teacher model, containing the i-th to j-th vectors from the first prediction result, i.e., s. i,j It can represent one word in the r-th group of words; represent Including another word in the r-th group of words output by the teacher model, containing the first prediction result. The vector to the first a vector, that is If s can represent another word in the r-th group of words, then s i,j and For the different words in the r-th group; N r This represents the number of words included in the r-th word group; The expansion can be found in equation (4).

[0139] Instructions i,j and Similarity between them The expansion can be found in equation (5), where s is calculated. i,j and Taking the cosine similarity between them as an example, it can also be calculated by s i,j and The Euclidean distance, L1 distance, or other methods are used to obtain s. i,j and Similarities between them, etc., will not be exhaustively listed here; β is a hyperparameter; The first prediction result represents the first word in the R group. Word formation, Represents s m,n Including the R-word group output by the teacher model, the first If the m-th vector to the n-th vector in the word combination is s, then i,j and s m,n For words in the R group, words from different groups, Instructions i,j With s m,n Similarity between them The calculation method can be found in [reference]. The calculation method will be explained in detail here; it can be seen from equation (4) that, It not only indicates the similarity between different words in the same group of words included in the first prediction result, but also the similarity between words in different groups of words included in the first prediction result.

[0140] Using L scl The purpose of training is to make L scl Getting smaller and smaller, which is why... It gets bigger and bigger, thus utilizing L scl The purpose of the training includes increasing the similarity between different words in the same group of words in the R group, and also reducing the similarity between words in different groups of words in the R group. It should be understood that the examples in Equations (3) to (5) are only for the convenience of understanding this scheme and are not intended to limit this scheme.

[0141] In this embodiment of the application, the purpose of training with the third loss term includes improving the similarity between different words in the same type of information. Since the task of the teacher model is to obtain at least one type of information from the input second text, the different words in the same type of information should be more similar. When the different words in the same type of information are more similar, it means that the first prediction result generated by the teacher model when performing the information acquisition task on the input second text is more accurate. It can reflect that the second text input into the teacher model is the text that the teacher model is better at processing, that is, the second text is data that the teacher model is familiar with. Therefore, the third loss term can improve the similarity between the final second text and the privacy training data used when training the teacher model.

[0142] The purpose of using the third loss term for training also includes reducing information from different categories. Since the task of the teacher model is to extract at least one type of information from the input second text, words from different categories should be less similar. When different words from different categories are less similar, it means that the first prediction result generated by the teacher model when performing the information extraction task on the input second text is more accurate, and it can reflect that the second text input to the teacher model is data that the teacher model is familiar with. Therefore, using the third loss term can improve the similarity between the final second text and the privacy-preserving training data used when training the teacher model.

[0143] Optionally, the first loss function may include a first loss term and a second loss term.

[0144] Optionally, if the task of the teacher model is to obtain at least one type of information from the input second text, the first loss function may include a first loss term and a third loss term, or the first loss function may include a second loss term and a third loss term, or the first loss function may include a first loss term, a second loss term, and a third loss term.

[0145] For a more intuitive understanding of this solution, please refer to [link / reference]. Figure 6 , Figure 6 This is another flowchart illustrating the training method for the model provided in this application embodiment. For example... Figure 6 As shown, after the first training device obtains the third text from the first dataset, it will display the prompt words (i.e., Figure 6 The Summarize, the expression of the first prompt, and the third text are concatenated to obtain the first text; the first text is then input into the first machine learning model to obtain the second text output by the first machine learning model.

[0146] The first training device inputs the second text into the teacher model and obtains the first prediction result output by the teacher model. Based on the first prediction result, the values ​​of the first loss term and the second loss term are calculated respectively. Based on the values ​​of the first loss term and the second loss term, the function value of the first loss function is generated. The meanings of "first loss term" and "second loss term" can be found in the above description and will not be repeated here.

[0147] The first training device calculates the gradient derivative of the first loss function and uses the backpropagation algorithm to update the expression of the first prompt, thus completing one training iteration of the first prompt. It should be understood that... Figure 6 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0148] In this embodiment, during the second training operation of the student model using the teacher model via distillation, training data from the second dataset is used. This training data is generated by a dedicated first machine learning model after the first training operation, which is based on the first prediction result output by the teacher model. The input to the teacher model is the output text of the aforementioned first machine learning model. Since the teacher model is trained using private data, it is familiar with the private data. Therefore, the first training operation leverages the teacher model's capabilities to guide the text generated by the first machine learning model to approximate the private data. Thus, the training data used when training the student model via distillation can... Similar to privacy data, this approach helps improve the accuracy of the trained student model when performing natural language processing tasks. The input to the first machine learning model is text from an existing first dataset, which contains rich text. Correspondingly, the output text of the first machine learning model is also rich, thus ensuring the richness of the training data for the student model. Furthermore, during the execution of the first training operation, the parameters of the first machine learning model and the teacher model remain unchanged, and only the first prompt is updated. This reduces the number of parameters that need to be updated during the execution of the first training operation, which helps to reduce the execution difficulty of the first training operation, reduce the computer resources consumed during the execution of the first training operation, and also helps to improve the execution efficiency of the first training operation.

[0149] II. Generation Stage of the Second Dataset

[0150] In this embodiment, the "second dataset generation stage" can also be understood as the stage where multiple second texts (i.e., the second dataset) are generated using the first machine learning model after the first training operation is completed; or, it can be understood as the training data generation stage used when the teacher model performs knowledge distillation on the student model. For details, please refer to [link to relevant documentation]. Figure 7 , Figure 7This is a flowchart illustrating a method for acquiring training data provided in an embodiment of this application. The method for acquiring training data provided in an embodiment of this application may include:

[0151] 701. Obtain the first text, which includes the third text and the first prompt after training. The third text belongs to the first dataset, and the first prompt after training is obtained through the first training operation.

[0152] 702. Input the first text into the first machine learning model to obtain the second text output by the first machine learning model. In the process of performing the second training operation on the student model using the teacher model by employing distillation, a second dataset is used. The second dataset includes multiple second texts. When performing the first training operation using the first loss function, the parameters of the first machine learning model and the teacher model remain unchanged, and the first prompt is updated. The input of the teacher model is the output text of the machine learning model, and the first loss function is obtained based on the first prediction result output by the teacher model.

[0153] In this application embodiment, the specific implementation of steps 701 and 702, the beneficial effects they bring, and the meanings of the terms in steps 701 and 702 can all be found in [reference needed]. Figure 3 The description in the corresponding embodiments will not be repeated here. The specific implementation process of the above-mentioned "first training operation" can be found in [reference needed]. Figure 3 In accordance with the description in the corresponding embodiments, it should be noted that in one implementation, as described in steps 701 and 702 above, the first dataset includes multiple third texts, each of which includes a third text and a first prompt. During the execution of the first training operation, the first prompt is iteratively updated (or the expression of the first prompt is iteratively updated). Therefore, the first text used in steps 701 and 702 includes the first prompt that has undergone the first training operation and the third text obtained from the first dataset.

[0154] Alternatively, in another implementation, the first dataset directly includes multiple first texts, and the parameters of the first machine learning model are iteratively updated during the execution of the first training operation. In this case, the first machine learning model used in steps 701 and 702 is the first machine learning model that has performed the first training operation.

[0155] III. Execution Phase of the Second Training Operation

[0156] In this embodiment, the "execution phase of the second training operation" can also be understood as the process of using a distillation method to perform a second training operation on the student model using the teacher model; it can also be understood as the process of using the teacher model to perform knowledge distillation on the student model. For details, please refer to... Figure 8 , Figure 8This is another schematic flowchart illustrating the training method for the model provided in this application embodiment. The training method for the model provided in this application embodiment may include:

[0157] 801. Input the second text into the teacher model and the student model respectively to obtain the second prediction result output by the teacher model and the third prediction result output by the student model.

[0158] In this embodiment, the second training device can obtain one or more second texts from the second dataset, input each second text into the teacher model and the student model respectively, process each second text through the teacher model to obtain the second prediction result output by the teacher model, and process each second text through the student model to obtain the third prediction result output by the student model. The task type performed by the "teacher model" can be found above. Figure 3 The descriptions in the corresponding embodiments will not be repeated here. The meaning of "second prediction result" is the same as that of "first prediction result," the difference being that "second prediction result" is generated by the teacher model during the execution phase of the second training operation, while "first prediction result" is generated by the teacher model during the execution phase of the first training operation. The meaning of "second prediction result" can be found above. Figure 3 The descriptions in the corresponding embodiments will not be repeated here.

[0159] The student model is also used to perform one or more natural language processing tasks. The student model and the teacher model are used to perform the same tasks. The content indicated by the "third prediction result" is the same as the content indicated by the "first prediction result," as described above. Figure 3 The descriptions in the corresponding embodiments will not be repeated here.

[0160] The difference between the student model and the teacher model is that the student model is smaller in size than the teacher model. In other words, the storage space occupied by the student model is smaller than that occupied by the teacher model, and the computer resources consumed when using the student model for data processing are less than those consumed when using the teacher model for data processing.

[0161] 802. Based on the second and third prediction results, the student model is trained using a distillation method. The second text is generated by the first machine learning model based on the first prompt after training. The input of the first machine learning model is the first text, which includes the third text and the aforementioned first prompt after training. The third text belongs to the first dataset. The first prompt after training is obtained through the first training operation. When performing the first training operation using the first loss function, the parameters of the first machine learning model and the teacher model remain unchanged, and the expression of the first prompt is updated. The input of the teacher model is the output text of the first machine learning model, and the first loss function is obtained based on the first prediction result output by the teacher model.

[0162] In this embodiment, the meanings of several terms in steps 801 and 802 can be found in [reference needed]. Figure 3 The description in the corresponding embodiments will not be repeated here. The specific implementation process of the above-mentioned "first training operation" can be found in [reference needed]. Figure 3 The descriptions in the corresponding embodiments will not be repeated here.

[0163] The second training device trains the student model using a distillation method based on the second prediction result, the third prediction result, and the function value of the second loss function. The second training device repeats steps 801 and 802 to iteratively train the student model until the convergence condition is met, thus obtaining the student model that has undergone the second training operation; the aforementioned convergence condition can be either satisfying the convergence condition of the second loss function or reaching a preset number of iterative training iterations of the student model.

[0164] For example, the second training device can generate the function value of the second loss function based on the second prediction result and the third prediction result, perform gradient differentiation on the function value of the second loss function, and use the backpropagation algorithm to update the parameters of the student model, thereby completing one training of the student model.

[0165] Optionally, the second loss function may include a fourth loss term, which indicates the similarity between the second prediction output by the teacher model and the third prediction output by the student model. The purpose of training with the fourth loss term includes improving the similarity between the second and third predictions, that is, the purpose of training with the fourth loss term includes reducing the degree of difference between the second and third predictions.

[0166] For example, the "similarity between the second and third prediction results" can be obtained by calculating the Jensen-Shannon divergence (JSD), the kullback-leibler divergence (KLD), or other information between the second and third prediction results, without limitation here.

[0167] To further understand this scheme, the following example illustrates the function used for the fourth loss term, assuming that both the teacher and student models output multiple vectors:

[0168]

[0169] Among them, L divg This represents the fourth loss term, where N represents the number of second texts within a batch. This represents the j-th second vector included in the third prediction result output by the student model after inputting the i-th second text out of N second texts. This represents the j-th second vector included in the second prediction result output by the teacher model after inputting the i-th second text out of N second texts. Representative calculation and The similarity between them should be understood. The examples in Equation (6) are only for the convenience of understanding this scheme and are not intended to limit this scheme.

[0170] Optionally, the second loss function may include a fifth loss term; the fifth loss term indicates the similarity between the first feature information and the second feature information, the first feature information being the feature information of the second text generated during the teacher model's processing of the second text, and the second feature information being the feature information of the second text generated during the student model's processing of the second text, the purpose of training using the fifth loss term includes improving the similarity between the first feature information and the second feature information.

[0171] To further understand this scheme, an example of the function used in the fifth loss term is disclosed below:

[0172] L hidd =‖(h S W h h T )‖2; (7)

[0173] Among them, L hiidd Representing the fifth loss term, ‖(h S W h h T)‖2 represents the calculation of h S W h with h T The L2 distance between them, h S h represents the second feature information generated by the student model. S W h This represents a linear transformation of the second feature information, h T The first feature information generated by the teacher model represents the information. It should be understood that the example in equation (7) is only for the convenience of understanding this scheme. For example, in actual practice, the calculation of h can also be performed. S W h with h T The L2 distance between them is replaced by calculating h. S W h with h T The similarity between the first feature information and the second feature information can be calculated by L1 distance, Euclidean distance or other means. The examples in Equation (7) are not intended to limit this scheme.

[0174] Optionally, the second loss function may include a fourth loss term and a fifth loss term. To further understand this scheme, an example of the second loss function is disclosed below:

[0175] L KD =λ1L divg +λ2Lh idd (8)

[0176] Among them, L KD λ1 represents the second loss function, and L represents the loss function. divg The weights, λ2 represents Lh idd The weights, that is, the weighted summation of the fourth and fifth loss terms, are used to obtain the second loss function. It should be understood that the example in Equation (8) is only for the convenience of understanding this scheme and is not intended to limit this scheme.

[0177] To further understand the beneficial effects of the method provided in this application, the following describes these effects in conjunction with experimental data. We tested the method on the student model for classification and information extraction tasks, and the test results are shown in Table 1 below.

[0178]

[0179] Table 1

[0180] Among them, general domain text (OOD) is a method for knowledge distillation of student models without original training data. When the student model performs the SST-2 task, that is, the student model outputs a predicted sentiment classification for the entire input text, the accuracy (ACC) is used to measure the accuracy of the sentiment classification output by the student model. The higher the accuracy of the classification value output by the student model, the higher the value of ACC.

[0181] When the student model performs the CoLA task, that is, the student model outputs a predicted classification value for the entire input text. The predicted classification value indicates whether the input text is grammatically correct. The Matthews correlation coefficient (MCC) is used to measure the accuracy of the classification value output by the student model. The higher the accuracy of the classification value output by the student model, the higher the value of the MCC.

[0182] When the task performed by the student model is information extraction, that is, the student model identifies at least one type of information with specific meaning from the entire input text, the F1 score is used to measure the accuracy of the prediction results output by the student model. The higher the accuracy of the prediction results output by the student model, the higher the F1 score.

[0183] As shown in Table 1, the prediction results of the student model trained using the method provided in this application are more accurate than those of the student model trained using OOD.

[0184] We also conducted more comparative experiments when the student model performed the SST-2 task, and the experimental results are shown in Table 2 below.

[0185] SST-2 Mission (ACC) Unlabel KD 84.90 Unlabel KD+Adv 85.90 The method provided in this application 88.19

[0186] Table 2

[0187] Unlabeled KD and adversarial KD+Adv are two existing methods for knowledge distillation of student models without training data. When the student model performs the SST-2 task, ACC is still used to measure the accuracy of the sentiment classification output by the student model. The higher the accuracy of the classification value output by the student model, the higher the value of ACC. As shown in Table 2, the prediction results output by the student model trained using the method provided in this application have the highest accuracy.

[0188] exist Figures 1 to 8Based on the corresponding embodiments, in order to better implement the above-described solutions of this application, related equipment for implementing the above solutions is also provided below. See details. Figure 9 , Figure 9 This is a schematic diagram of a model training device provided in an embodiment of this application. The model training device 900 includes: an input module 901, used to input a first text into a machine learning model to obtain a second text output by the machine learning model. The first text includes a third text and a first prompt, and the third text belongs to a first dataset. The input module 901 is also used to input the second text into a teacher model to obtain a first prediction result output by the teacher model. The training module 902 is used to perform a first training operation based on the first prediction result until the convergence condition of the loss function is met. During the execution of the first training operation, the parameters of the machine learning model and the teacher model remain unchanged, and the first prompt is updated. The trained first prompt is used to obtain a second dataset. The training data in the second dataset is used during the second training operation performed by the teacher model on the student model using a distillation method.

[0189] Optionally, the loss function includes a first loss term, and the first prediction result includes at least one vector corresponding one-to-one with at least one word. The purpose of training with the first loss term is to make each vector closer to a one-hot form.

[0190] Optionally, the loss function includes a second loss term, which indicates the information entropy of multiple first prediction results.

[0191] Optionally, the first prompt includes a prompt word and a second prompt, where the prompt word is a summary, and the second prompt is updated during the execution of the first training operation.

[0192] Optionally, the machine learning model includes a normalization index (softmax) layer. During the process of the machine learning model processing the first text, the softmax layer in the machine learning model needs to process the first data obtained based on the first text. The process of using the softmax layer to process the first data includes: adding perturbation to the first data to obtain updated first data, and using the softmax function to process the updated first data.

[0193] Optionally, the teacher model's task is to obtain at least one type of information from the second text, the first prediction result includes at least one set of words corresponding one-to-one with the at least one type, the loss function includes a third loss term, and the purpose of training with the third loss term includes improving the similarity between different words included in the same set of words in the at least one set of words.

[0194] Optionally, training with a third loss term also aims to reduce the similarity between different groups of words in at least one set of words.

[0195] It should be noted that the information interaction and execution process between the modules / units in the model training device 900 are different from those in this application. Figures 3 to 6 The various method embodiments are based on the same concept, and the details can be found in the descriptions of the method embodiments shown above in this application, which will not be repeated here.

[0196] Please continue reading. Figure 10 , Figure 10 This is a schematic diagram of a training data acquisition device provided in an embodiment of this application. The training data acquisition device 1000 includes: an acquisition module 1001, used to acquire a first text, which includes a third text and a first prompt after training. The third text belongs to a first dataset, and the first prompt after training is obtained through a first training operation; and an input module 1002, used to input the first text into a machine learning model to obtain a second text output by the machine learning model. The second dataset, which includes multiple second texts, is used during the second training operation performed on the student model using a teacher model in a distillation manner. When performing the first training operation using a loss function, the parameters of the machine learning model and the teacher model remain unchanged, and the first prompt is updated. The input to the teacher model is the output text of the machine learning model, and the loss function is obtained based on the first prediction result output by the teacher model.

[0197] Optionally, the first prediction result includes at least one vector corresponding one-to-one with at least one word, and the loss function includes a first loss term. The purpose of training with the first loss term is to make each vector closer to the one-hot form.

[0198] Optionally, the loss function includes a second loss term, which indicates the information entropy of multiple first prediction results.

[0199] It should be noted that the information interaction and execution process between the modules / units in the training data acquisition device 1000 are different from those in this application. Figure 7 The various method embodiments are based on the same concept, and the details can be found in the descriptions of the method embodiments shown above in this application, which will not be repeated here.

[0200] Please continue reading. Figure 11 , Figure 11The following is another schematic diagram of the training device for the model provided in the embodiments of this application. The training device 1100 for the model includes: an input module 1101, used to input the second text into the teacher model and the student model respectively, and obtain the second prediction result output by the teacher model and the third prediction result output by the student model;

[0201] Training module 1102 is used to train the student model using a distillation method based on the second prediction result and the third prediction result. The second text is generated by the first machine learning model based on the first prompt after training. The input of the first machine learning model is the first text, which includes the third text and the aforementioned first prompt after training. The third text belongs to the first dataset. The first prompt after training is obtained through the first training operation. When performing the first training operation using the first loss function, the parameters of the first machine learning model and the teacher model remain unchanged, and the expression of the first prompt is updated. The input of the teacher model is the output text of the first machine learning model, and the first loss function is obtained based on the first prediction result output by the teacher model.

[0202] It should be noted that the information interaction and execution process between the modules / units in the model training device 1100 are different from those in this application. Figure 8 The various method embodiments are based on the same concept, and the details can be found in the descriptions of the method embodiments shown above in this application, which will not be repeated here.

[0203] The following describes a device provided in an embodiment of this application. Please refer to [link / reference]. Figure 12 , Figure 12 This is a schematic diagram of a device provided in an embodiment of this application. The device 1200 can be used to perform... Figures 3 to 6 The steps performed by the first training device, or device 1200, can be used to perform... Figure 7 The steps to be executed by the device are as follows, or the device 1200 can be used to execute... Figure 8 The steps performed by the second training device.

[0204] Specifically, device 1200 is implemented by one or more servers. Device 1200 can vary significantly due to differences in configuration or performance, and may include one or more central processing units (CPUs) 1222 (e.g., one or more processors) and memory 1232, and one or more storage media 1230 (e.g., one or more mass storage devices) for storing application programs 1242 or data 1244. The memory 1232 and storage media 1230 can be temporary or persistent storage. The program stored in storage media 1230 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the training device. Furthermore, the CPU 1222 may be configured to communicate with storage media 1230 and execute the series of instruction operations in storage media 1230 on device 1200.

[0205] Device 1200 may also include one or more power supplies 1226, one or more wired or wireless network interfaces 1250, one or more input / output interfaces 1258, and / or one or more operating systems 1241, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0206] In one embodiment of this application, the central processing unit 1222 is used to execute... Figures 3 to 6 The training method of the model executed by the first training device in the corresponding embodiment. The specific manner in which the central processing unit 1222 executes the aforementioned steps is the same as that in this application. Figures 3 to 6 The various method embodiments are based on the same concept, and the technical effects they bring are the same as those in this application. Figures 3 to 6 The corresponding method embodiments are the same, and for details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0207] In another scenario, the central processing unit 1222 is used to execute... Figure 7 The method for acquiring training data executed by the execution device in the corresponding embodiment. The specific manner in which the central processing unit 1222 executes the aforementioned steps is the same as in this application. Figure 7 The various method embodiments are based on the same concept, and the technical effects they bring are the same as those in this application. Figure 7 The corresponding method embodiments are the same, and for details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0208] In another scenario, the central processing unit 1222 is used to execute... Figure 8The training method for the model executed by the second training device in the corresponding embodiment. The specific manner in which the central processing unit 1222 executes the aforementioned steps is the same as in this application. Figure 8 The various method embodiments are based on the same concept, and the technical effects they bring are the same as those in this application. Figure 8 The corresponding method embodiments are the same, and for details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0209] This application embodiment also provides a computer-readable storage medium storing a program for performing signal processing, which, when run on a computer, causes the computer to perform the aforementioned actions. Figures 3 to 6 The steps performed by the first training device in the method described in the illustrated embodiment, or causing the computer to perform the steps as described above. Figure 7 The method described in the illustrated embodiment executes steps performed by the execution device, or causes the computer to perform the steps as described above. Figure 8 The steps performed by the second training device in the method described in the illustrated embodiment.

[0210] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned actions. Figures 3 to 6 The steps performed by the first training device in the method described in the illustrated embodiment, or causing the computer to perform the steps as described above. Figure 7 The method described in the illustrated embodiment executes steps performed by the execution device, or causes the computer to perform the steps as described above. Figure 8 The steps performed by the second training device in the method described in the illustrated embodiment.

[0211] The execution device, first training device, or second training device provided in this application embodiment can specifically be a chip. The chip includes a processing unit and a communication unit. The processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, pins, or circuitry. The processing unit can execute computer execution instructions stored in the storage unit to cause the chip to perform the aforementioned operations. Figures 3 to 6 The training method for the model described in the illustrated embodiment, or, to cause the chip to perform the above... Figure 7 The method for acquiring training data described in the illustrated embodiment, or, to cause the chip to perform the above... Figure 8 The training method of the model described in the illustrated embodiment. Optionally, the storage unit is a storage unit within the chip, such as a register, cache, etc. The storage unit can also be a storage unit located outside the chip within the wireless access device, such as read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM), etc.

[0212] For details, please refer to Figure 13 , Figure 13 This is a schematic diagram of a chip provided in an embodiment of this application. The chip can be represented as a neural network processor (NPU) 130. The NPU 130 is mounted as a coprocessor on the host CPU, and tasks are assigned by the host CPU. The core part of the NPU is the arithmetic circuit 1303, which is controlled by the controller 1304 to extract matrix data from the memory and perform multiplication operations.

[0213] In some implementations, the arithmetic circuit 1303 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 1303 is a two-dimensional pulsating array. The arithmetic circuit 1303 can also be a one-dimensional pulsating array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1303 is a general-purpose matrix processor.

[0214] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 1302 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 1301 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 1308.

[0215] Unified memory 1306 is used to store input and output data. Weight data is directly transferred to weight memory 1302 via Direct Memory Access Controller (DMAC) 1305. Input data is also transferred to unified memory 1306 via DMAC.

[0216] BIU stands for Bus Interface Unit, which is used for interaction between the AXI bus and the DMAC and the Instruction Fetch Buffer (IFB) 1309.

[0217] The Bus Interface Unit (BIU) 1310 is used by the instruction fetch memory 1309 to fetch instructions from external memory, and also by the memory access controller 1305 to fetch the original data of the input matrix A or the weight matrix B from external memory.

[0218] The DMAC is mainly used to move input data from external memory DDR to unified memory 1306, or to weight data to weight memory 1302, or to input data to input memory 1301.

[0219] The vector computation unit 1307 includes multiple arithmetic processing units that, when necessary, further process the output of the computation circuit, such as vector multiplication, vector addition, exponential operations, logarithmic operations, size comparisons, etc. It is mainly used for computation in non-convolutional / fully connected layers of neural networks, such as batch normalization, pixel-level summation, and upsampling of feature planes.

[0220] In some implementations, the vector computation unit 1307 can store the processed output vector in the unified memory 1306. For example, the vector computation unit 1307 can apply linear and / or nonlinear functions to the output of the computation circuit 1303, such as performing linear interpolation on feature planes extracted by convolutional layers, or accumulating a vector of values ​​to generate activation values. In some implementations, the vector computation unit 1307 generates normalized values, pixel-level summed values, or both. In some implementations, the processed output vector can be used as activation input to the computation circuit 1303, for example, for use in subsequent layers of the neural network.

[0221] The instruction fetch buffer 1309 connected to the controller 1304 is used to store the instructions used by the controller 1304;

[0222] Unified memory 1306, input memory 1301, weighted memory 1302, and instruction fetch memory 1309 are all on-chip memories. External memory is proprietary to this NPU hardware architecture.

[0223] The operations of each layer in the first machine learning model, teacher model, or student model can be performed by the operation circuit 1303 or the vector calculation unit 1307.

[0224] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of a program in the first aspect of the method.

[0225] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0226] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0227] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0228] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method for training a model, characterized in that, The method includes: The first text is input into the machine learning model to obtain the second text output by the machine learning model. The first text includes a third text and a first prompt. The third text belongs to the first dataset. The second text is input into the teacher model to obtain the first prediction result output by the teacher model; The first training operation is performed based on the first prediction result until the convergence condition of the loss function is met. The first prediction result includes a vector, and the loss function includes a first loss term. The purpose of training with the first loss term is to make the vector closer to the one-hot form. During the execution of the first training operation, the parameters of the machine learning model and the teacher model are kept unchanged, and the first prompt is updated. The first prompt after training is used to obtain the second dataset. The training data in the second dataset will be used during the second training operation of the student model using the teacher model in the distillation method.

2. The method of claim 1, wherein, The loss function further includes a second loss term, which indicates the information entropy of multiple first prediction results.

3. The method according to claim 1 or 2, characterized in that, The first prompt includes a prompt word and a second prompt. The prompt word means a summary. The second prompt is updated during the execution of the first training operation.

4. The method according to claim 1 or 2, characterized in that, The machine learning model includes a normalized exponent softmax layer. During the process of the machine learning model processing the first text, the softmax layer in the machine learning model needs to process the first data obtained based on the first text. The process of processing the first data using the softmax layer includes: adding perturbation to the first data to obtain updated first data, and processing the updated first data using the softmax function.

5. The method according to claim 1 or 2, characterized in that, The task of the teacher model is to obtain at least one type of information from the second text. The first prediction result includes at least one set of words that correspond one-to-one with the at least one type. The loss function also includes a third loss term. The purpose of training with the third loss term is to improve the similarity between different words in the same set of words.

6. The method according to claim 5, characterized in that, The purpose of training using the third loss term also includes reducing the similarity between different groups of words in the at least one set of words.

7. A method for acquiring training data, characterized in that, The method includes: Obtain the first text, which includes the third text and the first prompt after training. The third text belongs to the first dataset, and the first prompt after training is obtained through the first training operation. The first text is input into the machine learning model to obtain the second text output by the machine learning model. In the process of performing the second training operation on the student model using the teacher model in the manner of distillation, a second dataset is used, which includes multiple second texts. When performing the first training operation using the loss function, the parameters of the machine learning model and the teacher model are kept unchanged, and the first prompt is updated. The input of the teacher model is the output text of the machine learning model. The loss function is obtained based on the first prediction result output by the teacher model. The first prediction result includes a vector. The loss function includes a first loss term. The purpose of training using the first loss term is to make the vector closer to the one-hot form.

8. The method according to claim 7, characterized in that, The loss function further includes a second loss term, which indicates the information entropy of multiple first prediction results.

9. A training device for a model, characterized in that, The device includes: The input module is used to input the first text into the machine learning model to obtain the second text output by the machine learning model. The first text includes a third text and a first prompt. The third text belongs to the first dataset. The input module is also used to input the second text into the teacher model to obtain the first prediction result output by the teacher model; The training module is used to perform a first training operation based on the first prediction result until the convergence condition of the loss function is met. The first prediction result includes a vector, and the loss function includes a first loss term. The purpose of training using the first loss term is to make the vector closer to the one-hot form. During the execution of the first training operation, the parameters of the machine learning model and the teacher model are kept unchanged, and the first prompt is updated. The first prompt after training is used to obtain the second dataset. The training data in the second dataset will be used during the second training operation of the student model using the teacher model in the distillation method.

10. The apparatus according to claim 9, characterized in that, The loss function further includes a second loss term, which indicates the information entropy of multiple first prediction results.

11. The apparatus according to claim 9 or 10, characterized in that, The first prompt includes a prompt word and a second prompt. The prompt word means a summary. The second prompt is updated during the execution of the first training operation.

12. The apparatus according to claim 9 or 10, characterized in that, The machine learning model includes a normalized exponent softmax layer. During the process of the machine learning model processing the first text, the softmax layer in the machine learning model needs to process the first data obtained based on the first text. The process of processing the first data using the softmax layer includes: adding perturbation to the first data to obtain updated first data, and processing the updated first data using the softmax function.

13. The apparatus according to claim 9 or 10, characterized in that, The task of the teacher model is to obtain at least one type of information from the second text. The first prediction result includes at least one set of words that correspond one-to-one with the at least one type. The loss function also includes a third loss term. The purpose of training with the third loss term is to improve the similarity between different words in the same set of words.

14. The apparatus according to claim 13, characterized in that, The purpose of training using the third loss term also includes reducing the similarity between different groups of words in the at least one set of words.

15. A device for acquiring training data, characterized in that, The device includes: The acquisition module is used to acquire the first text, which includes the third text and the first prompt after training. The third text belongs to the first dataset, and the first prompt after training is obtained through the first training operation. The input module is used to input the first text into the machine learning model to obtain the second text output by the machine learning model. In the process of performing the second training operation on the student model using the teacher model in the distillation method, a second dataset is used, and the second dataset includes multiple second texts. When performing the first training operation using the loss function, the parameters of the machine learning model and the teacher model are kept unchanged, and the first prompt is updated. The input of the teacher model is the output text of the machine learning model. The loss function is obtained based on the first prediction result output by the teacher model. The first prediction result includes a vector. The loss function includes a first loss term. The purpose of training using the first loss term is to make the vector closer to the one-hot form.

16. The apparatus according to claim 15, characterized in that, The loss function further includes a second loss term, which indicates the information entropy of multiple first prediction results.

17. A training device, characterized in that, It includes a processor and a memory, wherein the processor is coupled to the memory. The memory is used to store programs; The processor is configured to execute a program in the memory, causing the training device to perform the method as described in any one of claims 1 to 6.

18. An execution device, characterized in that, It includes a processor and a memory, wherein the processor is coupled to the memory. The memory is used to store programs; The processor is configured to execute a program in the memory, causing the execution device to perform the method as described in claim 7 or 8.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 8.

20. A computer program product, characterized in that, The computer program product includes a program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 8.

Images