Fine-tuning method and device of generative model, equipment, medium and product

Abstract

The application discloses a fine-tuning method, device, equipment, medium and product of a generative model, and belongs to the field of artificial intelligence. The generative model is associated with a prompt word model, the network parameters of the prompt word model are less than the network parameters of a first part of network layers in the generative model, and the method comprises the following steps: inputting a prompt word into the prompt word model to obtain prompt feature representation; in the process of predicting a training sample by the generative model, inputting the prompt feature representation into the generative model as a guide prompt to obtain a prediction result of the training sample; and based on the prediction result, updating the network parameters of the prompt word model and the network parameters of a second part of network layers in the generative model, wherein the second part of network layers comprises n network layers after the first part of network layers in the generative model. According to the method, only the network parameters of the second part of network layers of the generative model and the network parameters of the prompt word model are updated, and the performance requirement of a computer device for fine-tuning the generative model can be reduced.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method, apparatus, device, medium, and product for fine-tuning generative models. Background Technology

[0002] Generative models are neural network models with capabilities such as natural language processing, image processing, or speech processing.

[0003] To apply a generative model to a specific domain, such as healthcare, gaming, or industry, training samples from that domain are typically used to fine-tune a pre-trained model applicable to a general domain, resulting in a generative model specifically designed for that domain. A pre-trained model refers to a neural network model that has already been trained on a large-scale training sample set.

[0004] To achieve better fine-tuning training results, related techniques employ a method of fine-tuning all network parameters of the generative model. However, since the number of network parameters in a generative model is typically in the hundreds of millions, fine-tuning all network parameters requires extremely high-configuration hardware, resulting in high costs. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and product for fine-tuning generative models, the technical solution of which is as follows:

[0006] According to one aspect of this application, a method for fine-tuning a generative model is provided, the generative model being associated with a cue word model, the cue word model having fewer network parameters than the first part of the network layers in the generative model, the method comprising:

[0007] Input the prompt words into the prompt word model to obtain the prompt feature representation;

[0008] During the process of the generative model predicting training samples, the prompt feature representation is used as a guiding prompt input into the generative model to obtain the prediction result of the training samples;

[0009] Based on the prediction results, the network parameters of the prompt word model and the network parameters of the second part of the network layer in the generative model are updated. The second part of the network layer includes n network layers in the generative model that are located after the first part of the network layer, where n is a positive integer.

[0010] According to one aspect of this application, a fine-tuning apparatus for a generative model is provided, the generative model being associated with a cue word model, the cue word model having fewer network parameters than the network parameters of a first part of the network layers in the generative model, the apparatus comprising:

[0011] The acquisition module is used to input the prompt words into the prompt word model to obtain the prompt feature representation;

[0012] The prediction module is used to input the prompt feature representation as a guiding prompt into the generative model during the process of the generative model predicting training samples, so as to obtain the prediction result of the training samples.

[0013] The update module is used to update the network parameters of the prompt word model and the network parameters of the second part of the network layer in the generative model based on the prediction results. The second part of the network layer includes n network layers in the generative model that are located after the first part of the network layer, where n is a positive integer.

[0014] According to one aspect of this application, a computer device is provided, the computer device comprising: a processor and a memory, the memory storing at least one program; the processor being configured to execute the at least one program in the memory to implement the above-described method for fine-tuning the generative model.

[0015] According to one aspect of this application, a computer-readable storage medium is provided, wherein executable instructions are stored therein, the executable instructions being loaded and executed by a processor to implement the above-described method for fine-tuning the generative model.

[0016] According to one aspect of this application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium, wherein a processor reads from the computer-readable storage medium and executes the computer instructions to implement the above-described method for fine-tuning the generative model.

[0017] The beneficial effects of the technical solution provided in this application include at least the following:

[0018] In the fine-tuning process of the generative model, only the network parameters of the second part of the network layer and the network parameters of the prompt word model need to be updated. The network parameters of the first part of the network layer are frozen and not updated, while the network parameters of the prompt word model are less than or even far less than the network parameters of the first part of the network layer. That is, compared with updating all network parameters of the generative model in related technologies, the method provided by this application requires less overhead and has low performance requirements for the computer equipment used to fine-tune the generative model. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 An architectural diagram of a computer system provided in an exemplary embodiment of this application is shown;

[0021] Figure 2 A flowchart illustrating a method for fine-tuning a generative model provided in an exemplary embodiment of this application is shown;

[0022] Figure 3 A schematic diagram of a method for fine-tuning a generative model provided in an exemplary embodiment of this application is shown;

[0023] Figure 4 A flowchart illustrating a method for fine-tuning a generative model provided in an exemplary embodiment of this application is shown;

[0024] Figure 5 A flowchart illustrating a method for fine-tuning a generative model provided in an exemplary embodiment of this application is shown;

[0025] Figure 6 A flowchart illustrating a method for fine-tuning a generative model provided in an exemplary embodiment of this application is shown;

[0026] Figure 7 A schematic diagram of a method for fine-tuning a generative model provided in an exemplary embodiment of this application is shown;

[0027] Figure 8 A flowchart illustrating a method for fine-tuning a generative model provided in an exemplary embodiment of this application is shown;

[0028] Figure 9 A flowchart illustrating a method for fine-tuning a generative model provided in an exemplary embodiment of this application is shown;

[0029] Figure 10 A schematic diagram of a method for fine-tuning a generative model provided in an exemplary embodiment of this application is shown;

[0030] Figure 11 A flowchart illustrating a method for fine-tuning a generative model provided in an exemplary embodiment of this application is shown;

[0031] Figure 12 A flowchart illustrating a method for fine-tuning a generative model provided in an exemplary embodiment of this application is shown;

[0032] Figure 13 A flowchart illustrating a generative model-based prediction method provided in an exemplary embodiment of this application is shown.

[0033] Figure 14 A structural block diagram of a fine-tuning apparatus for a generative model provided in an exemplary embodiment of this application is shown.

[0034] Figure 15 A schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0036] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0037] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0038] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the settings and operation information involved in this application were obtained with full authorization.

[0039] It should be understood that although the terms first, second, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, a first parameter may also be referred to as a second parameter without departing from the scope of this disclosure, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0040] First, let me introduce the relevant terms used in this application:

[0041] Pre-trained models (PTMs), also known as foundational models or large models, refer to deep neural networks (DNNs) with a large number of parameters. These DNNs are trained on massive amounts of unlabeled data. Leveraging the function approximation capabilities of large-parameter DNNs, PTMs extract common features from the data and employ techniques such as fine-tuning, parameter-efficient fine-tuning (PEFT), and prompt-tuning to adapt them for downstream tasks. Therefore, pre-trained models can achieve ideal results in small-shot or zero-shot scenarios. PTMs can be categorized according to the data modality they process, including language models (ELMO, BERT, GPT), visual models (swin-transformer, ViT, V-MOE), speech models (VALL-E), and multimodal models (ViBERT, CLIP, Flamingo, Gato). Multimodal models refer to models that represent features from two or more data modalities. Pre-trained models are important tools for outputting AI-generated content and can also serve as a general interface for connecting multiple specific task models.

[0042] Generative models are models with capabilities in natural language processing, image processing, or speech processing, and the ability to generate content such as text, images, audio, and video. Based on the content generated, generative models can be categorized as follows: text generation models, image generation models, audio generation models, video generation models, and 3D model generation models.

[0043] Prompt: Information used to guide a generative model to perform a specified task; it can also be simply called a prompt. Typically, prompts are represented in text form, meaning they guide the generative model to perform a specified task, such as text generation, image generation, audio generation, or video generation. For example, for a text generation task, the prompt might be "Generate a summary of the above text" or "Generate a story based on the above text."

[0044] Figure 1 An architectural diagram of a computer system provided in an exemplary embodiment of this application is shown. The computer system includes: computer device 110.

[0045] Optionally, the computer device 110 can be at least one of a portable computer, a desktop computer, a server, a server cluster, an artificial intelligence (AI) computing cluster, or a cloud computing cluster. The AI ​​computing cluster can also be simply referred to as an intelligent computing cluster or a smart computing cluster.

[0046] Optionally, the computer device 110 deploys a generative model 10, which is associated with a cue word model 20. The computer device 110 executes a fine-tuning method for the generative model 10. During the fine-tuning process, the computer device 110 inputs cue words into the cue word model 20 to obtain cue feature representations. During the prediction of training samples by the generative model 10, the cue feature representations are used as guiding cue inputs to the generative model 10 to obtain prediction results for the training samples. Based on the prediction results, the network parameters of the cue word model 20 and the network parameters of the second part of the network layer in the generative model 10 are updated. The second part of the network layer includes n network layers following the first part of the network layer in the generative model, where n is a positive integer. The network parameters of the cue word model are fewer than the network parameters of the first part of the network layer in the generative model. During the update process of the second part of the network layer in the prompt word model 20 and the generative model 10, the model loss 9 is obtained based on the prediction results. The model loss 9 will be backpropagated according to the network parameters to be updated. During the backpropagation process, the model loss 9 will flow from the second part of the network layer to the prompt word model without passing through the first part of the network layer, that is, the network parameters of the first part of the network layer are not updated or are frozen.

[0047] Figure 2 A flowchart illustrating a fine-tuning method for a generative model provided in an exemplary embodiment of this application is shown. The generative model is associated with a cue word model, the cue word model having fewer network parameters than the first part of the network layers in the generative model. This method can be derived from the above... Figure 1 The method is performed by the computer device shown. The method includes:

[0048] In this context, the generative model is associated with the cue word model, meaning that the generative model and the cue word model are trained and used together. This can be understood as a connection between the generative model and the cue word model; for example, the output of the cue word model will be input into the generative model. Alternatively, the cue word model can be described as a trainable neural network added to the generative model.

[0049] Step 210: Input the prompt words into the prompt word model to obtain the prompt feature representation.

[0050] In other words, the prompt word model is used to extract prompt feature representations from prompt words. Optionally, prompt feature representations can also be called prompt embeddings, prompt vectors, prompt feature vectors, etc.

[0051] Optionally, the prompt word model is a first pre-trained model; or, the network parameters in the prompt word model are obtained through initialization. Here, "pre-trained model" means that the prompt word model uses a pre-trained model, which is used to extract prompt feature representations from the prompt words. The pre-trained model can also be called a general model. Initialization methods include random initialization, heuristic initialization, zero initialization, and constant initialization. Random initialization involves sampling data from a fixed distribution to initialize the network parameters, such as a normal distribution or a uniform distribution. Heuristic initialization involves initializing the network parameters of the prompt word model based on the features of the activation function used during training. Zero initialization involves initializing all network parameters in the prompt word model to zero. Constant initialization is similar to zero initialization, but initializes all network parameters in the prompt word model to a non-zero constant.

[0052] It should be noted that, compared to obtaining the network parameters in the prompt word model through initialization, using the first pre-trained model as the prompt word model can effectively improve the fine-tuning efficiency of the generative model. Therefore, this application embodiment will use the first pre-trained model as the prompt word model for illustration, but it is not limited thereto.

[0053] Step 220: During the process of the generative model predicting training samples, the cue feature representation is used as a guiding cue input to the generative model to obtain the prediction results of the training samples.

[0054] Optionally, the generative model is a second pre-trained model.

[0055] Optionally, during the process of the generative model predicting training samples, the generative model obtains prediction results based on the cue feature representation and the training samples. The cue feature representation is used to guide the generative model to generate prediction results for the target domain and task content indicated by the cue feature representation.

[0056] Optionally, the sample feature representation of the training samples is extracted by the generative model. That is, the cue feature representation and the training samples are input into the generative model. The generative model first extracts sample feature representations based on the training samples, and then obtains the prediction result based on the cue feature representation and the sample feature representation. In other words, the generative model can be understood to also include a feature extraction layer (also called an embedding layer, encoding layer, etc.), which is used to extract sample feature representations based on the training samples. Alternatively, the sample feature representation of the training samples is extracted by the feature extraction model. That is, the training samples are input into the feature extraction model to obtain sample feature representations, and then the cue feature representation and the sample feature representation are input into the generative model to obtain the prediction result. Alternatively, the generative model is an encoder-decoder structure. The training samples are input into the encoder to obtain sample feature representations, and the network layer is a decoder structure. The cue feature representation and the sample feature representation are input into the decoder to obtain the prediction result. The following explanation will focus on the extraction of sample feature representations by the generative model, but the scope of this application is not limited to this.

[0057] Optionally, when the output information of the network layer of the generative language model is the information of the prediction result, the generative model also includes an output layer. The output layer is used to obtain the prediction result based on the information of the prediction result. Here, the information of the prediction result can be understood as the identifier of the prediction result. For example, if the output information is a sequence of text identifiers, the output layer is used to obtain the text sequence based on each text identifier in the text identifier sequence.

[0058] Optionally, the training samples can be at least one of text, images, audio, and video. The form of the training samples is determined according to the task of the generative model. When the training samples are text, the sample feature representation can be called token feature representation, token embedding, etc.

[0059] For example, the generative model's task may be machine translation, where the training samples are the text to be translated and the prompts are the target language; for example, the prompt might be "Translate the following sentences into Chinese"; or, the generative model's task may be image style transfer, where the training samples are images with a fixed style and the prompt might be "Generate an image with the same style as the input image"; or, the generative model's task may be audio generation, where the training samples are reference audio clips and the prompt might be "Generate the complete audio based on the input audio clip"; or, the generative model's task may be video prediction, where the training samples are video clips to be predicted and the prompt might be "Generate the next frame of the video based on the input video clip," and so on.

[0060] Step 230: Based on the prediction results, update the network parameters of the prompt word model and the network parameters of the second part of the network layer in the generative model. The second part of the network layer includes n network layers in the generative model that are located after the first part of the network layer, where n is a positive integer.

[0061] Optionally, the generative model includes multiple network layers, which can be divided into a first part network layer and a second part network layer, where the second part network layer consists of the last n layers in the multi-layer network. For example, the generative model includes m network layers, where the first part network layer includes layers 1 to mn, and the second part network layer includes layers m-n+1 to m. Specifically, taking m=28 and n=2 as an example, the generative model includes 28 network layers, where the first part network layer includes layers 1 to (28-2=)26, and the second part network layer includes (28-2+1=)27 and 28.

[0062] Optionally, the network layer is used to generate prediction results based on the cue feature representation and the sample feature representation; or, the network layer is used to generate output information based on the cue feature representation and the sample feature representation. The generative model also includes an output layer, which is used to generate prediction results based on the output information. The output information can be represented as an output sequence, output feature representation, etc.

[0063] Optionally, during the update process of the network parameters of the prompt word model and the network parameters of the second part of the network layer in the generative model, the network parameters of the first part of the network layer in the generative model are frozen; or, the network parameters of the generative model other than those in the second part of the network layer are frozen.

[0064] In summary, the method provided in this application only requires updating the network parameters of the second part of the network layer and the network parameters of the prompt word model during the fine-tuning process of the generative model. The network parameters of the first part of the network layer are frozen and not updated, while the network parameters of the prompt word model are fewer than, or even far fewer than, the network parameters of the first part of the network layer. That is, compared to updating all network parameters of the generative model in related technologies, the method provided in this application requires less overhead and has lower performance requirements for the computer equipment used to fine-tune the generative model. Furthermore, in addition to the network layers of the generative model itself, a prompt word model is associated with the generative model. The prompt word model is used to extract prompt feature representations from the prompt words. The prompt words can guide the generative model to generate content for a specified task. Associating the prompt word model with the generative model and updating its network parameters during the fine-tuning process enables the prompt word model to learn how to extract prompt feature representations to guide the generative model in generating content for a specified task, without requiring the generative model to generate content based on an unknown task. This saves the generative model's exploration time for unknown tasks and accelerates the fine-tuning process of the generative model.

[0065] First, let's further explain the above-mentioned prompts.

[0066] Optionally, the prompt words are constructed based on the task content of the target domain, which is the domain in which the fine-tuned generative model is applied.

[0067] For example, if the target domain is the medical field and the task content is medical question answering, then the prompt words are constructed based on the medical question answering task in the medical field; or, if the target domain is the natural language processing field and the task content is text translation, then the prompt words are constructed based on the text translation task in the natural language processing field; or, if the target domain is the art field and the task content is image generation, then the prompt words are constructed based on the image generation task in the art field; or, if the target domain is the game field and the task content is 3D model generation, then the prompt words are constructed based on the 3D model generation task in the game field.

[0068] For example:

[0069] <Training Sample>: I feel dizzy, nauseous, and have a slight fever. What medicine should I take to make myself feel better?

[0070] <Prompt words>: Drug 1, Drug 2, Drug 3.

[0071] For example:

[0072] <Prompt>: The following is a communication technology document. Please translate it into Chinese, ensuring the translation is accurate, fluent, and elegant. Also, if you encounter the following terms, please follow the provided translation instructions:

[0073] UE: User terminal; eNB: Evolved base station; MAC: Media access control.

[0074] <Training Samples> are English excerpts from technical documents in the field of communications.

[0075] For example:

[0076] <Hint>: Please refer to the image style shown below and draw a puppy image in the same style.

[0077] The training sample is a realistic image.

[0078] It could also be:

[0079] <Prompt>: Please refer to the character image shown below to generate its corresponding 3D model. The overall style leans towards anime characters.

[0080] The training sample is a human image.

[0081] Among them, the generative model is guided to learn the generation of task-related content in the target domain based on prompt words, which eliminates the need for the generative model to generate based on unknown tasks, saving the exploration time of the generative model for unknown tasks and thus accelerating the fine-tuning process of the generative model.

[0082] Regarding the aforementioned "the network parameters of the prompt word model are fewer than the network parameters of the first part of the network layer in the generative model," optionally, the network parameters of the prompt word model are at least one order of magnitude fewer than the network parameters of the first part of the network layer.

[0083] In this context, one order of magnitude typically refers to 10 times, two orders of magnitude to 100 times, three orders of magnitude to 1000 times, and so on, with n orders of magnitude referring to 10 n .

[0084] Optionally, the number of network parameters in the prompt word model is less than the number of network parameters in the first part of the generative model; more specifically, the number of network parameters in the prompt word model is less than the number of network parameters in a single layer of the generative model. Since the number of network parameters in the prompt word model is very small, and far less than the number of network parameters in the first part of the generative model, updating the network parameters of the prompt word model during the fine-tuning of the generative model will not significantly increase the load on the computer. In other words, the prompt word model can both accelerate the fine-tuning efficiency of the generative model and avoid increasing the load on the computer.

[0085] The following section shows the specific training process for the prompt word model and the generative model.

[0086] 1. The generative model is associated with a prompt word model.

[0087] The following section uses a generative model consisting of m network layers, one feature extraction layer, and one output layer as an example to detail the fine-tuning process of a generative model. Figure 3 As shown, the generative model 10 includes a feature extraction layer 11, an m-layer network, and an output layer 12. The feature extraction layer 11 is used to extract sample feature representations 14 from the input training samples 13. The m-layer network in the generative model 10 can be divided into a first part network layer and a second part network layer. The first part network layer consists of layers 1 to k, and the second part network layer consists of layers (k+1) to m. The second part network layer has n layers, i.e., mk = n, where n, m, and k are all positive integers. The generative model 10 is also associated with a cue word model 20, which is used to extract cue feature representations 22 from cue words 21.

[0088] In the scenario where a generative model is associated with a prompt word model, the following section explains in detail how the generative model obtains the prediction result, where m is a positive integer greater than n. For example... Figure 4As shown, step 220 can be implemented as steps 310 to 330.

[0089] Step 310: For the first network layer in the m-layer network, input the cue feature representation and the sample feature representation of the training samples into the first network layer to obtain the first output information.

[0090] The method for obtaining the sample feature representation of the training samples can be found in step 220 above. Here, we will use the example of the sample feature representation being extracted by the feature extraction layer 11 to illustrate the process.

[0091] Optionally, for the first network layer in the m-layer network, the cue feature representation and the sample feature representation of the training samples are concatenated and then input into the first network layer to obtain the first output information. Alternatively, for the first network layer in the m-layer network, the cue feature representation and the sample feature representation of the training samples are input into the first network layer separately, such as inputting the cue feature representation first and then inputting the sample feature representation of the training samples, or inputting the sample feature representation of the training samples first and then inputting the cue feature representation.

[0092] Optionally, the first network layer in the m-layer network receives the concatenated cue feature representation and sample feature representation, and obtains the first output information based on the concatenated cue feature representation and sample feature representation. The first network layer then outputs the first output information to the second network layer. Specifically, each network layer in the m-layer network outputs the output information obtained in that layer to the next network layer.

[0093] Optionally, the prompt feature representation and the sample feature representation of the training sample can be concatenated in the following ways: concatenating the prompt feature representation before the sample feature representation of the training sample; or concatenating the prompt feature representation after the sample feature representation of the training sample; or concatenating the prompt feature representation in the sample feature representation of the training sample; or concatenating the prompt feature representation after splitting it into the sample feature representation of the training sample.

[0094] For example, let's take the sample feature representation of the training sample as "[S][S][S][S][S]" and the prompt feature representation as "[P][P][P]" as an example. Concatenating the prompt feature representation before the sample feature representation of the training sample results in the concatenated prompt feature representation and sample feature representation (hereinafter referred to as the concatenated feature representation): [P][P][P][S][S][S][S][S]. Concatenating the prompt feature representation after the sample feature representation of the training sample results in the concatenated feature representation: [S][S][S][S][S][P][P][P]. Concatenating the prompt feature representation into the sample feature representation of the training sample results in the concatenated feature representation: [S][S][P][P][P][S][S][S]. Concatenating the prompt feature representation after splitting it into the sample feature representation of the training sample results in the concatenated feature representation: [S][P][S][P][S][P][S][S]. Where [·] represents a sequence element.

[0095] It should be noted that the size, content, and form of the sample feature representations and prompt feature representations of the training samples described above are for illustrative purposes only. In practical applications, feature representations can also be represented as vectors, matrices, tensors, etc. The method of concatenating the prompt feature representations and the sample feature representations of the training samples varies depending on the form of the feature representations, and will not be elaborated here, but the protection scope of this application embodiment is not limited thereto.

[0096] Step 320: For the (i+1)th network layer in the m-layer network, input the cue feature representation and the i-th output information into the (i+1)th network layer to obtain the (i+1)-th output information. The i-th output information is output by the i-th network layer, where i is a positive integer less than m.

[0097] Optionally, for the (i+1)th network layer in the m-layer network, the cue feature representation and the ith output information are concatenated and then input into the (i+1)th network layer to obtain the (i+1)th output information. Alternatively, for the (i+1)th network layer in the m-layer network, the cue feature representation and the ith output information are input into the (i+1)th network layer separately to obtain the (i+1)th output information, such as inputting the cue feature representation first and then the ith output information, or inputting the ith output information first and then the cue feature representation.

[0098] Optionally, layers 2 through 1 in the m-th network layer receive the concatenated cue feature representation and the i-th output information, and obtain the (i+1)-th output information based on the concatenated cue feature representation and the i-th output information. Layers 2 through 1 also output the i-th output information to the (i+2)-th network layer. The m-th network layer outputs the m-th output information to the output layer.

[0099] Optionally, for generative models that do not include an output layer, the m-th network layer is also used to obtain the prediction result based on the m-th output information. That is, after the m-th network layer obtains the m-th output information based on the concatenated cue feature representation and the (m-1)-th output information, it will not output the m-th output information, but will further obtain the prediction result based on the m-th output information and then output the prediction result.

[0100] Step 330: Obtain the prediction result of the training sample based on the m-th output information.

[0101] Optionally, the generative model obtains the prediction result of the training sample based on the m-th output information; or, the output layer in the generative model obtains the prediction result of the training sample based on the m-th output information.

[0102] In summary, the method provided in this application embodiment inputs the cue feature representation into each layer of the generative model's network layer, so that the cue feature representation affects the output information of each layer of the generative model's network layer. This ensures that the cue feature representation can guide each layer of the generative model to generate content for a specified task, thereby improving the fine-tuning efficiency and adaptability of the generative model to the target domain.

[0103] The following section details how the network parameters of the prompt word model and the network parameters of the second part of the generative model are updated based on the prediction results. For example... Figure 5 As shown, step 230 can be implemented as steps 340 to 370.

[0104] Step 340: Calculate the model loss based on the prediction results.

[0105] Optionally, the training samples have corresponding real samples, sample labels, or expected results. The model loss can be calculated based on the predicted result and the real sample, or based on the predicted result and the sample label, or based on the predicted result and the expected result. For example, for a generative model in a question-answering task in the medical field, the training samples are questions, and the expected result is the expected answer. The prediction result of the generative model is the predicted answer generated by the generative model based on the training samples (i.e., the question) and the prompt words. The model loss can be calculated based on the error between the expected answer and the predicted answer.

[0106] The loss function used in calculating the model loss can be cross-entropy loss, mean squared error, mean absolute error, KL divergence, etc. This application does not limit this specific method.

[0107] Step 350: Calculate the gradient of the m-th layer based on the model loss; based on the gradient of the m-th layer, update the network parameters of the m-th network layer in the generative model, and the gradient of the m-th layer is the gradient of the m-th network layer.

[0108] Optionally, the gradient refers to the partial derivative of the loss function of the generative model with respect to the network parameters of the network layer. That is, the gradient of the m-th layer can be understood as a vector whose components are the partial derivatives of the loss function of the generative model with respect to each network parameter in the m-th network layer. Network parameters include at least one of network weights and network biases. Network weights are parameters that connect the input and neurons in a neural network, representing the degree of influence of the input on the neuron's output. Network weights determine the transmission strength of the signal (which can be understood as a form of representation of the concatenated cue feature representation and sample feature representation in the network layer) between neurons. Network bias is a constant term added to the input of the neuron's activation function, used to provide a threshold offset for neuron activation. If a neuron is represented as f(wx+b), then f(·) is the activation function, w is the weight, x is the signal input to the neuron, and b is the bias. The neuron is used to transform the input (wx+b) into the output f(wx+b).

[0109] The above phrase "update the network parameters of the m-th network layer in the generative model based on the gradient of the m-th layer" can be understood as "update the network weights and / or network biases of the m-th network layer in the generative model based on the gradient of the m-th layer".

[0110] That is, the gradient of the m-th layer includes the gradient corresponding to the weight of the m-th layer network and / or the gradient corresponding to the bias of the m-th layer network.

[0111] The update of network parameters in both the prompt word model and the generative model employs a backpropagation algorithm, also known as error backpropagation. This involves propagating the model error back to the input layer (the (k+1)th layer in either the prompt word model or the generative model) based on the network parameters in both models. During this backpropagation, the network parameters of both the prompt word model and the second part of the generative model's network layer need to be updated. These updates can be based on the partial derivatives of the model error with respect to the network parameters, i.e., the gradient. The goal of updating these parameters is to ensure that the updated network parameters generate predictions with smaller model errors, i.e., updating the network parameters in the direction of decreasing model error. From a gradient perspective, this means updating the network parameters in the direction of gradient descent. Therefore, the update of the network parameters in both the prompt word model and the generative model can be considered gradient-based. Furthermore, for adjacent network layers, the gradient of the i-th layer can be calculated based on the gradient of the (i+1)-th layer.

[0112] Specifically, network parameters can be updated using different optimizers. An optimizer is an algorithm for updating network parameters. Common optimizers include gradient descent and stochastic gradient descent.

[0113] Step 360: Calculate the gradient of the i-th layer based on the gradient of the (i+1)-th layer; based on the gradient of the i-th layer, update the network parameters of the i-th network layer in the generative model. The gradient of the (i+1)-th layer is the gradient of the (i+1)-th network layer, and the gradient of the i-th layer is the gradient of the i-th network layer, where i is an integer greater than k and less than m.

[0114] Specifically, the gradient of layer i is calculated based on the gradient of layer i+1 and the network parameters of layer i. Alternatively, the gradient of layer i is calculated based on the gradient of layer i+1 and the activation function of layer i. For example, the gradient of layer i is equal to the product of the gradient of layer i+1 and the derivative of the activation function of layer i.

[0115] Optionally, for layers m-1 to k+1 in an m-layer network, the gradient of layer i is calculated based on the gradient of layer i+1; and the network parameters of layer i are updated based on the gradient of layer i.

[0116] Optionally, for layers k through 1 in the m-layer network, i.e., for the first part of the network layers in the generative model, gradients are not calculated. That is, the network parameters of the first part of the network layers in the generative model are not updated.

[0117] Optionally, the gradient of the i-th layer is calculated based on the gradient of the (i+1)-th layer. Alternatively, the gradient of the i-th layer can be calculated based on the model loss.

[0118] Step 370: Update the network parameters of the prompt word model based on the gradient of the m-th layer or the gradient of the (k+1)-th layer.

[0119] Since the cue feature representation is fed into the m-th network layer by layer, the network parameters of the cue word model can be updated based on the gradient of any network layer from the (k+1)-th to the m-th network layer.

[0120] Optionally, based on the gradient of the i-th layer, update the network parameters of the prompt word model, where i is an integer greater than k and less than m+1.

[0121] In some embodiments, step 370 can be performed after "calculating the gradient of the m-th layer based on the model loss" in step 350 and before "updating the network parameters of the m-th network layer in the generative model based on the gradient of the m-th layer", or step 370 can be performed between steps 350 and 360, or step 370 can be performed together with "updating the network parameters of the m-th network layer in the generative model based on the gradient of the m-th layer" in step 350, or step 370 can be performed together with step 360.

[0122] In summary, the method provided in this application illustrates the specific update process of the network parameters of the second part of the generative model's network layer and the network parameters of the prompt word model. Specifically, the model loss is calculated based on the prediction results, and then the backpropagation algorithm is used to propagate the partial derivatives (i.e., gradients) of the model's loss function relative to the network parameters layer by layer, updating the network parameters of each layer based on these gradients. During the update process, the network parameters of the first part of the network layer are frozen, meaning the gradients of the first part of the network layer are not calculated. This reduces the computational load during the fine-tuning process of the generative model, thus reducing the performance requirements of the computer equipment. When fine-tuning on computer equipment with the same configuration, compared to the fine-tuning method used in related technologies that updates all network parameters, this method improves the fine-tuning efficiency of the generative model to a certain extent.

[0123] 2. Generative models are associated with at least two prompt word models.

[0124] For scenarios where the generative model is associated with at least two prompt word models, such as Figure 6 As shown, step 210 can be implemented as step 410, step 220 can be implemented as step 420, and step 230 can be implemented as step 430.

[0125] Step 410: Input the prompt word into each of the at least two prompt word models to obtain at least two prompt feature representations.

[0126] Optionally, each of the at least two prompt word models uses the same pre-trained model, i.e., the first pre-trained model; or, each of the at least two prompt word models uses the same initialization method; or, each of the at least two prompt word models uses a different pre-trained model; or, each of the at least two prompt word models uses a different initialization method. When each of the at least two prompt word models uses the same initialization method, the network parameters of each of the at least two prompt word models may be the same or different. For example, with zero initialization, the network parameters of each of the at least two prompt word models are the same; with constant initialization, random initialization, and heuristic initialization, the network parameters of each of the at least two prompt word models can be the same or different.

[0127] The specific process of “inputting the prompt word into each of the at least two prompt word models to obtain at least two prompt feature representations” can be found in step 210 above, and will not be repeated here.

[0128] Step 420: Input at least two cue feature representations into different network layers in the first part of the generative model and / or the second part of the network layer; and input the sample feature representations of the training samples into the first network layer of the generative model to obtain the prediction results of the training samples.

[0129] Optionally, at least two cue feature representations are respectively input into different network layers in the first part of the generative model and / or the second part of the network layer; this can be understood as inputting at least two cue feature representations respectively into the first part of the network layer and / or the second part of the network layer in the generative model.

[0130] For example, at least two cue features can be represented as two cue feature representations, where the first cue feature representation is input to each layer of the first part of the network layer, and the second cue feature representation is input to each layer of the second part of the network layer. Alternatively, at least two cue feature representations can be represented as n+1 cue feature representations, where the first cue feature representation is input to each layer of the first part of the network layer, and the second to (n+1)th cue feature representations are respectively input to each layer of the second part of the network layer (n layers of the network layer).

[0131] Optionally, adjacent network layers in the m-layer network are connected, meaning that the output information of the i-th network layer will be transmitted to the (i+1)-th network layer. Therefore, representing the sample features of the training samples as input to the first network layer of the generative model can be understood as representing the sample features of the training samples as input to the m-layer network layer of the generative model.

[0132] The specific process of “inputting at least two cue feature representations into different network layers in the first part of the generative model and / or the second part of the network layer; and inputting the sample feature representations of the training samples into the first network layer of the generative model to obtain the prediction results of the training samples” can be found in step 220 above, and will not be repeated here.

[0133] Step 430: Based on the prediction results, update the network parameters of at least two prompt word models and the network parameters of the second part of the network layer in the generative model.

[0134] The specific process of "updating the network parameters of at least two prompt word models and the network parameters of the second part of the network layer in the generative model based on the prediction results" can be found in step 230 above, and will not be repeated here.

[0135] In summary, the method provided in this application associates at least two prompt word models with a generative model. Each prompt word model is associated with a different network layer in the generative model, enabling the prompt word model to generate prompt feature representations that are easier for that network layer to understand, based on the capabilities of that network layer. Due to the characteristics of the backpropagation algorithm, generally speaking, later network layers are closer to the target domain for fine-tuning, while earlier network layers are closer to a general domain. Therefore, setting different prompt word models for network layers with different capabilities can improve the guiding ability of prompt feature representations to network layers, thereby improving the fine-tuning efficiency of the generative model.

[0136] The following sections illustrate two typical scenarios where a generative model is associated with at least two cue word models: one where the generative model is associated with two cue word models, and another where the generative model is associated with n+1 cue word models.

[0137] It should be noted that the number of prompt word models associated with the generative model, and how to design the connection relationship between the prompt word model and the m-layer network layer for different numbers of prompt word models, so as to fine-tune the relevant content of the generative model according to the connection relationship, will not be listed one by one in the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto.

[0138] 2.1 Generative model associates two prompt word models.

[0139] Optionally, the generative model is associated with a first prompt word model and a second prompt word model; the generative model consists of m network layers, where m is a positive integer greater than n; the first part of the network layers consists of layers 1 to k in the m network layers, and the second part of the network layers consists of layers k+1 to m in the m network layers. The model structure in this scenario is as follows: Figure 7 As shown, the generative model 10 includes a feature extraction layer 11, an m-layer network, and an output layer 12. The feature extraction layer 11 is used to extract sample feature representations 14 from the input training samples 13. The m-layer network in the generative model 10 can be divided into a first part network layer and a second part network layer. The first part network layer consists of layers 1 to k, and the second part network layer consists of layers k+1 to m. The second part network layer has n layers, i.e., mk = n, where n, m, and k are all positive integers. The generative model 10 is also associated with a first prompt word model 30 and a second prompt word model 40. The first prompt word model 30 is used to extract a first prompt feature representation 31 based on the prompt word 21, and the second prompt word model is used to extract a second prompt feature representation 41 based on the prompt word 21.

[0140] First, let's explain how generative models obtain prediction results. For example... Figure 8 As shown, step 410 can be implemented as step 510, and step 420 can be implemented as steps 520 to 550.

[0141] Step 510: Input the prompt word into the first prompt word model to obtain the first prompt feature representation; and input the prompt word into the second prompt word model to obtain the second prompt feature representation.

[0142] Optionally, the first prompt word model and the second prompt word model are the same pre-trained model, i.e., the first pre-trained model; or, the first prompt word model and the second prompt word model use the same initialization method; or, the first prompt word model and the second prompt word model are different pre-trained models; or, the first prompt word model and the second prompt word model use different initialization methods.

[0143] The specific process of "inputting the prompt words into the first prompt word model and the second prompt word model to obtain the first prompt feature representation and the second prompt feature representation" can be found in steps 210 and 410, and will not be repeated here.

[0144] Step 520: For the first network layer in the m-layer network, input the first prompt feature representation and the sample feature representation of the training samples into the first network layer to obtain the first output information.

[0145] Optionally, for the first network layer in the m-layer network, the first prompt feature representation and the sample feature representation are concatenated and then input into the first network layer to obtain the first output information. Alternatively, for the first network layer in the m-layer network, the first prompt feature representation and the sample feature representation are input into the first network layer separately, such as inputting the first prompt feature representation first and then inputting the sample feature representation, or inputting the sample feature representation first and then inputting the first prompt feature representation.

[0146] The specific process of “inputting the first prompt feature representation and the sample feature representation of the training samples into the first network layer to obtain the first output information” can be found in step 310 above, and will not be repeated here.

[0147] Step 530: For the (i+1)th network layer in the m-layer network, input the first prompt feature representation and the i-th output information into the (i+1)th network layer to obtain the (i+1)-th output information. The i-th output information is output by the i-th network layer, where i is a positive integer less than k.

[0148] Optionally, for the (i+1)th network layer in the m-layer network, the first prompt feature representation and the ith output information are concatenated and then input into the (i+1)th network layer to obtain the (i+1)th output information. Alternatively, for the (i+1)th network layer in the m-layer network, the first prompt feature representation and the ith output information are input into the (i+1)th network layer respectively to obtain the (i+1)th output information, such as inputting the first prompt feature representation first and then inputting the ith output information, or inputting the ith output information first and then inputting the first prompt feature representation.

[0149] Optionally, the first part of the m-layer network, or in other words, the second to the k-th layers of the m-layer network, receives the concatenated first cue feature representation and the i-th output information, and obtains the (i+1)-th output information based on the concatenated first cue feature representation and the i-th output information. Furthermore, the second to the k-th layers also output the (i+1)-th output information to the (i+2)-th layer.

[0150] The specific process of “inputting the first prompt feature representation and the i-th output information into the (i+1)-th network layer to obtain the (i+1)-th output information” can be found in step 320 above, and will not be repeated here.

[0151] Step 540: For the (j+1)th network layer in the m-layer network, input the second prompt feature representation and the j-th output information into the (j+1)th network layer to obtain the (j+1)-th output information. The j-th output information is output by the j-th network layer, where j is a positive integer greater than k-1 and less than m.

[0152] Optionally, for the (j+1)th network layer in the m-layer network, the second prompt feature representation and the j-th output information are concatenated and then input into the (j+1)th network layer to obtain the (j+1)th output information; or, for the (j+1)th network layer in the m-layer network, the second prompt feature representation and the j-th output information are input into the (j+1)th network layer respectively to obtain the (j+1)th output information, such as inputting the second prompt feature representation first and then inputting the j-th output information, or inputting the j-th output information first and then inputting the second prompt feature representation.

[0153] Optionally, the second part of the m-layer network, or in other words, the (k+1)th to the m-th layers, receives the concatenated second cue feature representation and the j-th output information, and obtains the (j+1)-th output information based on the concatenated second cue feature representation and the j-th output information. For the (k+1)th to (m-1)th layers, the (j+1)-th output information is also output to the (j+2)-th layer. For the m-th layer, the m-th output information is output to the output layer.

[0154] The specific process of “inputting the second prompt feature representation and the j-th output information into the (j+1)-th network layer to obtain the (j+1)-th output information” can be found in step 320 above, and will not be repeated here.

[0155] Step 550: Obtain the prediction result of the training sample based on the m-th output information.

[0156] Optionally, the generative model obtains the prediction result of the training sample based on the m-th output information; or, the output layer in the generative model obtains the prediction result of the training sample based on the m-th output information.

[0157] In summary, the method provided in this application's embodiments inputs a first cue feature representation into each layer of the first part of the generative model's network layer, and a second cue feature representation into each layer of the second part of the generative model's network layer. This is because during the fine-tuning of the generative model, the network parameters of the first part of the network layer are not updated (i.e., frozen), meaning the first part of the network layer does not learn the features of the training samples and cue words, while the second part of the network layer does. Therefore, during the fine-tuning of the generative model, the first and second part of the network layer may have different understandings of the same cue feature representation. Therefore, separate cue word models are set for them, thereby generating cue feature representations specifically based on the capabilities of the first and second part of the network layer, enabling both to better understand and output more accurate information, thus improving the fine-tuning efficiency and adaptability of the generative model to the target domain.

[0158] The following section details how the network parameters of the prompt word model and the network parameters of the second part of the generative model are updated based on the prediction results. For example... Figure 9 As shown, step 430 above can be implemented as steps 560 to 600.

[0159] Step 560: Calculate the model loss based on the prediction results.

[0160] The specific process of "calculating model loss based on prediction results" can be found in step 340 above, and will not be repeated here.

[0161] Step 570: Calculate the gradient of the m-th layer based on the model loss; based on the gradient of the m-th layer, update the network parameters of the m-th network layer in the generative model, and the gradient of the m-th layer is the gradient of the m-th network layer.

[0162] The specific process of "calculating the gradient of the m-th layer based on the model loss; updating the network parameters of the m-th network layer in the generative model based on the gradient of the m-th layer" can be found in step 350 above, and will not be repeated here.

[0163] Step 580: Calculate the gradient of the i-th layer based on the gradient of the (i+1)-th layer; based on the gradient of the i-th layer, update the network parameters of the i-th network layer in the generative model, the gradient of the (i+1)-th layer is the gradient of the (i+1)-th network layer, the gradient of the i-th layer is the gradient of the i-th network layer, and i is an integer greater than k and less than m.

[0164] The specific process of “calculating the gradient of the i-th layer based on the gradient of the (i+1)-th layer; updating the network parameters of the i-th network layer in the generative model based on the gradient of the i-th layer, and the gradient of the (i+1)-th layer being the gradient of the (i+1)-th network layer” can be found in step 360 above, and will not be repeated here.

[0165] Step 590: Update the network parameters of the second prompt word model based on the gradient of the m-th layer or the gradient of the (k+1)-th layer.

[0166] Since the second cue feature representation is input layer by layer into the next n network layers (i.e. the second part of the network layers), the network parameters of the second cue word model can be updated based on the gradient of any network layer from the (k+1)th to the mth network layer.

[0167] Optionally, based on the gradient of the i-th layer, update the network parameters of the prompt word model, where i is an integer greater than k and less than m+1.

[0168] Step 600: Calculate the gradient of the kth layer based on the gradient of the (k+1)th layer; update the network parameters of the first prompt word model based on the gradient of the kth layer.

[0169] While the first cue feature representation is input layer by layer into the network layers 1 through k, the network parameters of layers 1 through k are frozen during the fine-tuning of the generative model to save resources or reduce computational overhead. Specifically, the gradients of layers 1 through k are no longer calculated during fine-tuning. However, when the first cue word model is used to generate the first cue feature representation input to the first part of the network layers, the gradient of layer k can be additionally calculated, and the network parameters of the first cue word model are updated based on this gradient. It should be noted that in this case, the gradient of layer k is only used to update the network parameters of the first cue word model, and not to update the network parameters of the kth network layer itself.

[0170] In some embodiments, step 590 can be performed after step 570 "calculate the gradient of the m-th layer based on the model loss" and before "update the network parameters of the m-th network layer in the generative model based on the gradient of the m-th layer", or step 590 can be performed between steps 570 and 580, or step 590 can be performed together with "update the network parameters of the m-th network layer in the generative model based on the gradient of the m-th layer" in step 570, or step 590 can be performed together with step 360, or step 590 can be performed after step 600, or step 590 can be performed together with step 600.

[0171] In summary, the method provided in this application, during the fine-tuning of the generative model, ensures that the network parameters of the first part of the network layer are not updated (i.e., frozen), meaning the first part of the network layer does not learn the features of the training samples and prompt words, while the second part of the network layer does. Therefore, during the fine-tuning of the generative model, the first and second parts of the network layer may have different interpretations of the same prompt feature representation. Thus, separate prompt word models are set for them. During backpropagation, the network parameters of the first and second prompt word models are updated based on the gradients of the first part of the network layer (the additionally calculated gradient of the k-th layer) and the second part of the network layer (any one of the gradients from the (k+1)-th to the m-th layer). This allows the first and second prompt word models to generate prompt feature representations specifically based on the capabilities of the first and second parts of the network layer, enabling them to better understand and output more accurate information, thereby improving the fine-tuning efficiency and adaptability of the generative model to the target domain.

[0172] 2.2 Generative model associates n+1 prompt words.

[0173] Optionally, the generative model is associated with n+1 prompt word models; the generative model consists of m network layers, where m is a positive integer greater than n; the first part of the network layers consists of layers 1 to k of the m network layers, and the second part of the network consists of layers k+1 to m of the m network layers. The model structure in this scenario is as follows: Figure 10 As shown, the generative model 10 includes a feature extraction layer 11, m network layers, and an output layer 12. The feature extraction layer 11 is used to extract sample feature representations 14 from the input training samples 13. The m network layers included in the generative model 10 can be divided into a first part network layer and a second part network layer. The first part network layer consists of layers 1 to k, and the second part network layer consists of layers k+1 to m. The second part network layer has n network layers, i.e., mk = n, where n, m, and k are all positive integers. The generative model 10 is also associated with n+1 prompt word models. The first prompt word model 30 is used to extract the first prompt feature representation 31 based on prompt word 21, the second prompt word model is used to extract the second prompt feature representation 41 based on prompt word 21, ..., the nth prompt word model is used to extract the nth prompt feature representation based on prompt word 21, and the (n+1)th prompt word model is used to extract the (n+1)th prompt feature representation based on prompt word 21.

[0174] First, let's explain how generative models obtain prediction results. For example... Figure 11 As shown, step 410 can be implemented as step 610, and step 420 can be implemented as steps 620 to 650.

[0175] Step 610: Input the prompt words into the n+1 prompt word model to obtain n+1 prompt feature representations.

[0176] Optionally, the n+1 prompt word models are the same pre-trained model, i.e., the first pre-trained model; or, the n+1 prompt words use the same initialization method; or, the n+1 prompt word models are different pre-trained models; or, the n+1 prompt word models use different initialization methods.

[0177] The specific process of “inputting the prompt words into the n+1 prompt word model to obtain n+1 prompt feature representations” can be found in steps 310 and 410 above, and will not be repeated here.

[0178] Step 620: For the first network layer in the m-layer network, input the first prompt feature representation and the sample feature representation of the training samples into the first network layer to obtain the first output information.

[0179] Optionally, for the first network layer in the m-layer network, the first prompt feature representation and the sample feature representation of the training samples are concatenated and then input into the first network layer to obtain the first output information. Alternatively, for the first network layer in the m-layer network, the first prompt feature representation and the sample feature representation of the training samples are input into the first network layer separately, such as inputting the first prompt feature representation first and then inputting the sample feature representation of the training samples, or inputting the sample feature representation of the training samples first and then inputting the first prompt feature representation.

[0180] The specific process of "inputting the first prompt feature representation and the sample feature representation of the training samples into the first network layer to obtain the first output information" can be found in step 310 above, and will not be repeated here.

[0181] Step 630: For the (i+1)th network layer in the m-layer network, input the first prompt feature representation and the i-th output information into the (i+1)th network layer to obtain the (i+1)th output information. The i-th information is the output of the i-th network layer, and i is a positive integer less than k.

[0182] Optionally, for the (i+1)th network layer in the m-layer network, the first prompt feature representation and the ith output information are concatenated and then input into the (i+1)th network layer to obtain the (i+1)th output information. Alternatively, for the (i+1)th network layer in the m-layer network, the first prompt feature representation and the ith output information are input into the (i+1)th network layer respectively to obtain the (i+1)th output information, such as inputting the first prompt feature representation first and then inputting the ith output information, or inputting the ith output information first and then inputting the first prompt feature representation.

[0183] The specific process of “inputting the first prompt feature representation and the i-th output information into the (i+1)-th network layer to obtain the (i+1)-th output information” can be found in steps 320 and 530 above, and will not be repeated here.

[0184] Step 640: For the (j+1)th network layer in the m-layer network, input the (j-k+2)th cue feature representation and the jth output information into the (j+1)th network layer to obtain the (j+1)th output information. The jth information is output by the jth network layer, and j is a positive integer greater than k-1 and less than m.

[0185] Optionally, for the (j+1)th network layer in the m-layer network, the (j-k+1)th cue feature representation and the jth output information are concatenated and then input into the (j+1)th network layer to obtain the (j+1)th output information; or, for the (j+1)th network layer in the m-layer network, the (j-k+2)th cue feature representation and the jth output information are input into the (j+1)th network layer respectively to obtain the (j+1)th output information, such as first inputting the (j-k+2)th cue feature representation and then inputting the jth output information, or first inputting the jth output information and then inputting the (j-k+2)th cue feature representation.

[0186] Optionally, the second part of the m-layer network, or in other words, the (k+1)th to the m-th network layers, receives the concatenated (j-k+2)th cue feature representation and the j-th output information, and obtains the (j+1)th output information based on the concatenated (j-k+2)th cue feature representation and the j-th output information. For the (k+1)th to (m-1)th network layers, the (j+1)th output information is also output to the (j+2)th network layer. For the m-th network layer, the m-th output information is output to the output layer.

[0187] The second part of the network layer consists of n network layers. Each network layer in the second part of the network layer corresponds to a cue feature representation, or in other words, each network layer in the second part of the network layer corresponds to a cue word model.

[0188] For example, given n = mk, when j = k, for the (k+1)th network layer, the (k-k+2) = 2nd cue feature representation and the kth output information are input into the (k+1)th network layer to obtain the (k+1)th output information; when j = k+1, for the (k+1+1) = k+2th network layer, the (k+1-k+2) = 3rd cue feature representation and the (k+1)th output information are input into the (k+2)th network layer to obtain the (k+2)th output information; ...; when j = m-2, for the (m-2+1) = m-1th network layer, the (m-2-k+2) = nth cue feature representation and the (m-2)th output information are input into the (m-1)th network layer to obtain the (m-1)th output information; when j = m-1, for the mth network layer, the (m-1-k+2) = n+1th cue feature representation and the (m-1)th output information are input into the mth network layer to obtain the mth output information.

[0189] The specific process of "inputting the (j-k+1)th cue feature representation and the jth output information into the (j+1)th network layer to obtain the (j+1)th output information" can be found in steps 320, 540, and 640 above, and will not be repeated here.

[0190] Step 650: Obtain the prediction result of the training sample based on the m-th output information.

[0191] Optionally, the generative model obtains the prediction result of the training sample based on the m-th output information; or, the output layer in the generative model obtains the prediction result of the training sample based on the m-th output information.

[0192] In summary, the method provided in this application inputs the first cue feature representation into each layer of the first part of the generative model's network layer, and inputs the second to (n+1)th cue feature representations into each layer of the second part of the generative model's network layer. This is because during the fine-tuning of the generative model, the network parameters of the first part of the network layer are not updated (i.e., frozen), meaning the first part of the network layer does not learn the features of the training samples and cue words, while the second part of the network layer does. Furthermore, due to the characteristics of the backpropagation algorithm, the degree to which each layer in the second part of the network layer learns the features of the training samples and cue words is different. Therefore, during the fine-tuning of the generative model, different layers in the first and second parts of the network layer may have different understandings of the same cue feature representation. Therefore, separate cue word models are set for them, thereby generating cue feature representations specifically based on the capabilities of the different layers in the first and second parts of the network layer, enabling them to better understand and output more accurate output information, thus improving the fine-tuning efficiency and adaptability of the generative model to the target domain.

[0193] The following section details how the network parameters of the prompt word model and the network parameters of the second part of the generative model are updated based on the prediction results. For example... Figure 12 As shown, step 430 above can be implemented as steps 660 to 690.

[0194] Step 660: Calculate the model loss based on the prediction results.

[0195] The specific process of "calculating model loss based on prediction results" can be found in step 340 above, and will not be repeated here.

[0196] Step 670: Calculate the gradient of the m-th layer based on the model loss; based on the gradient of the m-th layer, update the network parameters of the m-th network layer and the network parameters of the n+1-th prompt word model in the generative model, and the gradient of the m-th layer is the gradient of the m-th network layer.

[0197] Here, the (n+1)th prompt word model is the prompt word model corresponding to the m-th network layer, or in other words, the (n+1)th prompt feature representation is the prompt feature representation input to the m-th network layer.

[0198] The specific process of "calculating the gradient of the m-th layer based on the model loss; updating the network parameters of the m-th network layer in the generative model based on the gradient of the m-th layer" can be found in step 350 above. The specific process of "updating the network parameters of the (n+1)-th prompt word model based on the gradient of the m-th layer" can also be found in steps 370 and 590 above, and will not be repeated here.

[0199] Step 680: Calculate the gradient of the i-th layer based on the gradient of the (i+1)-th layer; based on the gradient of the i-th layer, update the network parameters of the i-th network layer and the network parameters of the (i-k+1)-th prompt word model in the generative model. The gradient of the (i+1)-th layer is the gradient of the (i+1)-th network layer, and the gradient of the i-th layer is the gradient of the i-th network layer, where i is an integer greater than k and less than m.

[0200] Here, the i-k+1th prompt word model is the prompt word model corresponding to the i-th network layer, or in other words, the i-k+1th prompt feature representation is the prompt feature representation input to the i-th network layer.

[0201] For example, given n = mk, when i = m-1, based on the gradient of the (m-1)th layer, update the (m-1)th network layer and the (m-1-k+1) = nth prompt word model; when i = m-2, based on the gradient of the (m-2)th layer, update the (m-2)th network layer and the (m-2-k+1) = n-1th prompt word; ...; when i = k+2, based on the gradient of the (k+2)th layer, update the (k+2)th network layer and the (k+2-k+2) = 3rd prompt word model; when i = k+1, based on the gradient of the (k+1)th layer, update the (k+1)th network layer and the (k+1-k+2) = 2nd prompt word model.

[0202] The specific process of “calculating the gradient of the i-th layer based on the gradient of the (i+1)-th layer; updating the network parameters of the i-th network layer and the network parameters of the i-k+1-th prompt word model based on the gradient of the i-th layer” can be found in steps 350, 370, 590 and 670 above, and will not be repeated here.

[0203] Step 690: Calculate the gradient of the kth layer based on the gradient of the (k+1)th layer; update the network parameters of the first prompt word model based on the gradient of the kth layer.

[0204] In this model, although the first cue feature representation is fed layer by layer into the network layers 1 through k, the network parameters of these layers are frozen during fine-tuning of the generative model to save resources or reduce computational overhead. Specifically, the gradients of these layers are no longer calculated during fine-tuning. However, when the first cue word model is used to generate the first cue feature representation input to the first part of the network layers, the gradient of the k-th layer can be additionally calculated, and the first cue word model is updated based on this gradient. It should be noted that in this case, the gradient of the k-th layer is only used to update the first cue word model, and not to update the network parameters of the k-th network layer.

[0205] In summary, the method provided in this application, during the fine-tuning of the generative model, ensures that the network parameters of the first part of the network layer are not updated (i.e., frozen), meaning the first part of the network layer does not learn the features of the training samples and prompt words, while the second part of the network layer does. Therefore, during the fine-tuning of the generative model, the first and second parts of the network layer may have different interpretations of the same prompt feature representation. Thus, separate prompt word models are set for them. During backpropagation, the first to the (n+1)th prompt word models are updated based on the gradient of the first part of the network layer (the additionally calculated gradient of the k-th layer) and the gradients of each layer in the second part of the network layer (i.e., the gradients from the (k+1)-th to the (m)-th layers). This allows these (n+1) prompt word models to generate prompt feature representations specifically based on the different capabilities of the first and second parts of the network layer, enabling them to better understand and output more accurate information, thereby improving the fine-tuning efficiency and adaptability of the generative model to the target domain.

[0206] Figure 13 A flowchart illustrating a generative model-based prediction method provided in an exemplary embodiment of this application is shown. The method is performed by the fine-tuned generative model shown above (i.e., the updated prompt word model and the generative model), and includes:

[0207] Step 240: In the reasoning stage, input the prompt words into the prompt word model and input the input information to be reasoned into the generative model to obtain the prediction results of the input information.

[0208] Optionally, the prompt word is input into the prompt word model to obtain the prompt feature representation; during the process of the generative model predicting the input information, the prompt feature representation is used as the guide input to update the generative model to obtain the prediction result.

[0209] The input information can be at least one of text, image, audio, or video. The prediction result can be at least one of text, image, audio, or video. The format of the input information and the prediction result is determined according to the task of the generative model.

[0210] Optionally, the network layers of the generative model are used to generate prediction results or output information based on the input feature representations of the cue features and the input information. The extraction of input feature representations can refer to the extraction of sample feature representations described above, and will not be repeated here.

[0211] In some embodiments, the generative model is associated with a cue word model, and the process of the generative model predicting based on input information can be referred to steps 310 to 330 above.

[0212] In some embodiments, the generative model is associated with at least two cue word models. Most typically, the generative model is associated with either two cue word models or n+1 cue word models, where n is the number of layers in the second part of the network. When the generative model is associated with two cue word models, the process of prediction based on the input information can refer to steps 510 to 550 above. When the generative model is associated with n+1 cue word models, the process of prediction based on the input information can refer to steps 610 to 650 above.

[0213] In summary, the method provided in this application illustrates the reasoning process of a generative model. The generative model still uses prompt words for guidance, thereby generating prediction results that meet the task content of the target domain and improving the accuracy of the generative model.

[0214] For example, the fine-tuning method for updating all network parameters in related technologies typically requires a graphics card with at least 80GB of video memory. However, such graphics cards are expensive to purchase, and in many scenarios, they may be difficult to obtain, requiring the use of a graphics card with approximately 32GB of video memory. The fine-tuning method for updating all network parameters is generally difficult to run on a graphics card with 32GB of video memory. The method shown in this application embodiment can run on a graphics card with 32GB of video memory and achieves the fine-tuning effect of the method for updating all network parameters. The overall process will be described below.

[0215] Step 1: Select a large open-source model, such as chatglm-6b, llama-13b, bloom, etc. We will take the chatglm-6b model as an example, which has 28 network layers.

[0216] Step 2: Select the last n layers of the network as the second part of the network layer. During fine-tuning, only update the network parameters of the second part of the network layer. The network parameters of the remaining network layers (such as the first part of the network layer) are frozen.

[0217] Here, 'n' is a hyperparameter, and its selection depends on the graphics card's memory. Generally, the larger the value of 'n', the better the effect. Taking a graphics card with 32GB of memory as an example, we set the length of the input (i.e., the length of the input sequence) to 256. The maximum value of 'n' can be 3, but we choose 2 here (considering that the prompt word model also needs fine-tuning and will occupy memory, this step cannot fill up all the memory; some memory needs to be reserved). That is, the last two layers of the chatglm-6b model are used as the second part of the network layers.

[0218] Step 3: In each of the 28 network layers, a cue feature representation is added before the sample feature representation.

[0219] The cue feature representation is extracted from the cue by the cue word model. The cue word model can be a BiLSTM+MLP model, a transformer model, or other models.

[0220] After the fine-tuning process of the generative model is complete, it can be applied to the target domain. Taking the target domain as the medical field and the task as medical question answering as an example, in the process of using the generative model, prompts are input into the prompt word model, and questions are input into the generative model, which then outputs the answers.

[0221] For example:

[0222] <Question>: I feel dizzy, nauseous, and have a slight fever. What medicine should I take to make myself feel better?

[0223] <Prompt words>: Drug 1, Drug 2, Drug 3.

[0224] <Answer generated by the generative model>: Drug 2. Of course, the best advice is to go to the hospital for a consultation and confirm the specific situation before taking any medication.

[0225] Please refer to Figure 14 This diagram illustrates a structural block diagram of a fine-tuning apparatus for a generative model provided in an exemplary embodiment of this application. The apparatus has the functionality to implement the example of the fine-tuning method for the generative model described above; this functionality can be implemented in hardware or by hardware executing corresponding software. The apparatus can be the computer device described above, or it can be located within a computer device. Figure 14 As shown, the device may include: an acquisition module 710, a prediction module 720, and an update module 730.

[0226] The generative model is associated with a cue word model, and the network parameters of the cue word model are fewer than the network parameters of the first part of the network layer in the generative model.

[0227] The acquisition module 710 is used to input the prompt words into the prompt word model to obtain the prompt feature representation;

[0228] The prediction module 720 is used to input the prompt feature representation as a guiding prompt into the generative model during the process of the generative model predicting training samples, so as to obtain the prediction result of the training samples.

[0229] The update module 730 is used to update the network parameters of the prompt word model and the network parameters of the second part of the network layer in the generative model based on the prediction result. The second part of the network layer includes n network layers in the generative model that are located after the first part of the network layer, where n is a positive integer.

[0230] In some embodiments, the prompt words are constructed based on task content in a target domain, which is the domain in which the fine-tuned generative model is applied.

[0231] In some embodiments, the network parameters of the prompt word model are at least an order of magnitude less than the network parameters of the first part of the network layer.

[0232] In some embodiments, the generative model includes m network layers, where m is a positive integer greater than n; the prediction module 720 is further configured to input the cue feature representation and the sample feature representation of the training sample into the first network layer of the m network layers to obtain first output information; input the cue feature representation and the i-th output information into the (i+1)-th network layer of the m network layers to obtain (i+1)-th output information, where the i-th output information is output by the i-th network layer and i is a positive integer less than m; and obtain the prediction result of the training sample based on the m-th output information.

[0233] In some embodiments, the generative model includes m network layers, where m is a positive integer greater than n; the first part of the network layers consists of layers 1 to k in the m network layers, and the second part of the network layers consists of layers k+1 to m in the m network layers; the update module 730 is further configured to calculate the model loss based on the prediction result; calculate the gradient of the m-th layer based on the model loss; update the network parameters of the m-th network layer in the generative model based on the gradient of the m-th layer, where the gradient of the m-th layer is the gradient of the m-th network layer; calculate the gradient of the i-th layer based on the gradient of the (i+1)-th layer; update the network parameters of the i-th network layer in the generative model based on the gradient of the i-th layer, where the gradient of the (i+1)-th layer is the gradient of the (i+1)-th layer, and the i-th gradient is the gradient of the i-th network layer, where i is an integer greater than k and less than m; and update the network parameters of the prompt word model based on the gradient of the m-th layer or the gradient of the (k+1)-th layer.

[0234] In some embodiments, the generative model is associated with at least two prompt word models; the acquisition module 710 is further configured to input the prompt word into each of the at least two prompt word models to obtain at least two prompt feature representations; the prediction module 720 is further configured to input the at least two prompt feature representations into a first part of the network layer of the generative model, and / or different network layers in the second part of the network layer; and input the sample feature representation of the training sample into the first network layer of the generative model to obtain the prediction result of the training sample; the update module 730 is further configured to update the network parameters of the at least two prompt word models and the network parameters of the second part of the network layer in the generative model based on the prediction result.

[0235] In some embodiments, the generative model is associated with a first prompt word model and a second prompt word model; the generative model includes m network layers, where m is a positive integer greater than n; the first part of the network layers consists of layers 1 to k in the m network layers, and the second part of the network consists of layers k+1 to m in the m network layers, where k is a positive integer; the acquisition module 710 is further configured to input the prompt word into the first prompt word model to obtain a first prompt feature representation; and input the prompt word into the second prompt word model to obtain a second prompt feature representation; the prediction module 720 is further configured to, for the first network layer in the m network layers, combine the first prompt feature representation with the training... The sample feature representation of the training sample is input into the first network layer to obtain the first output information; for the (i+1)th network layer in the m-layer network, the first prompt feature representation and the i-th output information are input into the (i+1)th network layer to obtain the (i+1)th output information, where the i-th output information is output by the i-th network layer and i is a positive integer less than k; for the (j+1)th network layer in the m-layer network, the second prompt feature representation and the j-th output information are input into the (j+1)th network layer to obtain the (j+1)th output information, where the j-th output information is output by the j-th network layer and j is a positive integer greater than k-1 and less than m; the prediction result of the training sample is obtained based on the m-th output information.

[0236] In some embodiments, the update module 730 is further configured to: calculate the model loss based on the prediction result; calculate the gradient of the m-th layer based on the model loss; update the network parameters of the m-th network layer in the generative model based on the gradient of the m-th layer, wherein the gradient of the m-th layer is the gradient of the m-th network layer; calculate the gradient of the i-th layer based on the gradient of the (i+1)-th layer; update the network parameters of the i-th network layer in the generative model based on the gradient of the i-th layer, wherein the gradient of the (i+1)-th layer is the gradient of the (i+1)-th network layer, and the i-th gradient is the gradient of the i-th network layer, where i is an integer greater than k and less than m; update the network parameters of the second prompt word model based on the gradient of the m-th layer or the (k+1)-th layer; calculate the gradient of the k-th layer based on the gradient of the (k+1)-th layer; and update the network parameters of the first prompt word model based on the gradient of the k-th layer.

[0237] In some embodiments, the generative model is associated with n+1 prompt word models; the generative model includes m network layers, where m is a positive integer greater than n; the first part of the network layers consists of layers 1 to k in the m network layers, and the second part of the network consists of layers k+1 to m in the m network layers; the acquisition module 710 is further configured to input the first prompt feature representation and the sample feature representation of the training samples into the first network layer in the m network layers to obtain the first output information; for the i-th prompt word model in the m network layers... +1 network layer, input the first prompt feature representation and the i-th output information into the (i+1)-th network layer to obtain the (i+1)-th output information, where the i-th output information is output by the i-th network layer and i is a positive integer less than k; for the (j+1)-th network layer in the m-th network layer, input the (j-k+2)-th prompt feature representation and the j-th output information into the (j+1)-th network layer to obtain the (j+1)-th output information, where the j-th information is output by the j-th network layer and j is a positive integer greater than k-1 and less than m; based on the m-th output information, obtain the prediction result of the training sample.

[0238] In some embodiments, the update module 730 is further configured to: calculate the model loss based on the prediction result; calculate the gradient of the m-th layer based on the model loss; update the network parameters of the m-th network layer and the network parameters of the (n+1)-th prompt word model in the generative model based on the gradient of the m-th layer, wherein the gradient of the m-th layer is the gradient of the m-th network layer; calculate the gradient of the i-th layer based on the gradient of the (i+1)-th layer; update the network parameters of the i-th network layer and the network parameters of the (i-k+1)-th prompt word model in the generative model based on the gradient of the i-th layer, wherein the gradient of the (i+1)-th layer is the gradient of the (i+1)-th network layer, and the i-th gradient is the gradient of the i-th network layer, where i is an integer greater than k and less than m; calculate the gradient of the k-th layer based on the gradient of the (k+1)-th layer; and update the network parameters of the first prompt word model based on the gradient of the k-th layer.

[0239] In some embodiments, the apparatus further includes a generation module. The generation module is configured to, during the inference phase, input the cue words into the cue word model and input the input information to be inferred into the generative model to obtain a prediction result for the input information.

[0240] In summary, the apparatus provided in this application only needs to update the network parameters of the second part of the network layer and the network parameters of the prompt word model during the fine-tuning process of the generative model. The network parameters of the first part of the network layer are frozen and not updated, while the network parameters of the prompt word model are fewer than, or even far fewer than, the network parameters of the first part of the network layer. That is, compared with updating all network parameters of the generative model in related technologies, the method provided in this application requires less overhead and has low performance requirements for the computer equipment used to fine-tune the generative model. In addition, besides the network layers of the generative model itself, a prompt word model is also associated with the generative model. The prompt word model is used to extract prompt feature representations from the prompt words. The prompt words can guide the generative model to generate content for a specified task. Associating the prompt word model with the generative model and updating the network parameters of the prompt word model during the fine-tuning process of the generative model enables the prompt word model to learn how to extract prompt feature representations to guide the generative model to generate content for a specified task, without requiring the generative model to generate based on unknown tasks. This saves the generative model's exploration time for unknown tasks and thus accelerates the fine-tuning process of the generative model.

[0241] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0242] Figure 15 A structural block diagram of a computer device provided in an exemplary embodiment of this application is shown.

[0243] The computer device 800 includes a central processing unit (CPU) 801, a system memory 804 including random access memory (RAM) 802 and read-only memory (ROM) 803, and a system bus 805 connecting the system memory 804 and the CPU 801. The computer device 800 also includes a basic input / output system (I / O system) 806 to facilitate information transfer between various components within the computer device, and a mass storage device 807 for storing the operating system 813, application programs 814, and other program modules 815.

[0244] The basic input / output system 806 includes a display 808 for displaying information and an input device 809 for user input, such as a mouse or keyboard. Both the display 808 and the input device 809 are connected to the central processing unit 801 via an input / output controller 810 connected to the system bus 805. The basic input / output system 806 may also include the input / output controller 810 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 810 also provides output to a display screen, printer, or other types of output devices.

[0245] The mass storage device 807 is connected to the central processing unit 801 via a mass storage controller (not shown) connected to the system bus 805. The mass storage device 807 and its associated computer-readable storage media provide non-volatile storage for the computer device 800. That is, the mass storage device 807 may include computer-readable storage media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0246] Without loss of generality, the computer-readable storage medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable storage instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the above-mentioned types. The system memory 804 and mass storage device 807 described above can be collectively referred to as memory.

[0247] The memory stores one or more programs, which are configured to be executed by one or more central processing units 801. The one or more programs contain instructions for implementing the above method embodiments, and the central processing unit 801 executes the one or more programs to implement the methods provided by the above method embodiments.

[0248] According to various embodiments of this application, the computer device 800 can also be connected to a remote computer device on a network, such as the Internet. That is, the computer device 800 can be connected to a network 812 via a network interface unit 811 connected to the system bus 805, or the network interface unit 811 can be used to connect to other types of networks or remote computer device systems (not shown).

[0249] The memory further includes one or more programs stored in the memory, and the one or more programs include steps performed by a computer device in the methods provided in the embodiments of this application.

[0250] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the above-described method for fine-tuning the generative model.

[0251] In an exemplary embodiment, a computer program product is also provided, which, when executed by a processor, is used to implement the fine-tuning method of the generative model described above.

[0252] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0253] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for fine-tuning a generative model, characterized in that, The generative model is associated with a cue word model, the cue word model having fewer network parameters than the first part of the network layer in the generative model, and the method includes: Input the prompt words into the prompt word model to obtain the prompt feature representation; During the process of the generative model predicting training samples, the prompt feature representation is used as a guiding prompt input into the generative model to obtain the prediction result of the training samples; Based on the prediction results, the model loss of the generative model is obtained. According to the backpropagation process of the model loss, the network parameters of the prompt word model and the network parameters of the second part of the network layer in the generative model are updated. The backpropagation process includes the flow from the second part of the network layer to the prompt word model. The second part of the network layer includes n network layers in the generative model located after the first part of the network layer, where n is a positive integer.

2. The method according to claim 1, characterized in that, The prompt words are constructed based on the task content of the target domain, which is the domain in which the fine-tuned generative model is applied.

3. The method according to claim 1, characterized in that, The network parameters of the prompt word model are at least one order of magnitude less than the network parameters of the first part of the network layer.

4. The method according to any one of claims 1 to 3, characterized in that, The generative model comprises m network layers, where m is a positive integer greater than n; In the process of the generative model predicting training samples, the cue feature representation is used as a guiding cue input into the generative model to obtain the prediction result of the training samples, including: For the first network layer in the m-layer network, the prompt feature representation and the sample feature representation of the training sample are input into the first network layer to obtain the first output information; For the (i+1)th network layer in the m-layer network, the prompt feature representation and the i-th output information are input into the (i+1)th network layer to obtain the (i+1)-th output information, where the i-th output information is output by the i-th network layer and i is a positive integer less than m; The prediction result of the training sample is obtained based on the m-th output information.

5. The method according to claim 4, characterized in that, The generative model includes m network layers, where m is a positive integer greater than n; the first part of the network layers consists of layers 1 to k in the m network layers, and the second part of the network layers consists of layers k+1 to m in the m network layers. The process of obtaining the model loss of the generative model based on the prediction results, and updating the network parameters of the prompt word model and the network parameters of the second part of the network layer in the generative model according to the backpropagation process of the model loss, includes: The model loss of the generative model is calculated based on the prediction results; The gradient of the m-th layer is calculated based on the model loss; the network parameters of the m-th network layer in the generative model are updated based on the gradient of the m-th layer, where the gradient of the m-th layer is the gradient of the m-th network layer. Calculate the gradient of the i-th layer based on the gradient of the (i+1)-th layer; update the network parameters of the i-th network layer in the generative model based on the gradient of the i-th layer, where the gradient of the (i+1)-th layer is the gradient of the (i+1)-th network layer, and the i-th gradient is the gradient of the i-th network layer, and i is an integer greater than k and less than m; Based on the gradient of the m-th layer or the gradient of the (k+1)-th layer, the network parameters of the prompt word model are updated through the backpropagation algorithm.

6. The method according to any one of claims 1 to 3, characterized in that, The generative model is associated with at least two prompt word models; The step of inputting the prompt words into the prompt word model to obtain the prompt feature representation includes: The prompt word is input into each of the at least two prompt word models to obtain at least two prompt feature representations; In the process of the generative model predicting training samples, the cue feature representation is used as a guiding cue input into the generative model to obtain the prediction result of the training samples, including: The at least two cue feature representations are respectively input into the first part of the network layer of the generative model, and / or, different network layers in the second part of the network layer; and the sample feature representation of the training sample is input into the first network layer of the generative model to obtain the prediction result of the training sample; The process of obtaining the model loss of the generative model based on the prediction results, and updating the network parameters of the prompt word model and the network parameters of the second part of the network layer in the generative model according to the backpropagation process of the model loss, includes: Based on the prediction results, the model loss of the generative model is obtained. According to the backpropagation process of the model loss, the network parameters of the at least two prompt word models and the network parameters of the second part of the network layer in the generative model are updated.

7. The method according to claim 6, characterized in that, The generative model is associated with a first prompt word model and a second prompt word model; the generative model includes m network layers, where m is a positive integer greater than n; the first part of the network layers consists of layers 1 to k in the m network layers, and the second part of the network consists of layers k+1 to m in the m network layers, where k is a positive integer; The step of inputting the prompt word into each of the at least two prompt word models to obtain at least two prompt feature representations includes: The prompt word is input into the first prompt word model to obtain a first prompt feature representation; and the prompt word is input into the second prompt word model to obtain a second prompt feature representation; The step of inputting each of the at least two cue feature representations into a first part of the network layer of the generative model, and / or different network layers in the second part of the network layer; and inputting the sample feature representation of the training sample into the first network layer of the generative model to obtain the prediction result of the training sample includes: For the first network layer in the m-layer network, the first prompt feature representation and the sample feature representation of the training sample are input into the first network layer to obtain the first output information; For the (i+1)th network layer in the m-layer network, the first prompt feature representation and the i-th output information are input into the (i+1)th network layer to obtain the (i+1)th output information, where the i-th output information is output by the i-th network layer and i is a positive integer less than k; For the (j+1)th network layer in the m-layer network, the second prompt feature representation and the j-th output information are input into the (j+1)th network layer to obtain the (j+1)-th output information. The j-th output information is output by the j-th network layer, where j is a positive integer greater than k-1 and less than m. The prediction result of the training sample is obtained based on the m-th output information.

8. The method according to claim 7, characterized in that, The process of obtaining the model loss of the generative model based on the prediction results, and updating the network parameters of the at least two prompt word models and the network parameters of the second part of the network layer in the generative model according to the backpropagation process of the model loss, includes: The model loss of the generative model is calculated based on the prediction results; The gradient of the m-th layer is calculated based on the model loss; the network parameters of the m-th network layer in the generative model are updated based on the gradient of the m-th layer, where the gradient of the m-th layer is the gradient of the m-th network layer. Calculate the gradient of the i-th layer based on the gradient of the (i+1)-th layer; update the network parameters of the i-th network layer in the generative model based on the gradient of the i-th layer, where the gradient of the (i+1)-th layer is the gradient of the (i+1)-th network layer, and the i-th gradient is the gradient of the i-th network layer, and i is an integer greater than k and less than m; Based on the gradient of the m-th layer or the gradient of the (k+1)-th layer, the network parameters of the second prompt word model are updated through the backpropagation algorithm; The gradient of the kth layer is calculated based on the gradient of the (k+1)th layer; the network parameters of the first prompt word model are updated based on the gradient of the kth layer using the backpropagation algorithm.

9. The method according to claim 6, characterized in that, The generative model is associated with n+1 prompt word models, where n is a positive integer greater than one; the generative model includes m network layers, where m is a positive integer greater than n; the first part of the network layers consists of layers 1 to k in the m network layers, and the second part of the network consists of layers k+1 to m in the m network layers; The step of inputting the prompt word into each of the at least two prompt word models to obtain at least two prompt feature representations includes: The prompt words are input into the n+1 prompt word model to obtain n+1 prompt feature representations; The step of inputting each of the at least two cue feature representations into a first part of the network layer of the generative model, and / or different network layers in the second part of the network layer; and inputting the sample feature representation of the training sample into the first network layer of the generative model to obtain the prediction result of the training sample includes: For the first network layer in the m-layer network, the first prompt feature representation and the sample feature representation of the training sample are input into the first network layer to obtain the first output information; For the (i+1)th network layer in the m-layer network, the first prompt feature representation and the ith output information are input into the (i+1)th network layer to obtain the (i+1)th output information, where the ith output information is output by the ith network layer and i is a positive integer less than k; For the (j+1)th network layer in the m-layer network, the (j-k+2)th prompt feature representation and the jth output information are input into the (j+1)th network layer to obtain the (j+1)th output information. The jth output information is output by the jth network layer, where j is a positive integer greater than k-1 and less than m. The prediction result of the training sample is obtained based on the m-th output information.

10. The method according to claim 9, characterized in that, The process of obtaining the model loss of the generative model based on the prediction results, and updating the network parameters of the at least two prompt word models and the network parameters of the second part of the network layer in the generative model according to the backpropagation process of the model loss, includes: The model loss of the generative model is calculated based on the prediction results; Calculate the gradient of the m-th layer based on the model loss; based on the gradient of the m-th layer, update the network parameters of the m-th network layer in the generative model and update the network parameters of the (n+1)-th prompt word model through the backpropagation algorithm, wherein the gradient of the m-th layer is the gradient of the m-th network layer; Calculate the gradient of the i-th layer based on the gradient of the (i+1)-th layer; based on the gradient of the i-th layer, update the network parameters of the i-th network layer in the generative model and update the network parameters of the (i-k+1)-th prompt word model through the backpropagation algorithm, wherein the gradient of the (i+1)-th layer is the gradient of the (i+1)-th network layer, and the i-th gradient is the gradient of the i-th network layer, where i is an integer greater than k and less than m; Calculate the gradient of the kth layer based on the gradient of the (k+1)th layer; based on the gradient of the kth layer, update the network parameters of the first prompt word model through the backpropagation algorithm.

11. The method according to any one of claims 1 to 3, characterized in that, The method further includes: During the inference phase, the prompt words are input into the prompt word model, and the input information to be inferred is input into the generative model to obtain the prediction result of the input information.

12. A fine-tuning device for a generative model, characterized in that, The generative model is associated with a cue word model, the cue word model having fewer network parameters than the first part of the network layer in the generative model, and the device includes: The acquisition module is used to input the prompt words into the prompt word model to obtain the prompt feature representation; The prediction module is used to input the prompt feature representation as a guiding prompt into the generative model during the process of the generative model predicting training samples, so as to obtain the prediction result of the training samples. An update module is used to obtain the model loss of the generative model based on the prediction result, and update the network parameters of the prompt word model and the network parameters of the second part of the network layer in the generative model according to the backpropagation process of the model loss. The backpropagation process includes flowing from the second part of the network layer to the prompt word model. The second part of the network layer includes n network layers in the generative model located after the first part of the network layer, where n is a positive integer.

13. A computer device, characterized in that, The computer device includes: a processor and a memory, wherein the memory stores at least one program; the processor is configured to execute the at least one program in the memory to implement the fine-tuning method of the generative model as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores executable instructions that are loaded and executed by a processor to implement the fine-tuning method of the generative model as described in any one of claims 1 to 11.

15. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, and a processor reads and executes the computer instructions from the computer-readable storage medium to implement the fine-tuning method of the generative model as described in any one of claims 1 to 11.

Images