Language model self-adaptive fine tuning method and device, equipment and medium

Abstract

The invention relates to the field of artificial intelligence, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a self-adaptive fine tuning method, device, equipment and medium for a language model, comprising: collecting a fine tuning training sample set, each fine tuning training sample comprising a training text and label data corresponding to at least one fine tuning task; inputting the training text and the corresponding label data into a pre-trained language model to obtain hidden representation of the training text; performing adaptive generation processing on the hidden representation by using a pre-constructed soft prompt generator to obtain a corresponding soft prompt vector; inputting the hidden representation and the soft prompt vector into the language model again to obtain a corresponding prediction output result; and performing parameter fine tuning according to the difference between the predicted output result and the label data until a preset ending condition is met. The parameter quantity capable of being finely adjusted is improved while self-adaptive fine adjustment is achieved through the soft prompt generator, and more effective and reliable model fine adjustment is achieved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to an adaptive fine-tuning method, device, equipment and medium for a language model. Background Art

[0002] With the development of pre-trained language models, the number of parameters of more and more language models has reached hundreds of millions of levels, which makes it impossible for enterprises to deploy one model for each task. Therefore, the parameter-efficient fine-tuning method is very important.

[0003] For each different task (dataset), the parameter-efficient fine-tuning method only needs to fine-tune tens of millions of parameters to adapt a large pre-trained model to the task, and obtains a performance that is basically the same as or very close to that of directly fine-tuning all the parameters of the entire model for a task. This has very important and broad application prospects in various fields, including the medical and health field and the financial field, etc. For example, in the medical and health field, a large model backbone needs to support multiple scenarios including physical examination report extraction, case understanding, knowledge Q&A, patient management, etc.; and for another example, in the financial field, a large model backbone may involve tasks including intelligent customer service, risk control, product personalized recommendation, etc. Therefore, the parameter-efficient fine-tuning method that saves video memory for model fine-tuning and inference is very important.

[0004] Currently, the soft prompt tuning method (soft prompt tuning, or p-tuning method) is a widely used parameter-efficient fine-tuning method. However, due to the increase in model size, the existing method is limited by the soft prompt length and cannot set too large a soft prompt length, resulting in too few parameters that can be optimized, poor controllability, difficult to achieve effective and reliable model fine-tuning, and reducing the fine-tuning effect of the pre-trained language model. Summary of the Invention

[0005] In view of the above deficiencies of the prior art, the purpose of the present invention is to provide an adaptive fine-tuning method, device, equipment and medium for a language model that can be applied to the medical field, fintech or other related fields, and its main purpose is to increase the number of parameters that can be fine-tuned during fine-tuning and achieve effective and reliable model fine-tuning.

[0006] The technical solution of the present invention is as follows:

[0007] The first aspect of the present invention provides an adaptive fine-tuning method for a language model, including:

[0008] Collect a fine-tuning training sample set, where each fine-tuning training sample in the fine-tuning training sample set includes training text and label data corresponding to at least one fine-tuning task;

[0009] Input the training text and the corresponding label data into a pre-trained language model to obtain the hidden representation of the training text;

[0010] Use a pre-constructed soft prompt generator to perform adaptive generation processing on the hidden representation to obtain the corresponding soft prompt vector;

[0011] Re-input the hidden representation and the soft prompt vector into the language model to obtain the corresponding predicted output result;

[0012] Perform parameter fine-tuning according to the difference between the predicted output result and the label data until a preset end condition is met.

[0013] The second aspect of the present invention provides an adaptive fine-tuning device for a language model, including:

[0014] An acquisition module for acquiring a fine-tuning training sample set, where each fine-tuning training sample in the fine-tuning training sample set includes a training text and label data corresponding to at least one fine-tuning task;

[0015] A representation module for inputting the training text and the corresponding label data into a pre-trained language model to obtain the hidden representation of the training text;

[0016] An adaptive generation module for using a pre-constructed soft prompt generator to perform adaptive generation processing on the hidden representation to obtain the corresponding soft prompt vector;

[0017] A prediction output module for re-inputting the hidden representation and the soft prompt vector into the language model to obtain the corresponding predicted output result;

[0018] A parameter fine-tuning module for performing parameter fine-tuning according to the difference between the predicted output result and the label data until a preset end condition is met.

[0019] The third aspect of the present invention provides a computer device, including at least one processor; and,

[0020] A memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned adaptive fine-tuning method for a language model.

[0022] A fourth aspect of the present invention provides a non - volatile computer - readable storage medium storing computer - executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the above - mentioned adaptive fine - tuning method of the language model.

[0023] Beneficial effects: The present invention discloses an adaptive fine - tuning method, device, equipment and medium for a language model. Compared with the prior art, in the embodiments of the present invention, a fine - tuning training sample set is collected, and each fine - tuning training sample in the fine - tuning training sample set includes training text and label data corresponding to at least one fine - tuning task; the training text and the corresponding label data are input into a pre - trained language model to obtain hidden representations of the training text; a pre - constructed soft prompt generator is used to perform adaptive generation processing on the hidden representations to obtain corresponding soft prompt vectors; the hidden representations and the soft prompt vectors are re - input into the language model to obtain corresponding predicted output results; parameter fine - tuning is performed according to the difference between the predicted output results and the label data until a preset end condition is met. By introducing a soft prompt generator to dynamically generate appropriate soft prompt vectors according to the input text for model fine - tuning, the amount of tunable parameters in prompt fine - tuning is increased without increasing the length of the soft prompt, the complexity of model inference is not increased, and it can help the model better obtain semantic knowledge during fine - tuning, realizing effective and reliable model fine - tuning. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] To more clearly illustrate the solutions in the present invention, the following will briefly introduce the drawings required for the description of the embodiments of the present invention. Obviously, the following - described drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0025] Figure 1 It is a schematic diagram of an application environment for the adaptive fine - tuning method of the language model provided by the embodiment of the present invention;

[0026] Figure 2 It is a flowchart of the adaptive fine - tuning method of the language model provided by the embodiment of the present invention;

[0027] Figure 3 It is a flowchart of step S203 of the adaptive fine - tuning method of the language model provided by the embodiment of the present invention;

[0028] Figure 4 It is a schematic diagram of the functional modules of the adaptive fine - tuning device for the language model provided by the embodiment of the present invention;

[0029] Figure 5Schematic diagram of the hardware structure of the computer device provided by the embodiment of the present invention. Detailed implementation manners

[0030] To make the objectives, technical solutions and effects of the present invention clearer and more definite, the present invention will be further described in detail below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. The embodiments of the present invention will be introduced below with reference to the accompanying drawings.

[0031] The adaptive fine-tuning method for the language model provided by the embodiment of the present invention can be applied in an application environment such as Figure 1 , which includes a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0032] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0033] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.

[0034] The server 105 can be a server that provides various services, such as a background server that provides support for the content browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (only for example). The background server can analyze and process data such as received user requests, and feedback the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device. The server 105 can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability in traditional physical hosts and VPS services (″Virtual Private Server″, or simply ″VPS″). The server 105 can also be a server of a distributed system, or a server combined with a blockchain.

[0035] It should be noted that the adaptive fine-tuning method for the language model provided in the embodiments of the present application can generally be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103. Correspondingly, the adaptive fine-tuning device for the language model provided in the embodiments of the present invention can also be set in the first terminal device 101, the second terminal device 102, or the third terminal device 103. Alternatively, the adaptive fine-tuning method for the language model provided in the embodiments of the present invention can generally also be executed by the server 105. Correspondingly, the adaptive fine-tuning device for the language model provided in the embodiments of the present invention can generally be set in the server 105.

[0036] It should be understood that the numbers of the above terminal devices, networks, and servers are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers.

[0037] As Figure 2 shown, the adaptive fine-tuning method for the language model provided in the embodiments of the present invention specifically includes the following steps:

[0038] S201. Collect a fine-tuning training sample set, where each fine-tuning training sample in the fine-tuning training sample set includes training text and label data corresponding to at least one fine-tuning task.

[0039] In this embodiment, the corresponding fine-tuning training sample set is collected according to the model fine-tuning requirements in different fields, where each fine-tuning training sample in the fine-tuning training sample set includes training text and label data corresponding to at least one fine-tuning task. That is, the training text in a fine-tuning training sample set usually belongs to the same field, such as the medical field, the financial field, the sports field, etc., and the label data of each training text can correspond to one or more fine-tuning tasks in the field, so that the model can be fine-tuned according to the characteristics of different tasks to improve the adaptation effect of the model fine-tuning to multiple tasks.

[0040] For example, in the fine-tuning task in the medical field scenario, it is necessary to classify medical record reports. After authorization, 1000 medical record reports and corresponding disease classification labels can be collected from the hospital's electronic health record system as the fine-tuning training sample set. These labels may be "heart disease", "diabetes", "hypertension", etc. Then each medical record report is a fine-tuning training sample, including the training text (medical record report) and the corresponding label data (disease classification). To ensure the diversity and representativeness of the data, medical records of patients of different ages, genders, and races can be randomly selected to ensure that the model can generalize to different patient groups after fine-tuning. If the entity extraction task of medical record reports needs to be performed simultaneously, entity labels in the medical record reports, such as names, dates, visit dates, departments visited, etc., can be collected, so as to obtain a fine-tuning training sample set that can perform multi-task fine-tuning. It provides the necessary data for model fine-tuning, enabling it to learn the semantic features of specific tasks (such as disease classification, entity extraction), and thus better adapt to the specific needs of the medical field during the fine-tuning process.

[0041] S202. Input the training text and the corresponding label data into the pre-trained language model to obtain the hidden representation of the training text.

[0042] In this embodiment, the collected training samples and the corresponding label data are input into the pre-trained language model. The specific language model can be a Transformer model. The Transformer model is a deep learning model based on the attention mechanism, consisting of an encoder and a decoder, and captures the relationships between different positions in the input sequence through the self-attention mechanism to process sequence data. The Transformer model is widely used in natural language processing tasks, such as text classification, machine translation, named entity recognition, sentiment analysis, etc.

[0043] After the collected training text is input into the language model, the hidden representation of the training text can be obtained through the processing of the language model. Specifically, the training text first undergoes an embedding transformation through an embedding layer to convert the input text into high-dimensional word embeddings. These embedding vectors not only include word vectors but also contain positional encoding for representing the position information of words in the sequence. The self-attention mechanism is applied to the transformed embedding vectors to obtain the corresponding hidden representation. The hidden representation of the training text can capture the semantic information of the text, enabling the model to understand the deep semantics of the input text and providing a basis for subsequent soft prompt generation and parameter fine-tuning.

[0044] S203. Use the pre-constructed soft prompt generator to perform adaptive generation processing on the hidden representation to obtain the corresponding soft prompt vector.

[0045] In this embodiment, the hidden representation is processed by the pre-constructed soft prompt generator for adaptive generation, so as to dynamically generate appropriate soft prompt vectors according to the semantics of the current input text. The specific soft prompt generator may include a downsampling matrix, a pooling layer, an activation function, and an upsampling matrix, which can adjust the generated soft prompts according to the characteristics of the input data, thereby increasing the amount of tunable parameters without depending on the soft prompt length. For example, adjusting parameters such as weights and biases in the generator can effectively increase the fine-tuning ability of the model.

[0046] In traditional soft prompt methods, soft prompts are usually a sequence of vectors with a fixed length. These vectors are updated as part of the model parameters during the fine-tuning process. The limitation of this method is that the number of optimizable parameters is limited by the length of the soft prompt, and randomly initialized soft prompts are likely to be mismatched with the task requirements, thus affecting the fine-tuning effect. Different from the way of randomly initializing a soft prompt, in this embodiment, the soft prompt generator can adaptively and dynamically adjust the content of the soft prompt according to the input data, rather than increasing its length, so that the input size processed by the model during inference remains unchanged. Therefore, it will not increase the complexity of model inference, and can generate soft prompts more relevant to the current task, helping the model better understand and process related tasks, thus improving the fine-tuning effect while maintaining the inference efficiency of the model.

[0047] S204. Re-input the hidden representation and the soft prompt vector into the language model to obtain the corresponding predicted output result.

[0048] In this embodiment, after the soft prompt vector is adaptively generated, the hidden representation and the soft prompt vector are input into the pre-trained language model together. The language model combines these two parts of information and performs inference and prediction on the hidden representation under the guidance of the soft prompt vector, so as to obtain the corresponding predicted output result, such as obtaining text classification results, named entity recognition results, and so on.

[0049] S205. Perform parameter fine-tuning according to the difference between the predicted output result and the label data until the preset end condition is met.

[0050] In this embodiment, based on the difference between the preset output result and the label data, parameter fine-tuning is performed under the constraint of the loss function. The specific loss function used can be the cross-entropy loss. During the fine-tuning process, the cross-entropy loss function is used to calculate the difference between the predicted result output by the evaluation model and the true label. According to this difference, the parameters of the model are fine-tuned through the gradient descent algorithm, especially the parameters of the soft prompt generator, so as to increase the number of tunable parameters without relying on the length of the soft prompt and improve the model fine-tuning effect. This fine-tuning process will be repeated until the preset end condition is met, such as the performance of the model reaches a predetermined index or the number of fine-tuning times reaches a predetermined number of iterations, etc. Through adaptive parameter fine-tuning, the model can more accurately adapt to specific fine-tuning tasks, improving the effectiveness and accuracy of the model in practical applications. At the same time, since only the parameters of the soft prompt generator can be updated, while maintaining the parameter adjustment efficiency, the consumption of computing resources is also reduced.

[0051] In the above embodiment, the present invention discloses an adaptive fine-tuning method for a language model. By collecting a fine-tuning training sample set, each fine-tuning training sample in the fine-tuning training sample set includes training text and label data corresponding to at least one fine-tuning task; the training text and the corresponding label data are input into a pre-trained language model to obtain the hidden representation of the training text; the hidden representation is adaptively generated by a pre-constructed soft prompt generator to obtain a corresponding soft prompt vector; the hidden representation and the soft prompt vector are re-input into the language model to obtain a corresponding predicted output result; parameter fine-tuning is performed according to the difference between the predicted output result and the label data until the preset end condition is met. By introducing a soft prompt generator to dynamically generate appropriate soft prompt vectors according to the input text for model fine-tuning, the amount of tunable parameters in the prompt fine-tuning is increased, without increasing the length of the soft prompt, without increasing the complexity of model inference, and can help the model better obtain semantic knowledge in fine-tuning, realizing effective and reliable model fine-tuning.

[0052] In one embodiment, as Figure 3 shown, step S203 includes:

[0053] S301. Input the hidden representation into a pre-constructed soft prompt generator;

[0054] S302. Perform downsampling processing on the hidden representation to reduce the hidden dimension of the hidden representation to a preset dimension, obtaining a downsampled vector;

[0055] S303. Confirm the currently set soft prompt length, and perform pooling processing on the downsampled vector according to the soft prompt length to obtain a pooled vector with the soft prompt length;

[0056] S304. Apply a preset activation function to the pooled vector for non-linear transformation processing, and perform upsampling processing on the corresponding activation vector to obtain a soft prompt vector with the dimension restored to the hidden dimension.

[0057] In this embodiment, the pre-constructed soft prompt generator includes a downsampling matrix, a pooling layer, an activation function, and an upsampling matrix. When adaptively generating a soft prompt vector, the obtained hidden representation is input into the soft prompt generator. First, the input hidden representation is downsampled by the downsampling matrix to reduce the dimension of the hidden representation and obtain the corresponding downsampled vector. For example, if the dimension of the input hidden representation is (s, d), where s is the length of the training text and d is the hidden dimension, then in the downsampling matrix, the dimension of the input hidden representation is reduced from d to k through a linear transformation (matrix multiplication), where k is a dimension less than d, to obtain a downsampled vector with the dimension of (s, k), thereby reducing the dimension of the data, which helps to reduce the complexity of subsequent calculations while retaining important features.

[0058] After that, confirm the currently set soft prompt length l. This soft prompt length can be a preset default length value, or it can be a length value changed according to the current task requirements, or it can also be a length value adaptively adjusted according to the parameters related to the current fine-tuning task. Based on the currently set soft prompt length, perform pooling processing on the downsampled vector through the pooling layer, such as average pooling processing or max pooling processing, etc., to reduce the length of the vector, compress the downsampled vector with length s to the pooled vector with soft prompt length l, and obtain a pooled vector with the dimension of (l, k). Further reduce the dimension of the data, make the soft prompt vector more compact, and at the same time retain the global information useful for the task.

[0059] After that, apply an activation function to the pooled vector for non-linear transformation, such as ReLu or GELU function, etc. By introducing non-linearity, it helps the model learn more complex features, thereby increasing the expression ability of the model. After being processed by the activation function, the dimension of the activation vector remains (l, k). Perform a linear transformation on the activation vector through the upsampling matrix to restore its dimension from k to the original hidden dimension d, thereby obtaining a soft prompt vector with the dimension of (l, d), making it compatible with the original hidden representation, providing an adapted soft prompt vector for the subsequent input of the language model. By adaptively generating a soft prompt vector adapted to the hidden representation through the soft prompt generator, the adaptability and performance of the model for specific tasks are improved.

[0060] In one embodiment, the confirming the currently set soft prompt length includes:

[0061] Obtain the length parameter of the training text and the input limit parameter of the language model, and confirm the currently fine-tuned task type;

[0062] According to the length parameter of the training text, the input limit parameter of the language model, and the current fine-tuning task type, obtain the corresponding optimal prompt length according to the adaptive length strategy;

[0063] Confirm the optimal prompt length as the currently set soft prompt length.

[0064] In this embodiment, when adaptively generating the soft prompt vector, the soft prompt length is adaptively scaled based on different features and parameters of the current fine-tuning task, so that the soft prompt length can be dynamically adjusted according to indicators such as the complexity of the input data and the capacity of the model. For example, for tasks with high complexity, the soft prompt length can be appropriately increased, and vice versa, so as to achieve an adaptive match between the soft prompt length and the fine-tuning task. Specifically, the length parameter of the training text can include, for example, the input length of the training text currently input into the model, the average length of all training texts in the fine-tuning training sample set, etc., the input limit parameter of the language model can include, for example, the maximum sequence length, the model depth, etc., and the task type can be, for example, a text classification task, a question-answering task, a sentiment analysis task, a text summarization task, etc. Based on the obtained length parameter of the training text, the input limit parameter of the language model, and the current fine-tuning task type, an optimal prompt length is determined by comprehensively considering factors from different perspectives according to the adaptive length strategy, and it is determined as the currently set soft prompt length, so that the soft prompt length can be determined more flexibly and comprehensively to adapt to different task requirements and model limitations, and further improve the model fine-tuning effect.

[0065] In one embodiment, the obtaining the corresponding optimal prompt length according to the length parameter of the training text, the input limit parameter of the language model, and the current fine-tuning task type according to the adaptive length strategy includes:

[0066] Calculate the maximum prompt length according to the input length of the currently input training text and the maximum sequence length;

[0067] Obtain the pre-allocated task coefficient according to the current fine-tuning task type, and calculate the reference prompt length according to the task coefficient and the average length of all training texts;

[0068] Confirm whether the reference prompt length is less than the maximum prompt length. If so, use the reference prompt length as the current optimal prompt length.

[0069] In this embodiment, before determining the optimal prompt length, based on the input length of the current input training text and the maximum sequence length, the maximum prompt length is calculated as the verification basis for the soft prompt length. For example, if the model limits the maximum sequence length to M and the length of the current input training text is s, since the generated soft prompt vector and the hidden representation (with the same length s) need to be input into the language model during fine-tuning, the length of the generated soft prompt vector cannot exceed the maximum prompt length, that is, M - s. This ensures that the generated soft prompt vector does not exceed the processing capacity of the model and can also meet the requirements of specific tasks.

[0070] When applying the adaptive length strategy, the task coefficients pre-allocated are obtained based on the type of the current fine-tuning task. Specifically, corresponding task coefficients of appropriate sizes can be pre-allocated according to the task complexity. For example, the context information required for tasks such as sentiment analysis, text classification, question answering tasks, and text summarization gradually increases, so correspondingly increasing task coefficients can be allocated, such as 0.1, 0.2, 0.3, 0.4, etc. The reference prompt length is calculated based on the task coefficient of the current fine-tuning task obtained and the average length of all training texts. For example, multiplying the two gives the reference prompt length, or in multi-task fine-tuning, the initial reference prompt lengths under different tasks are first obtained by multiplying respectively, and then the initial reference prompt lengths under different tasks are weighted and summed to obtain the final reference prompt length. By introducing the task coefficient, the length of the soft prompt can be adjusted according to the requirements of different tasks, making the model more adaptable to specific tasks. Finally, the generated reference prompt length is verified by the maximum prompt length. If the reference prompt length is less than the maximum prompt length, the reference prompt length is used as the optimal prompt length. On the premise of ensuring that it does not exceed the model input limit, a suitable prompt length is selected, balancing the model performance and the utilization of computing resources and ensuring the efficient operation of the model.

[0071] In one embodiment, step S204 includes:

[0072] Concatenate the hidden representation and the soft prompt vector to obtain a concatenated input vector;

[0073] Input the concatenated input vector into the language model for inference and prediction of the current fine-tuning task to obtain the corresponding prediction output result.

[0074] In this embodiment, based on the soft prompt vector adaptively generated by the soft prompt generator, the initially obtained hidden representation and the soft prompt vector are concatenated to obtain a concatenated output vector. The concatenation operation enables the language model to simultaneously consider the features of the original text and the information provided by the soft prompt in its input, which helps the model better understand and process the current fine-tuning task. Then, the concatenated input vector is input into the language model, and the text sequence of the concatenated input vector is reprocessed based on the self-attention mechanism, so as to perform inference and prediction according to the newly concatenated input. For example, if the current fine-tuning task is sentiment analysis, the language model will predict and output a sentiment classification prediction probability based on the concatenated input vector. By inputting the concatenated input vector into the language model, it is possible to combine the information of the original text and the soft prompt for prediction, more accurately capture the task-related features, thereby improving the prediction accuracy and robustness of the model for specific tasks and achieving more effective inference and prediction in the fine-tuning task.

[0075] In one embodiment, when the language model is a multi-layer architecture, the adaptive generation process of the hidden representation using the pre-constructed soft prompt generator to obtain the corresponding soft prompt vector specifically includes:

[0076] The pre-constructed soft prompt generator is used to perform an adaptive generation process on the hidden representation of the nth layer to obtain the soft prompt vector of the nth layer, where n = 1, 2,... L, and L is the number of layers of the language model;

[0077] The step of re-inputting the hidden representation and the soft prompt vector into the language model to obtain the corresponding prediction output result includes:

[0078] The hidden representation of the nth layer and the soft prompt vector of the nth layer are concatenated and input into the nth layer of the language model to obtain a new hidden representation of the nth layer;

[0079] According to the input length of the training text, a corresponding number of elements are extracted from the new hidden representation of the nth layer as the hidden representation of the (n + 1)th layer;

[0080] The above process of generating the soft prompt vector and extracting the hidden representation is looped until the new hidden representation of the Lth layer is obtained;

[0081] Based on the new hidden representation of the Lth layer, the inference and prediction of the current fine-tuning task are performed to obtain the corresponding prediction output result.

[0082] In this embodiment, in order to capture semantic information at different levels and long-distance dependencies, a general language model such as the Transformer model usually adopts a multi-layer architecture. The multi-layer Transformer model is composed of multiple encoder layers, and each layer contains a self-attention mechanism and a feed-forward neural network (FFN). This architecture enables the model to process sequential data and capture complex relationships and long-distance dependencies within the sequence. When the language model is a multi-layer structure, the adaptive generation of soft prompt vectors and the acquisition of hidden representations is a cyclic interaction process. A soft prompt generator is applied to each layer of the language model. After generating the corresponding soft prompt vector based on the input representation of the current layer, i.e., the hidden representation, the two are concatenated and then input into the current layer, and the input representation of the next layer, i.e., the hidden representation of the next layer, is obtained based on the output of the current layer.

[0083] Specifically, if the number of layers of the current language model is L, where L is a positive integer greater than 1, such as 6, 12, etc., the same soft prompt generator is used for each layer. After obtaining the hidden representation Hn of that layer, an adaptive process is performed, that is, an adaptive generation process is performed on the hidden representation of the nth layer to obtain the soft prompt vector P of the nth layer n , where n = 1, 2,..., L. In particular, the hidden representation H1 of the first layer is the representation obtained by inputting the initial embedding vector E into the first layer of the model, where the initial embedding vector E is obtained by performing an embedding transformation on the input training text, and the hidden representations of subsequent layers are extracted based on the output of the previous layer. The generated soft prompt vector P n is concatenated with the hidden representation H n along the feature dimension, and then the concatenated vector is input into the nth layer to obtain the new hidden representation H n ' of the nth layer. According to the input length s of the training text, a corresponding number of elements are extracted from the new hidden representation H n ', for example, the first s elements are extracted, and these elements are used as the input representation of the (n + 1)th layer, that is, the hidden representation H n+1 of the (n + 1)th layer; continue to perform an adaptive generation process on the hidden representation H n+1 through the soft prompt generator to generate the soft prompt vector P n+1 of the (n + 1)th layer, and after concatenating and inputting it into the (n + 1)th layer, the hidden representation H n+2 of the next layer is extracted from the output. The process of generating soft prompt vectors and extracting hidden representations is continuously looped until the output of the last layer, that is, the new hidden representation H L ' of the Lth layer, and the hidden representation H L' is the output of the last layer of the language model, which contains the model's comprehensive understanding of the input data. Therefore, subsequent processing is carried out based on the task requirements of the current fine-tuning task, such as text classification, question answering, text generation, etc., to obtain the prediction output result of the current fine-tuning task.

[0084] By repeatedly executing the above process of generating soft prompt vectors and extracting hidden representations, a dynamic soft prompt vector is provided for each layer by the soft prompt generator, enhancing the performance of the model at each layer. And by combining the soft prompt vector with the original hidden representation and inputting it into each layer for transmission, the model can more effectively capture task-related features at each layer, ensuring the effective transmission of information between model layers, achieving more reliable and effective inference prediction during the fine-tuning process, and improving the fine-tuning effect.

[0085] In one embodiment, step S205 includes:

[0086] Calculate the loss value of the current fine-tuning round according to the difference between the prediction output result and the label data;

[0087] Freeze the parameters of the language model, adjust the parameters of the current soft prompt generator according to the loss value of the current fine-tuning round, and use the adjusted soft prompt generator as the current soft prompt generator for the next fine-tuning round;

[0088] Repeatedly execute the above process of adjusting the parameters of the soft prompt generator until a preset end condition is met.

[0089] In this embodiment, the parameter adjustment process of model fine-tuning is an iterative loop. Based on the difference between the predicted output result of the current round and the label data, the loss value of the current fine-tuning round is calculated. The loss value can be specifically calculated according to the loss function applicable to the supervised learning method. For example, cross-entropy is used for calculation. During the fine-tuning process, the parameters of the language model are kept unchanged (i.e., the parameters of the language model are frozen), and only the parameters of the soft prompt generator are adjusted. Specifically, the gradient can be calculated based on the loss value, and backpropagation is used to adjust the parameters of the current soft prompt generator, so as to adapt to a specific task by adjusting the parameters of the soft prompt generator without changing the parameters of the pre-trained language model, retaining the general knowledge of the language model and helping to improve the performance of the model on specific tasks. The adjusted soft prompt generator is used as the current soft prompt generator for the next fine-tuning round, that is, the soft prompt generator for each fine-tuning round uses the latest adjusted parameters to continuously improve the performance of the model on the fine-tuning task under the constraint of the loss function. The above-mentioned parameter adjustment process of the soft prompt generator is repeatedly executed until a preset end condition such as reaching a predetermined number of rounds or the loss value drops to a threshold is met, then the model fine-tuning process is completed. By updating the parameters of the soft prompt generator, the fine-tuning process does not require increasing the length of the soft prompt, nor does it increase the complexity of model inference, increasing the number of tunable parameters while ensuring computational efficiency.

[0090] It should be noted that there is not necessarily a certain sequence among the above steps. Those of ordinary skill in the art can understand according to the description of the embodiments of the present invention that in different embodiments, the above steps can have different execution sequences, that is, they can be executed in parallel or exchanged, etc.

[0091] For further reference Figure 4 , as an implementation of the method shown above Figure 2 , the present invention provides an embodiment of an adaptive fine-tuning device for a language model. This device embodiment corresponds to the method embodiment shown in Figure 2 , and this device can be specifically applied to various electronic devices.

[0092] As Figure 4 shown, the adaptive fine-tuning device 40 of the language model described in this embodiment includes:

[0093] An acquisition module 401, configured to acquire a fine-tuning training sample set, where each fine-tuning training sample in the fine-tuning training sample set includes training text and label data corresponding to at least one fine-tuning task;

[0094] A characterization module 402, configured to input the training text and the corresponding label data into a pre-trained language model to obtain a hidden representation of the training text;

[0095] An adaptive generation module 403, configured to perform adaptive generation processing on the hidden representation by using a pre-constructed soft prompt generator to obtain a corresponding soft prompt vector;

[0096] A prediction output module 404, configured to re-input the hidden representation and the soft prompt vector into the language model to obtain a corresponding prediction output result;

[0097] A parameter fine-tuning module 405, configured to perform parameter fine-tuning according to the difference between the prediction output result and the label data until a preset end condition is met.

[0098] As used in the present invention, a module refers to a series of computer program instruction segments that can complete specific functions, which is more suitable for describing the adaptive fine-tuning execution process of a language model. For the specific implementation manners of each module, please refer to the corresponding method embodiments above, and details will not be described herein again.

[0099] In one embodiment, the adaptive generation module 403 includes:

[0100] An input unit, configured to input the hidden representation into a pre-constructed soft prompt generator;

[0101] A downsampling unit, configured to perform downsampling processing on the hidden representation to reduce the hidden dimension of the hidden representation to a preset dimension, obtaining a downsampled vector;

[0102] A pooling unit, configured to confirm the currently set soft prompt length, and perform pooling processing on the downsampled vector according to the soft prompt length to obtain a pooled vector with the soft prompt length;

[0103] An activation and upsampling unit, configured to perform non-linear transformation processing on the pooled vector by applying a preset activation function, and perform upsampling processing on the corresponding activation vector to obtain a soft prompt vector with the dimension restored to the hidden dimension.

[0104] In one embodiment, the adaptive generation module 403 further includes:

[0105] A parameter acquisition unit, configured to acquire the length parameter of the training text and the input limit parameter of the language model, and confirm the currently fine-tuned task type;

[0106] A prompt length generation unit, configured to obtain a corresponding optimal prompt length according to the length parameter of the training text, the input limit parameter of the language model, and the currently fine-tuned task type according to an adaptive length strategy;

[0107] A prompt length setting unit, configured to confirm the optimal prompt length as the currently set soft prompt length.

[0108] In one embodiment, the length parameter of the training text includes the average length of all training texts and the input length of the currently input training text, the input limit parameter of the language model is the maximum sequence length, and the prompt length generation unit includes:

[0109] A first calculation unit, configured to calculate a maximum prompt length according to the input length of the currently input training text and the maximum sequence length;

[0110] A second calculation unit, configured to obtain a pre-allocated task coefficient according to the currently fine-tuned task type, and calculate a reference prompt length according to the task coefficient and the average length of all training texts;

[0111] A verification unit, configured to confirm whether the reference prompt length is less than the maximum prompt length, and if so, use the reference prompt length as the current optimal prompt length.

[0112] In one embodiment, the prediction output module 404 includes:

[0113] A splicing unit, configured to splice the hidden representation and the soft prompt vector to obtain a spliced input vector;

[0114] A prediction output unit, configured to input the spliced input vector into the language model for inference prediction of the current fine-tuning task, and obtain a corresponding prediction output result.

[0115] In one embodiment, when the language model is a multi-layer architecture, the adaptive generation module 403 is specifically configured to:

[0116] Perform adaptive generation processing on the hidden representation of the nth layer by using a pre-constructed soft prompt generator to obtain a soft prompt vector of the nth layer, where n = 1, 2,... L, and L is the number of layers of the language model;

[0117] The prediction output module 404 includes:

[0118] A splicing unit, configured to splice and input the hidden representation of the nth layer and the soft prompt vector of the nth layer into the nth layer of the language model to obtain a new hidden representation of the nth layer;

[0119] An extraction unit, configured to extract a corresponding number of elements from the new hidden representation of the nth layer according to the input length of the training text as the hidden representation of the (n + 1)th layer;

[0120] A loop execution unit, configured to loop and execute the processes of generating the soft prompt vector and extracting the hidden representation until a new hidden representation of the Lth layer is obtained;

[0121] An inference output unit, configured to perform inference prediction of the current fine-tuning task based on the new hidden representation of the L-th layer, and obtain a corresponding prediction output result.

[0122] In one embodiment, the parameter fine-tuning module 405 includes:

[0123] A loss calculation unit, configured to calculate the loss value of the current fine-tuning round according to the difference between the prediction output result and the label data;

[0124] A parameter adjustment unit, configured to freeze the parameters of the language model, adjust the parameters of the current soft prompt generator according to the loss value of the current fine-tuning round, and use the adjusted soft prompt generator as the current soft prompt generator for the next fine-tuning round;

[0125] A loop adjustment unit, configured to loop through the parameter adjustment process of the above soft prompt generator until a preset end condition is met.

[0126] In the above embodiment, the present invention discloses an adaptive fine-tuning device for a language model. By collecting a fine-tuning training sample set, each fine-tuning training sample in the fine-tuning training sample set includes training text and label data corresponding to at least one fine-tuning task; inputting the training text and the corresponding label data into a pre-trained language model to obtain a hidden representation of the training text; using a pre-constructed soft prompt generator to perform adaptive generation processing on the hidden representation to obtain a corresponding soft prompt vector; re-inputting the hidden representation and the soft prompt vector into the language model to obtain a corresponding prediction output result; performing parameter fine-tuning according to the difference between the prediction output result and the label data until a preset end condition is met. By introducing a soft prompt generator to dynamically generate appropriate soft prompt vectors according to the input text for model fine-tuning, the amount of tunable parameters in prompt fine-tuning is increased, without increasing the length of the soft prompt, without increasing the complexity of model inference, and can help the model better obtain semantic knowledge in fine-tuning, realizing effective and reliable model fine-tuning.

[0127] Another embodiment of the present invention provides a computer device, as Figure 5 shown, the computer device 50 includes:

[0128] One or more processors 501 and a memory 502, Figure 5 Taking one processor 501 as an example for introduction, the processor 501 and the memory 502 can be connected by a bus or other means, Figure 5 Taking the connection by bus as an example.

[0129] The processor 501 is used to complete various control logics of the computer device 50, and it can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISCMachine), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components. Moreover, the processor 501 can also be any conventional processor, microprocessor, or state machine. The processor 501 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0130] The memory 502, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions corresponding to the adaptive fine-tuning method of the language model in the embodiments of the present invention. The processor 501 executes various functional applications and data processing of the computer device 50 by running the non-volatile software programs, instructions, and units stored in the memory 502, that is, implements the adaptive fine-tuning method of the language model in the above method embodiments.

[0131] The memory 502 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the computer device 50, etc. In addition, the memory 502 can include high-speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 502 optionally includes a memory remotely set relative to the processor 501, and these remote memories can be connected to the computer device 50 through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and their combinations. One or more units are stored in the memory 502 and, when executed by one or more processors 501, perform the steps of the adaptive fine-tuning method of the language model in any of the above method embodiments.

[0132] In the above embodiments, the present invention discloses a computer device. By collecting a fine-tuning training sample set, each fine-tuning training sample in the fine-tuning training sample set includes training text and label data corresponding to at least one fine-tuning task; inputting the training text and the corresponding label data into a pre-trained language model to obtain a hidden representation of the training text; using a pre-constructed soft prompt generator to perform adaptive generation processing on the hidden representation to obtain a corresponding soft prompt vector; re-inputting the hidden representation and the soft prompt vector into the language model to obtain a corresponding predicted output result; performing parameter fine-tuning according to the difference between the predicted output result and the label data until a preset end condition is satisfied. By introducing a soft prompt generator to dynamically generate appropriate soft prompt vectors according to the input text for model fine-tuning, the amount of tunable parameters in prompt fine-tuning is increased without increasing the length of the soft prompt, the complexity of model inference is not increased, and it can help the model better obtain semantic knowledge during fine-tuning, realizing effective and reliable model fine-tuning.

[0133] An embodiment of the present invention provides a non-volatile computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the steps of the adaptive fine-tuning method of the language model in any of the above method embodiments are executed.

[0134] In the above embodiments, the present invention discloses a non-volatile computer-readable storage medium. By collecting a fine-tuning training sample set, each fine-tuning training sample in the fine-tuning training sample set includes training text and label data corresponding to at least one fine-tuning task; inputting the training text and the corresponding label data into a pre-trained language model to obtain a hidden representation of the training text; using a pre-constructed soft prompt generator to perform adaptive generation processing on the hidden representation to obtain a corresponding soft prompt vector; re-inputting the hidden representation and the soft prompt vector into the language model to obtain a corresponding predicted output result; performing parameter fine-tuning according to the difference between the predicted output result and the label data until a preset end condition is satisfied. By introducing a soft prompt generator to dynamically generate appropriate soft prompt vectors according to the input text for model fine-tuning, the amount of tunable parameters in prompt fine-tuning is increased without increasing the length of the soft prompt, the complexity of model inference is not increased, and it can help the model better obtain semantic knowledge during fine-tuning, realizing effective and reliable model fine-tuning.

[0135] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0136] The present invention can be used in numerous general-purpose or special-purpose computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet-type devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0137] In summary, in an adaptive fine-tuning method, device, device, and medium of a language model disclosed by the present invention, the method includes: collecting a fine-tuning training sample set, where each fine-tuning training sample in the fine-tuning training sample set includes training text and label data corresponding to at least one fine-tuning task; inputting the training text and the corresponding label data into a pre-trained language model to obtain a hidden representation of the training text; using a pre-constructed soft prompt generator to perform adaptive generation processing on the hidden representation to obtain a corresponding soft prompt vector; re-inputting the hidden representation and the soft prompt vector into the language model to obtain a corresponding predicted output result; and performing parameter fine-tuning according to the difference between the predicted output result and the label data until a preset end condition is met. By introducing a soft prompt generator to dynamically generate appropriate soft prompt vectors according to the input text for model fine-tuning, the amount of tunable parameters in prompt fine-tuning is increased without increasing the length of the soft prompt, without increasing the complexity of model inference, and can help the model better obtain semantic knowledge during fine-tuning, achieving effective and reliable model fine-tuning.

[0138] Of course, those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium, and when the computer program is executed, it can include the processes of the above method embodiments. The storage medium can be a memory, a magnetic disk, a floppy disk, a flash memory, an optical memory, etc.

[0139] It should be noted that if there are software tools or components of other companies in the embodiments of the present application, they are only used for illustrative introduction and do not represent actual use. It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. An adaptive fine-tuning method for a language model, characterized in that, Including: Collecting a fine-tuning training sample set, where each fine-tuning training sample in the fine-tuning training sample set includes training text and label data corresponding to at least one fine-tuning task; Inputting the training text and the corresponding label data into a pre-trained language model to obtain a hidden representation of the training text; Performing an adaptive generation process on the hidden representation by using a pre-constructed soft prompt generator to obtain a corresponding soft prompt vector; Re-inputting the hidden representation and the soft prompt vector into the language model to obtain a corresponding predicted output result; Performing parameter fine-tuning according to the difference between the predicted output result and the label data until a preset end condition is met.

2. The adaptive fine-tuning method of the language model according to claim 1, wherein The performing an adaptive generation process on the hidden representation by using a pre-constructed soft prompt generator to obtain a corresponding soft prompt vector includes: Inputting the hidden representation into a pre-constructed soft prompt generator; Performing downsampling on the hidden representation to reduce the hidden dimension of the hidden representation to a preset dimension to obtain a downsampled vector; Confirming the currently set soft prompt length, and performing pooling on the downsampled vector according to the soft prompt length to obtain a pooled vector with the soft prompt length; Performing a non-linear transformation on the pooled vector by applying a preset activation function, and performing upsampling on the corresponding activated vector to obtain a soft prompt vector with the dimension restored to the hidden dimension.

3. The adaptive fine-tuning method of the language model according to claim 2, wherein The confirming the currently set soft prompt length includes: Obtaining the length parameter of the training text and the input limit parameter of the language model, and confirming the currently fine-tuned task type; According to the length parameter of the training text, the input limit parameter of the language model, and the currently fine-tuned task type, obtaining a corresponding optimal prompt length according to an adaptive length strategy; Confirming the optimal prompt length as the currently set soft prompt length.

4. The adaptive fine-tuning method of the language model according to claim 3, characterized in that The length parameter of the training text includes the average length of all training texts and the input length of the currently input training text. The input limit parameter of the language model is the maximum sequence length. The obtaining a corresponding optimal prompt length according to the length parameter of the training text, the input limit parameter of the language model, and the currently fine-tuned task type according to an adaptive length strategy includes: Calculating a maximum prompt length according to the input length of the currently input training text and the maximum sequence length; Obtaining a pre-allocated task coefficient according to the currently fine-tuned task type, and calculating a reference prompt length according to the task coefficient and the average length of all training texts; Confirming whether the reference prompt length is less than the maximum prompt length. If so, taking the reference prompt length as the current optimal prompt length.

5. The adaptive fine-tuning method of the language model according to claim 1, characterized in that The re-inputting the hidden representation and the soft prompt vector into the language model to obtain a corresponding predicted output result includes: Concatenating the hidden representation and the soft prompt vector to obtain a concatenated input vector; Inputting the concatenated input vector into the language model for inference prediction of the current fine-tuning task to obtain a corresponding predicted output result.

6. The adaptive fine-tuning method of the language model according to claim 1, characterized in that When the language model is a multi-layer architecture, adaptively generating the hidden representation by using a pre-constructed soft prompt generator to obtain a corresponding soft prompt vector, specifically including: Using the pre-constructed soft prompt generator to perform adaptive generation processing on the hidden representation of the n-th layer to obtain the soft prompt vector of the n-th layer, where n = 1, 2, … L, and L is the number of layers of the language model; Re-inputting the hidden representation and the soft prompt vector into the language model to obtain a corresponding prediction output result, including: Concatenating and inputting the hidden representation of the n-th layer and the soft prompt vector of the n-th layer into the n-th layer of the language model to obtain a new hidden representation of the n-th layer; Extracting a corresponding number of elements from the new hidden representation of the n-th layer according to the input length of the training text as the hidden representation of the (n + 1)-th layer; Repeatedly executing the processes of generating the soft prompt vector and extracting the hidden representation until the new hidden representation of the L-th layer is obtained; Performing inference prediction of the current fine-tuning task according to the new hidden representation of the L-th layer to obtain a corresponding prediction output result.

7. The adaptive fine-tuning method of the language model according to claim 1, characterized in that Performing parameter fine-tuning according to the difference between the prediction output result and the label data until a preset end condition is met, including: Calculating the loss value of the current fine-tuning round according to the difference between the prediction output result and the label data; Freezing the parameters of the language model, adjusting the parameters of the current soft prompt generator according to the loss value of the current fine-tuning round, and using the adjusted soft prompt generator as the current soft prompt generator for the next fine-tuning round; Repeatedly executing the parameter adjustment process of the soft prompt generator until a preset end condition is met.

8. An adaptive fine-tuning device for a language model, characterized in that, Including: An acquisition module for acquiring a fine-tuning training sample set, where each fine-tuning training sample in the fine-tuning training sample set includes training text and label data corresponding to at least one fine-tuning task; A representation module for inputting the training text and the corresponding label data into a pre-trained language model to obtain the hidden representation of the training text; An adaptive generation module for adaptively generating the hidden representation by using a pre-constructed soft prompt generator to obtain a corresponding soft prompt vector; A prediction output module for re-inputting the hidden representation and the soft prompt vector into the language model to obtain a corresponding prediction output result; A parameter fine-tuning module for performing parameter fine-tuning according to the difference between the prediction output result and the label data until a preset end condition is met.

9. A computer device, characterized in that, Including at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the adaptive fine-tuning method of the language model according to any one of claims 1-7.

10. A non-volatile computer-readable storage medium, characterized in that, The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the adaptive fine-tuning method of the language model according to any one of claims 1-7.

Images