Large language model fine-tuning method and device based on network layer similarity, electronic equipment and storage medium

Abstract

The present disclosure relates to a large language model fine-tuning method and device based on network layer similarity, electronic equipment and storage medium. The method comprises: determining the network layer similarity between any two adjacent network layers in an initial large language model according to a sample data set corresponding to a target natural language processing task; constructing a sub-model corresponding to the initial large language model according to the network layer similarity between any two adjacent network layers in the initial large language model; training the sub-model according to the sample data set to determine a trained sub-model; and adjusting the initial large language model according to the model parameters of the trained sub-model to determine a target large language model having the ability to process the target natural language processing task. Through the embodiments of the present disclosure, the network layer similarity can be used to select network layers, construct a sub-model for fine-tuning of the initial large language model, protect user data and model parameter privacy, and improve the efficiency of model fine-tuning.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and storage medium for fine-tuning a large language model based on network layer similarity. Background Technology

[0002] In existing technologies, to adapt Large Language Models (LLMs) to specific domain tasks and datasets, fine-tuning is typically required. However, this fine-tuning process may involve users' private data, necessitating measures to prevent data leakage and enhance the security and privacy of the fine-tuning process. Furthermore, since LLMs usually have a large number of parameters, directly fine-tuning the entire LLM requires significant computational and time costs, limiting its usability and flexibility, and making efficient model fine-tuning difficult in resource-constrained application scenarios. Summary of the Invention

[0003] In view of this, this disclosure proposes a technical solution for fine-tuning a large language model based on network layer similarity, an apparatus, an electronic device, and a storage medium.

[0004] According to one aspect of this disclosure, a method for fine-tuning a large language model based on network layer similarity is provided, comprising: determining the network layer similarity between any two adjacent network layers in an initial large language model based on a sample dataset corresponding to a target natural language processing task; constructing a sub-model corresponding to the initial large language model based on the network layer similarity between any two adjacent network layers in the initial large language model; training the sub-model based on the sample dataset to determine the trained sub-model; and adjusting the initial large language model based on the model parameters corresponding to the trained sub-model to determine a target large language model, wherein the target large language model has the ability to process the target natural language processing task.

[0005] In one possible implementation, determining the network layer similarity between any two adjacent network layers in the initial large language model based on the sample dataset corresponding to the target natural language processing task includes: inputting any sample data included in the sample dataset into the initial large language model; for any network layer of the initial large language model, determining the output feature corresponding to the network layer based on the output data of the network layer; and determining the network layer similarity between any two adjacent network layers based on the output feature corresponding to each network layer.

[0006] In one possible implementation, the output features corresponding to any network layer include the sentence vector corresponding to that network layer; after inputting any sample data from the sample dataset into the initial large language model, for any network layer of the initial large language model, determining the output features corresponding to that network layer based on the output data of that network layer includes: for any network layer of the initial large language model, determining the hidden layer output state and position weight corresponding to each position token in the output data of that network layer based on the length of the input sequence of the sample data input to the initial large language model; for any network layer of the initial large language model, determining the sentence vector corresponding to that network layer based on the hidden layer output state and position weight corresponding to each position token in the output data of that network layer.

[0007] In one possible implementation, determining the network layer similarity between any two adjacent network layers based on the output features corresponding to each network layer includes: using the central kernel analysis method to determine the network layer similarity between any two adjacent network layers based on the sentence vector corresponding to each network layer.

[0008] In one possible implementation, constructing a sub-model corresponding to the initial large language model based on the network layer similarity between any two adjacent network layers in the initial large language model includes: using a hierarchical clustering method, iteratively clustering all network layers of the initial large language model based on the network layer similarity between any two adjacent network layers to determine multiple cluster sets and a clustering index corresponding to each cluster set, until the number of cluster sets equals the preset number of network layers that can be stored. Each cluster set includes multiple adjacent network layers, and the clustering index corresponding to any cluster set includes the number of network layers included in the cluster set and the position of each network layer in the initial large language model. For any cluster set, based on the clustering index corresponding to the cluster set, determining the representative network layer corresponding to the cluster set among the multiple network layers included in the cluster set. Finally, determining the sub-model based on the representative network layer corresponding to each cluster set.

[0009] In one possible implementation, determining the sub-model based on the representative network layer corresponding to each cluster set includes: for any cluster set, copying the representative network layer corresponding to the cluster set according to the clustering index corresponding to the cluster set to determine multiple copied representative network layers corresponding to the cluster set, wherein the number of copied representative network layers is equal to the number of network layers included in the cluster set; and connecting the multiple representative network layers corresponding to each cluster set according to the clustering index corresponding to each cluster set to determine the sub-model.

[0010] In one possible implementation, adjusting the initial large language model based on the model parameters corresponding to the trained sub-models to determine the target large language model includes: using the clustering index corresponding to each cluster set to determine the parameter correspondence between the trained sub-models and the initial large language model; and replacing the model parameters corresponding to the initial large language model based on the parameter correspondence and the model parameters corresponding to the trained sub-models to determine the target large language model.

[0011] According to another aspect of this disclosure, a large language model fine-tuning device based on network layer similarity is provided, comprising: a similarity analysis module, used to determine the network layer similarity between any two adjacent network layers in an initial large language model based on a sample dataset corresponding to a target natural language processing task; a sub-model construction module, used to construct a sub-model corresponding to the initial large language model based on the network layer similarity between any two adjacent network layers in the initial large language model; a sub-model training module, used to train the sub-model based on the sample dataset to determine the trained sub-model; and a model parameter adjustment module, used to adjust the initial large language model based on the model parameters corresponding to the trained sub-model to determine a target large language model, wherein the target large language model has the ability to process the target natural language processing task.

[0012] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing instructions stored in the memory.

[0013] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided that stores computer program instructions thereon, wherein the computer program instructions, when executed by a processor, implement the above-described method.

[0014] In this embodiment, based on the sample dataset corresponding to the target natural language processing task, the network layer similarity between any two adjacent network layers in the initial large language model is determined. This allows for the identification of structurally and functionally similar network layers within the initial large language model based on this similarity. By removing redundant network layers with high similarity, and fully utilizing the emerging modular structure within the initial large language model, a sub-model corresponding to the initial large language model is constructed. Based on the sample dataset, the sub-model can be trained to determine the trained sub-model. Then, based on the model parameters corresponding to the trained sub-model, the initial large language model can be adjusted in a targeted and efficient manner. The knowledge related to the target natural language processing task acquired from the trained sub-model is integrated into the initial large language model, resulting in a target large language model capable of handling the target natural language processing task. This embodiment ensures that the model parameters corresponding to the initial large language model are not fully disclosed and avoids the external transmission of sample datasets that may involve user privacy. This improves both the efficiency and security of model fine-tuning, enhancing both security and privacy protection.

[0015] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0016] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0017] Figure 1 A flowchart is shown illustrating a method for fine-tuning a large language model based on network layer similarity according to an embodiment of the present disclosure;

[0018] Figure 2 A schematic diagram illustrating the process of a large language model fine-tuning method based on network layer similarity according to an embodiment of the present disclosure is shown.

[0019] Figure 3 A block diagram of a large language model fine-tuning apparatus based on network layer similarity according to an embodiment of the present disclosure is shown.

[0020] Figure 4 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0021] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0022] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0023] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0024] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0025] In existing technologies, large language models (LLMs), such as BERT (Bidirectional Encoder Representation from Transformers), GPT (Generative Pre-Training) series models, T5 (Transfer Text-to-Text Transformer) models, and LlaMA (Large Language Model Meta AI) models, have become an important foundation for research and applications in the field of natural language processing (NLP). These models typically consist of billions or even trillions of parameters and, through pre-training on large-scale text data, have acquired powerful language understanding and generation capabilities. They have demonstrated outstanding performance in various NLP tasks, including but not limited to text classification, question answering, machine translation, and text generation. As the model size continues to increase, the generalization ability of LLMs also continues to improve, enabling them to handle more complex language tasks.

[0026] To adapt LLMs to specific domain tasks and datasets, fine-tuning of their model parameters is often necessary. However, this fine-tuning process may involve the use of users' private data, such as personal identification, health records, or financial information. Furthermore, the model structure and parameters of LLMs are the intellectual property of the LLM owner, and there is a risk of improper disclosure or copying during fine-tuning, leading to infringement of the LLM owner's intellectual property rights. Therefore, to improve the security and privacy of the LLM fine-tuning process, it is necessary to prevent the leakage of LLM-related data.

[0027] On the other hand, since the number of parameters in an LLM model is usually large, fine-tuning the entire LLM directly would require a lot of computational and time costs, and would limit the availability and flexibility of the LLM model, making it difficult to achieve efficient model fine-tuning in resource-constrained application scenarios.

[0028] To address the aforementioned issues, various parameter fine-tuning techniques for LLMs have been proposed in the prior art. These include: Adapter optimization, which involves inserting small trainable neural network modules into existing LLMs and updating only the parameters of these modules, rather than the entire model; BitFit optimization, which updates only a small subset of parameters in the LLM model, such as bias terms, to reduce the number of parameters requiring fine-tuning; Low-rank optimization (LoRA), which captures parameter changes during fine-tuning by inserting low-rank matrices into the weight matrix, thereby reducing the number of parameters that need to be stored and updated; and Offsite-Tuning, which protects model privacy by compressing and transferring model layers between the server and the user, where the user receives the compressed model layers, fine-tunes them on local data, and then sends the updated layer parameters back to the server for integration.

[0029] While the aforementioned parameter fine-tuning techniques can improve data privacy and / or tuning efficiency, they often come with performance sacrifices, failing to fully realize the performance and efficiency of LLMs. For example, adapter optimization and bit parameter optimization techniques, while reducing the number of parameters requiring fine-tuning, may not fully utilize all the information in the LLM, thus affecting the final performance of the fine-tuned LLM. Low-rank optimization techniques have high requirements for the model structure of the LLM itself and may not be suitable for all types of LLMs. Offline fine-tuning techniques have strong advantages in privacy protection, but may lose some LLM performance due to the uniform selection of layers, and cannot be customized for tasks or designed for the characteristics of the internal network of LLMs.

[0030] In view of this, this disclosure provides a method for fine-tuning a large language model based on network layer similarity. This method utilizes the network layer similarity of LLMs to selectively choose network layers and construct sub-models that fully leverage the model information of the initial large language model. After training the sub-models using sample datasets, the model parameters of the trained sub-models can be used to adjust the initial large language model. This effectively improves the efficiency of model fine-tuning while protecting user data and model parameter privacy. The following details the embodiment of the large language model fine-tuning method based on network layer similarity of this disclosure.

[0031] Figure 1 A flowchart illustrating a large language model fine-tuning method based on network layer similarity according to an embodiment of this disclosure is shown. This large language model fine-tuning method based on network layer similarity can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. This large language model fine-tuning method based on network layer similarity can be implemented by a processor calling computer-readable instructions stored in memory. Alternatively, this large language model fine-tuning method based on network layer similarity can be executed by a server. Figure 1 As shown, this method for fine-tuning a large language model based on network layer similarity includes:

[0032] In step S11, the network layer similarity between any two adjacent network layers in the initial large language model is determined based on the sample dataset corresponding to the target natural language processing task.

[0033] The specific content of the target natural language processing task here can be flexibly set according to the actual use scenario. It can include customized tasks corresponding to natural language processing tasks such as text classification, question answering, machine translation and text generation in different use scenarios. For example, it can include chatbots for specific vertical fields (finance, medical fields, etc.). This disclosure does not make specific limitations on this.

[0034] The specific content of the sample dataset corresponding to the target natural language processing task can be flexibly set according to the specific content of the target natural language processing task, and this disclosure does not impose specific limitations on it.

[0035] In one example, the target natural language processing task is a text classification task in the financial field. The corresponding sample dataset may include multiple text data related to the financial field, as well as the corresponding classification label in the financial field for each text data.

[0036] For specific methods of obtaining sample datasets, refer to the implementation methods in related technologies. For example, sample datasets can be constructed based on downstream task pre-data provided by the target user, or sample datasets can be constructed using open-source data corresponding to other natural language processing tasks similar to the target natural language processing task. This disclosure does not make specific limitations in this regard.

[0037] The initial large language model can represent a large language model that has been pre-trained without any specific adjustments. Its specific form can be found in related technical implementations, and this disclosure does not impose any specific limitations on it. Typically, the initial large language model is deployed on a cloud server, and users can interact with it through a client connected to the cloud server.

[0038] After inputting sample data from the sample dataset into the initial large language model, the network layer similarity between any two adjacent network layers in the initial large language model can be calculated. The network layer similarity between any two adjacent network layers can quantitatively reflect the degree of functional similarity between these two network layers for the target natural language processing task, facilitating the analysis of differences in the internal knowledge distribution of the initial large language model when processing tasks in different domains. The specific method for determining network layer similarity can be flexibly set according to actual usage requirements; this disclosure does not impose specific limitations on it.

[0039] The following sections will describe in detail the process of determining the network layer similarity between any two adjacent network layers in the initial large language model based on the sample dataset corresponding to the target natural language processing task, in conjunction with the possible implementation methods disclosed herein. These details will not be elaborated upon here.

[0040] In step S12, a sub-model corresponding to the initial large language model is constructed based on the network layer similarity between any two adjacent network layers in the initial large language model.

[0041] The sub-model here can refer to a model constructed using at least one network layer in the initial large language model, which can be used for back-transfer of model parameters of the initial large language model; its specific form and the number of network layers it includes depend on the specific form and the number of network layers of the initial large language model, and this disclosure does not make specific limitations on them.

[0042] By leveraging the similarity between any two adjacent network layers in the initial large language model, the functional similarity of different network layers can be analyzed. This allows for the rational selection of network layers for constructing sub-models based on their functionality. This fully utilizes the differences in knowledge distribution within the initial large language model when handling different domain tasks, as well as the modularity of the initial large language model, to construct sub-models capable of specifically transferring and transmitting knowledge relevant to the target natural language processing task. Furthermore, the sub-models retain sufficient model information from the initial large language model, maintaining their efficiency and improving their representativeness. The specific method for constructing sub-models using network layers from the initial large language model can be flexibly configured according to actual usage requirements; this disclosure does not impose specific limitations on this.

[0043] The following sections will describe in detail the process of constructing a sub-model corresponding to the initial large language model based on the network layer similarity between any two adjacent network layers in the initial large language model, in conjunction with the possible implementation methods disclosed herein. These details will not be elaborated upon here.

[0044] In step S13, the sub-model is trained based on the sample dataset to determine the trained sub-model.

[0045] After the cloud server completes the construction of the sub-model, the user can obtain the sub-model from the cloud server through the user client and perform local offline training on the user client using the sample dataset to determine the trained sub-model. This avoids the external transmission of the sample dataset, which may involve user privacy, thereby improving the protection of user privacy. The specific method for training the sub-model can be found in the implementation methods in related technologies, and this disclosure does not impose specific limitations on it.

[0046] In one possible implementation, the user can pre-select pre-selected sample data from the sample dataset based on the training time resources available on the user's end, and define the target number of training steps required to train the sub-model. Based on the pre-selected sample data, the sub-model can be iteratively trained until the number of iterations meets the target number of training steps, resulting in the trained sub-model. Here, training time resources represent the time the user can provide for training the sub-model; its specific value can be flexibly set according to actual usage needs, depending on the user's own time allocation and the data processing capabilities of the user's end, etc. This disclosure does not impose specific limitations on this.

[0047] In step S14, the initial large language model is adjusted according to the model parameters corresponding to the trained sub-models to determine the target large language model, wherein the target large language model has the ability to process the target natural language processing task.

[0048] The sub-model trained through the above process initially possesses the ability to process the target natural language processing task. Based on this, the model parameters corresponding to the trained sub-model can be fine-tuned to the model parameters corresponding to the initial large language model. This enables the reverse transfer and transfer of knowledge in the domain corresponding to the target natural language processing task, allowing the adjusted target large language model to process the target natural language processing task and improving the efficiency of model fine-tuning. Furthermore, it avoids users from accessing all model parameters of the initial large language model, achieving data protection for the initial large language model and improving the security of the model fine-tuning process.

[0049] Furthermore, during the construction of sub-models, the model size of the sub-models can be reasonably controlled based on the training resources that users can provide, such as training time resources and user-end hardware performance resources. The smaller sub-models can be used to fine-tune the initial large language model, thereby ensuring the efficiency of model fine-tuning under resource constraints.

[0050] The following sections will describe in detail the process of adjusting the initial large language model based on the model parameters corresponding to the trained sub-models to determine the target large language model, taking into account the possible implementation methods disclosed herein. These details will not be elaborated upon here.

[0051] In this embodiment, based on the sample dataset corresponding to the target natural language processing task, the network layer similarity between any two adjacent network layers in the initial large language model is determined. This allows for the identification of structurally and functionally similar network layers within the initial large language model based on this similarity. By removing redundant network layers with high similarity, and fully utilizing the emerging modular structure within the initial large language model, a sub-model corresponding to the initial large language model is constructed. Based on the sample dataset, the sub-model can be trained to determine the trained sub-model. Then, based on the model parameters corresponding to the trained sub-model, the initial large language model can be adjusted in a targeted and efficient manner. The knowledge related to the target natural language processing task acquired from the trained sub-model is integrated into the initial large language model, resulting in a target large language model capable of handling the target natural language processing task. This embodiment ensures that the model parameters corresponding to the initial large language model are not fully disclosed and avoids the external transmission of sample datasets that may involve user privacy. This improves both the efficiency and security of model fine-tuning, enhancing both security and privacy protection.

[0052] In one possible implementation, the network layer similarity between any two adjacent network layers in the initial large language model is determined based on the sample dataset corresponding to the target natural language processing task. This includes: inputting any sample data from the sample dataset into the initial large language model; for any network layer of the initial large language model, determining the output feature corresponding to that network layer based on the output data of that network layer; and determining the network layer similarity between any two adjacent network layers based on the output features corresponding to each network layer.

[0053] By inputting any sample data from the sample dataset into the initial large oracle model, the output data corresponding to each network layer in the initial large language model can be obtained. For any given network layer, feature analysis of its output data can determine the output features of that network layer. Based on the output features of any two adjacent network layers, the network layer similarity between these two adjacent network layers can be determined.

[0054] The output features of any network layer can capture the semantic information and syntactic structure of the output data of that network layer, and can then be used to judge the processing performance of that network layer for the target natural language processing task. The specific form and content of the output features of any network layer can be flexibly set according to actual usage requirements, and this disclosure does not impose any specific limitations on them.

[0055] The specific method for calculating the network layer similarity between any two adjacent network layers based on the output features corresponding to each network layer can be flexibly set according to actual usage requirements, depending on the specific form of the network layer's output features. This disclosure does not impose any specific limitations on this.

[0056] In one possible implementation, the output features corresponding to any network layer include the sentence vector corresponding to that network layer. After inputting any sample data from the sample dataset into the initial large language model, for any network layer of the initial large language model, the output features corresponding to that network layer are determined based on the output data of that network layer, including: for any network layer of the initial large language model, determining the hidden layer output state and position weight corresponding to each position token in the output data of that network layer based on the length of the input sequence of the sample data input to the initial large language model; and for any network layer of the initial large language model, determining the sentence vector corresponding to that network layer based on the hidden layer output state and position weight corresponding to each position token in the output data of that network layer.

[0057] For any network layer in the initial large language model, the length of the sequence corresponding to its output data is the same as the length of the input sequence corresponding to the sample data input to the initial large language model. Based on the length of the input sequence, the hidden layer output state and position weight corresponding to each position token in the output data of the network layer can be determined. Then, through the hidden layer output state and position weight corresponding to each position token, the sentence vector corresponding to the network layer can be determined, thereby mapping the output data corresponding to the network layer to the vector space, which can quantitatively reflect the semantic information and syntactic structure of the output data corresponding to the network layer. Among them, the sentence vector corresponding to the i-th network layer of the initial large language model can be expressed as formula (1):

[0058]

[0059] Among them, R i The vector represents the sentence vector corresponding to the i-th layer of the initial large language model; L represents the length of the input sequence corresponding to the initial large language model; h k This represents the hidden layer output state corresponding to the k-th position token in the output data of the i-th network layer; w k This represents the position weight corresponding to the k-th position token in the output data of the i-th network layer.

[0060] w k This can be expressed as formula (2):

[0061]

[0062] In one possible implementation, the network layer similarity between any two adjacent network layers is determined based on the output features corresponding to each network layer. This includes: using the central kernel analysis method to determine the network layer similarity between any two adjacent network layers based on the sentence vector corresponding to each network layer.

[0063] Using the Centered Kernel Analysis (CKA) method, the network layer similarity between any two adjacent network layers can be calculated based on the sentence vectors corresponding to those two adjacent network layers. Specifically, the network layer similarity between the i-th network layer and its adjacent j-th network layer can be expressed as formula (3):

[0064]

[0065] Among them, CKA(s) i ,s j ) represents the network layer similarity between the i-th network layer and its adjacent j-th network layer; s i s represents the sentence vector corresponding to the i-th network layer; jRepresents the sentence vector corresponding to the j-th network layer; T represents the transpose operation; ||·|| F This represents the Frobenius norm.

[0066] Through the above process, the CKA method can be used to analyze whether any two adjacent network layers have similar functions when processing the target natural language processing task by using the sentence vectors corresponding to each network layer. This determines the network layer similarity between the two adjacent network layers, so as to selectively screen network layers in the initial large language model and construct sub-models in a reasonable manner.

[0067] In one possible implementation, a sub-model corresponding to the initial large language model is constructed based on the network layer similarity between any two adjacent network layers in the initial large language model. This includes: using a hierarchical clustering method, iteratively clustering all network layers of the initial large language model based on the network layer similarity between any two adjacent network layers to determine multiple cluster sets and a clustering index corresponding to each cluster set, until the number of cluster sets equals the preset number of network layers that can be stored. Each cluster set includes multiple adjacent network layers, and the clustering index corresponding to any cluster set includes the number of network layers included in that cluster set and the position of each network layer in that cluster set within the initial large language model; for any given cluster set, determining the representative network layer corresponding to that cluster set among the multiple network layers included in that cluster set based on the clustering index; and determining the sub-model based on the representative network layer corresponding to each cluster set.

[0068] After obtaining the network layer similarity between all adjacent network layers in the initial large language model, a hierarchical clustering method can be used to traverse and cluster from the first network layer, and determine the two adjacent network layers with the highest network layer similarity as a cluster set. After completing the first traversal clustering, the above traversal clustering process is repeated iteratively to determine multiple cluster sets until the number of cluster sets is equal to the preset number of network layers that can be saved.

[0069] In one example, the initial large language model comprises 23 network layers. Clustering is performed starting from the first layer, calculating the similarity between any two adjacent layers and sorting these similarities in descending order. Based on this sorting, the similarity between the 22nd and 23rd layers can be determined, and the first cluster set can be established using these two layers. The sorting can then be repeated to determine the second cluster set, or new layers can be added to the first cluster set. Specifically, if the similarity between layers 2 and 3 is the highest (excluding the similarity between layers 22 and 23), then a second cluster set can be determined based on these two layers. Similarly, if the similarity between layers 21 and 22 is the highest (excluding the similarity between layers 22 and 23), then layer 21 can be added to the first cluster set. This process is repeated until the number of cluster sets equals the preset number of storable layers, thus determining multiple cluster sets. Each cluster set includes at least two adjacent layers.

[0070] The preset number of network layers that can be saved here can be used to indicate the number of network layers that a sub-model needs to include, thereby controlling the model size and performance of the sub-model. The specific value of the preset number of network layers that can be saved can be flexibly set according to actual usage needs, and this disclosure does not impose specific limitations on it.

[0071] For any given cluster set, the cluster index can be determined based on the number of network layers included in the cluster set and the position of each network layer in the initial large language model. This index can then serve as the basis for subsequent sub-model construction and parameter fine-tuning.

[0072] By using the clustering index corresponding to each cluster set, the representative network layer corresponding to each cluster set can be determined. The specific method for determining the representative network layer corresponding to any given cluster set can be flexibly set according to actual usage requirements, and this disclosure does not impose specific limitations on it.

[0073] In one example, for any given cluster set, the order of the network layers within that cluster set can be determined based on its cluster index, and the network layer in the middle position can be identified as the representative network layer for that cluster set. For instance, based on the cluster index of cluster set A, it can be determined that cluster set A contains 5 network layers, and according to the position of each network layer in the initial large language model, they can be identified as network layers A1, A2, A3, A4, and A5, respectively; and based on the cluster index of cluster set A, network layer A3, located in the middle position, can be identified as the representative network layer for cluster set A.

[0074] Based on the cluster index corresponding to each cluster set, the representative network layers corresponding to each cluster set can be sequentially connected according to their positions in the initial large model, thereby constructing a sub-model whose model size meets the preset number of network layers that can be stored.

[0075] Through the above process, the modular structure emerging within the initial large language model can be utilized. Based on the similarity between any two network layers, hierarchical clustering operations can be used to dynamically select the network layer for the target natural language processing task. In this way, while maintaining the integrity and efficiency of the model performance, a sub-model capable of specifically handling the target natural language processing task can be constructed.

[0076] In one possible implementation, the sub-model is determined based on the representative network layer corresponding to each cluster set, including: for any cluster set, copying the representative network layer corresponding to the cluster set according to the cluster index, and determining multiple copied representative network layers corresponding to the cluster set, wherein the number of copied representative network layers is equal to the number of network layers included in the cluster set; and connecting the multiple representative network layers corresponding to each cluster set according to the cluster index to determine the sub-model.

[0077] Furthermore, to maximize the performance of the sub-model and improve the accuracy and reliability of parameter fine-tuning of the initial large language model, the model size of the sub-model can be increased by copying network layers, ensuring that the number of network layers in the sub-model is the same as that in the initial large language model. Specifically, for any given cluster set, the representative network layer corresponding to that cluster set can be copied based on the cluster index and the number of network layers included in that cluster set.

[0078] Taking the above cluster set A, with network layer A3 as the representative network layer, according to the cluster index of cluster set A, it can be determined that cluster set A includes 5 network layers. Then, network layer A3 can be copied 4 times to determine the 4 copied network layers A.

[0079] Based on the cluster index corresponding to each cluster set, multiple representative network layers corresponding to each cluster set can be connected sequentially to construct a sub-model with the same model size as the initial large language model.

[0080] Through the above process, the inter-layer parameter sharing mechanism of the network layers can be used to make the model size of the sub-model the same as that of the complete initial large language model. This makes the performance of the sub-model closer to that of the complete initial large language model and makes up for the gap between offline fine-tuning and inference on the complete initial large language model. This improves the efficiency of subsequent model fine-tuning and the performance of back-transferring model parameters during fine-tuning.

[0081] In one possible implementation, the initial large language model is adjusted based on the model parameters corresponding to the trained sub-models to determine the target large language model. This includes: using the clustering index corresponding to each cluster set to determine the parameter correspondence between the trained sub-models and the initial large language model; and replacing the model parameters corresponding to the initial large language model based on the parameter correspondence and the model parameters corresponding to the trained sub-models to determine the target large language model.

[0082] After training the sub-model on the sample dataset, the trained sub-model can be uploaded from the user's client to the cloud server where the initial large language model resides. This allows the model parameters of the initial large language model to be adjusted using the parameters of the trained sub-model. Specifically, by using each cluster and its corresponding cluster index, the network layer in the trained sub-model corresponds to the network layer in the initial large language model. This, in turn, determines the model parameters in the trained sub-model correspond to the model parameters in the initial large language model, thus establishing the parameter correspondence between the trained sub-model and the initial large language model.

[0083] By utilizing the parameter correspondence and the model parameters corresponding to the trained sub-models, the model parameters corresponding to the initial large language model can be replaced, thereby achieving fine-tuning of the initial large language model, determining the target large language model, and ensuring that the sample datasets used by users that may involve user privacy information do not leave the user's corresponding user terminal during the entire fine-tuning process, thus improving data security and privacy protection for large language model fine-tuning.

[0084] Figure 2This diagram illustrates a process of fine-tuning a large language model based on network layer similarity according to an embodiment of the present disclosure. Figure 2 As shown, after the cloud server prepares the initial large language model, the user can preset the number of networks that can be saved according to the user-end resources, so as to control the model scale of the sub-models built subsequently.

[0085] If the user can directly provide a sample dataset corresponding to the target natural language processing task, at least one sample data can be obtained from the sample dataset and input into the initial large language model to determine the network layer similarity between any two adjacent network layers in the initial large language model.

[0086] When the user cannot directly provide the sample dataset corresponding to the target natural language processing task, at least one existing dataset corresponding to a similar task of the target natural language processing task can be obtained to determine the network layer similarity between any two adjacent network layers in the initial large language model.

[0087] Based on the similarity between any two adjacent network layers in the initial large language model, and combined with the preset number of network layers that can be saved, hierarchical clustering can be performed to construct sub-models corresponding to the initial large language model, and the sub-models can be distributed to the user terminal.

[0088] Depending on the available training time on the user's end, the option to copy and share network layers of the sub-model can be flexibly chosen to adjust its size. After users define pre-selected sample data and the target number of training steps, they can iteratively train the sub-model on the user's end. This ensures that not all model parameters corresponding to the initial large language model are leaked, and avoids the external transmission of sample datasets that may involve user privacy. This improves both the efficiency and security of model fine-tuning, enhancing both its security and privacy protection.

[0089] After the number of iterations meets the target training steps, the trained sub-models can be sent back to the cloud server. Based on the trained sub-models and the clustering index corresponding to each cluster set, the initial large language model is adjusted to obtain the target large language model capable of handling the target natural language processing task.

[0090] In this embodiment, based on the sample dataset corresponding to the target natural language processing task, the network layer similarity between any two adjacent network layers in the initial large language model is determined. This allows for the identification of structurally and functionally similar network layers within the initial large language model based on this similarity. By removing redundant network layers with high similarity, and fully utilizing the emerging modular structure within the initial large language model, a sub-model corresponding to the initial large language model is constructed. Based on the sample dataset, the sub-model can be trained to determine the trained sub-model. Then, based on the model parameters corresponding to the trained sub-model, the initial large language model can be adjusted efficiently and in a targeted manner. The knowledge related to the target natural language processing task acquired from the trained sub-model is integrated into the initial large language model, thus determining a target large language model capable of handling the target natural language processing task. Through this embodiment, on the one hand, it ensures that the model parameters corresponding to the initial large language model are not completely leaked; on the other hand, it avoids the external transmission of sample datasets that may involve user privacy. This improves the efficiency of model fine-tuning while comprehensively enhancing the security and privacy protection capabilities of model fine-tuning.

[0091] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0092] In addition, this disclosure also provides a large language model fine-tuning device, electronic device and computer-readable storage medium based on network layer similarity. All of the above can be used to implement any of the large language model fine-tuning methods based on network layer similarity provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding records in the method section, and will not be repeated here.

[0093] Figure 3 A block diagram of a large language model fine-tuning apparatus based on network layer similarity according to an embodiment of the present disclosure is shown. Figure 3 As shown, the device 300 includes:

[0094] The similarity analysis module 301 is used to determine the network layer similarity between any two adjacent network layers in the initial large language model based on the sample dataset corresponding to the target natural language processing task.

[0095] The sub-model construction module 302 is used to construct the sub-model corresponding to the initial large language model based on the network layer similarity between any two adjacent network layers in the initial large language model.

[0096] The sub-model training module 303 is used to train the sub-model based on the sample dataset and determine the trained sub-model.

[0097] The model parameter adjustment module 304 is used to adjust the initial large language model according to the model parameters corresponding to the trained sub-models to determine the target large language model, wherein the target large language model has the ability to process the target natural language processing task.

[0098] In one possible implementation, the similarity analysis module 301 is specifically used to: input any sample data included in the sample dataset into the initial large language model, and for any network layer of the initial large language model, determine the output feature corresponding to the network layer based on the output data of the network layer; and determine the network layer similarity between any two adjacent network layers based on the output feature corresponding to each network layer.

[0099] In one possible implementation, the output features corresponding to any network layer include the sentence vector corresponding to that network layer; the similarity analysis module 301 is further configured to: for any network layer of the initial large language model, determine the hidden layer output state and position weight corresponding to each position token in the output data of that network layer based on the length of the input sequence of the sample data input to the initial large language model; and for any network layer of the initial large language model, determine the sentence vector corresponding to that network layer based on the hidden layer output state and position weight corresponding to each position token in the output data of that network layer.

[0100] In one possible implementation, the similarity analysis module 301 is further used to: determine the network layer similarity between any two adjacent network layers based on the sentence vector corresponding to each network layer using the central kernel analysis method.

[0101] In one possible implementation, the sub-model construction module 302 is specifically used to: utilize a hierarchical clustering method to iteratively cluster all network layers of the initial large language model based on the network layer similarity between any two adjacent network layers, determine multiple cluster sets, and the clustering index corresponding to each cluster set, until the number of cluster sets equals the preset number of network layers that can be stored. Each cluster set includes multiple adjacent network layers, and the clustering index corresponding to any cluster set includes the number of network layers included in that cluster set and the position of each network layer in that cluster set within the initial large language model; for any given cluster set, based on the clustering index corresponding to that cluster set, determine the representative network layer corresponding to that cluster set among the multiple network layers included in that cluster set; and determine the sub-model based on the representative network layer corresponding to each cluster set.

[0102] In one possible implementation, the sub-model building module 302 is further configured to: for any cluster set, copy the representative network layer corresponding to the cluster set according to the cluster index corresponding to the cluster set, and determine the multiple copied representative network layers corresponding to the cluster set, wherein the number of copied representative network layers is equal to the number of network layers included in the cluster set; and connect the multiple representative network layers corresponding to each cluster set according to the cluster index corresponding to each cluster set to determine the sub-model.

[0103] In one possible implementation, the model parameter adjustment module 304 is further configured to: determine the parameter correspondence between the trained sub-model and the initial large language model using the cluster index corresponding to each cluster set; and replace the model parameters corresponding to the initial large language model according to the parameter correspondence and the model parameters corresponding to the trained sub-model to determine the target large language model.

[0104] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0105] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium can be volatile or non-volatile.

[0106] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0107] Figure 4 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. For example, device 1900 may be provided as a server or terminal device. (Refer to...) Figure 4 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0108] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output interface 1958 (I / O interface). Device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM macOS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0109] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of the device 1900 to perform the above-described method.

[0110] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0111] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0112] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0113] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0114] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0115] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0116] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0118] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for fine-tuning a large language model based on network layer similarity, characterized in that, include: Based on the sample dataset corresponding to the target natural language processing task, determine the network layer similarity between any two adjacent network layers in the initial large language model; Based on the network layer similarity between any two adjacent network layers in the initial large language model, construct the sub-model corresponding to the initial large language model; The sub-model is trained based on the sample dataset to determine the trained sub-model; Based on the model parameters corresponding to the trained sub-models, the initial large language model is adjusted to determine the target large language model, wherein the target large language model has the ability to process the target natural language processing task; The step of constructing a sub-model corresponding to the initial large language model based on the network layer similarity between any two adjacent network layers in the initial large language model includes: Using a hierarchical clustering method, based on the similarity between any two adjacent network layers, all network layers of the initial large language model are iteratively clustered to determine multiple cluster sets and the clustering index corresponding to each cluster set, until the number of cluster sets is equal to the preset number of network layers that can be stored. Here, any cluster set includes multiple adjacent network layers, and the clustering index corresponding to any cluster set includes the number of network layers included in the cluster set and the position of each network layer in the cluster set in the initial large language model. For any given cluster set, based on the cluster index corresponding to the cluster set, determine the representative network layer corresponding to the cluster set among the multiple network layers included in the cluster set; The sub-model is determined based on the representative network layer corresponding to each cluster set.

2. The method according to claim 1, characterized in that, The step of determining the network layer similarity between any two adjacent network layers in the initial large language model based on the sample dataset corresponding to the target natural language processing task includes: After inputting any sample data from the sample dataset into the initial large language model, for any network layer of the initial large language model, the output feature corresponding to that network layer is determined based on the output data of that network layer. Based on the output features of each network layer, determine the network layer similarity between any two adjacent network layers.

3. The method according to claim 2, characterized in that, The output features of any network layer include the sentence vector corresponding to that network layer; After inputting any sample data from the sample dataset into the initial large language model, for any network layer of the initial large language model, determining the output features corresponding to that network layer based on the output data of that network layer includes: For any network layer of the initial large language model, the hidden layer output state and position weight corresponding to each position token in the output data of that network layer are determined based on the length of the input sequence of the sample data input to the initial large language model. For any network layer of the initial large language model, the sentence vector corresponding to the network layer is determined based on the hidden layer output state and position weight corresponding to each position token in the output data of the network layer.

4. The method according to claim 3, characterized in that, The step of determining the network layer similarity between any two adjacent network layers based on the output features corresponding to each network layer includes: Using the central kernel analysis method, the network layer similarity between any two adjacent network layers is determined based on the sentence vector corresponding to each network layer.

5. The method according to claim 1, characterized in that, The step of determining the sub-model based on the representative network layer corresponding to each cluster set includes: For any cluster set, based on the cluster index corresponding to the cluster set, the representative network layer corresponding to the cluster set is copied to determine the multiple copied representative network layers corresponding to the cluster set. The number of copied representative network layers is equal to the number of network layers included in the cluster set. Based on the clustering index corresponding to each cluster set, multiple representative network layers corresponding to each cluster set are connected to determine the sub-model.

6. The method according to claim 1, characterized in that, The step of adjusting the initial large language model based on the model parameters corresponding to the trained sub-model to determine the target large language model includes: Using the clustering index corresponding to each cluster set, the parameter correspondence between the trained sub-model and the initial large language model is determined; Based on the parameter correspondence and the model parameters corresponding to the trained sub-model, the model parameters corresponding to the initial large language model are replaced to determine the target large language model.

7. A fine-tuning device for a large language model based on network layer similarity, characterized in that, include: The similarity analysis module is used to determine the network layer similarity between any two adjacent network layers in the initial large language model based on the sample dataset corresponding to the target natural language processing task. The sub-model construction module is used to construct a sub-model corresponding to the initial large language model based on the network layer similarity between any two adjacent network layers in the initial large language model. The sub-model training module is used to train the sub-model based on the sample dataset and determine the trained sub-model. The model parameter adjustment module is used to adjust the initial large language model according to the model parameters corresponding to the trained sub-models to determine the target large language model, wherein the target large language model has the ability to process the target natural language processing task; Specifically, the sub-model construction module is used for: Using a hierarchical clustering method, based on the similarity between any two adjacent network layers, all network layers of the initial large language model are iteratively clustered to determine multiple cluster sets and the clustering index corresponding to each cluster set, until the number of cluster sets is equal to the preset number of network layers that can be stored. Here, any cluster set includes multiple adjacent network layers, and the clustering index corresponding to any cluster set includes the number of network layers included in the cluster set and the position of each network layer in the cluster set in the initial large language model. For any given cluster set, based on the cluster index corresponding to the cluster set, determine the representative network layer corresponding to the cluster set among the multiple network layers included in the cluster set; The sub-model is determined based on the representative network layer corresponding to each cluster set.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 6 when executing instructions stored in the memory.

9. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images