A method and system for command data mining based on LoRA embedding

Abstract

A method and system for command data mining based on LoRA embedding, relating to the field of command data mining. It solves the common problems that command data generated by models are often highly repetitive, similar command data appear repeatedly, and data set homogeneity and redundancy are common. The method comprises: training the ShareGPT data set in a LoRA manner to obtain an embedded representation of the command data in the ShareGPT data set; reducing the dimension of the embedded representation by the UMAP algorithm to obtain a representation of each command in a three-dimensional space; screening each command representation according to minimizing the density of the command data in the three-dimensional space to obtain a data subset with enhanced diversity. The present invention is applied to the field of artificial neural network training.

Description

Technical Field

[0001] The present invention relates to the field of instruction data mining, and in particular to an instruction data mining method based on LoRA embedding. Background Art

[0002] Artificial neural network training technology has been developed for decades. In the process of exploring model training, people have come up with a rule: as the model size, data set size, and the amount of computing used for training increase, the performance of the model will improve. However, there are also studies that show that improving data quality will greatly change the trend of this rule, allowing small models to achieve the effect of large models.

[0003] In the era of large models, instruction data is crucial for training and fine-tuning models, and constructing high-quality instruction data sets faces various challenges, one of the main challenges being to ensure the diversity and non-repetition of instructions. Diversity means that instruction data should cover a wide range of fields and scenarios, and its style, complexity and other characteristics need to be varied. Diversified instruction data can allow models to access knowledge close to the scope of large data sets in smaller data sets, reduce the risk of overfitting or forming fixed memory patterns, and increase the model's ability to generalize to unseen instructions.

[0004] However, achieving diversity is not easy. Due to the constraints of prior knowledge and training data, the instruction data currently generated by the model is often highly repetitive, similar instruction data appears repeatedly, and data sets are homogeneous and redundant. Therefore, it is necessary to find a suitable method to enhance the diversity of data sets and reduce redundancy from the aspects of generation or screening. Summary of the invention

[0005] Aiming at the common problems that instruction data generated by models are often highly repetitive, similar instruction data appear repeatedly, and data set homogeneity and redundancy are common, the present invention proposes an instruction data mining method based on LoRA embedding, the method comprising:

[0006] The ShareGPT dataset is trained using LoRA to obtain an embedded representation of the instruction data in the ShareGPT dataset;

[0007] Performing dimensionality reduction on the embedded representation by using a UMAP algorithm to obtain a representation of each instruction in a three-dimensional space;

[0008] Each instruction representation is screened according to minimizing the density of instruction data in three-dimensional space to obtain a data subset with enhanced diversity.

[0009] Furthermore, a preferred method is provided, wherein the ShareGPT dataset is trained using LoRA, including:

[0010] Fix all parameters of the Llama model;

[0011] The rank decomposition matrix is ​​injected into the embedding layer of the Llama model input using the LoRA method;

[0012] Connect the model with the LoRA-injected rank factorization matrix to the training dataset and fine-tune the embedding layer of the model;

[0013] The model injected with the LoRA matrix is ​​trained on the ShareGPT dataset to obtain the embedded representation of the instruction data in the ShareGPT dataset.

[0014] Furthermore, a preferred method is provided, in which the model injected with the LoRA matrix is ​​used to train the ShareGPT dataset, specifically: the LoRA dimensionality reduction matrix is ​​used to train the ShareGPT dataset.

[0015] Furthermore, a preferred method is provided, wherein the embedding representation is reduced in dimension by using a UMAP algorithm to obtain a representation of each instruction in a three-dimensional space, including:

[0016] Calculate the manifold structure characteristics of high-dimensional space and obtain the positional relationship of each data point;

[0017] Constructing a data distribution structure in a high-dimensional space according to the positional relationship between the data points;

[0018] The data distribution structure of the high-dimensional space is projected into the low-dimensional space to obtain the representation of each instruction in the three-dimensional space.

[0019] Furthermore, a preferred method is provided, wherein the calculation of the manifold structure characteristics of the high-dimensional space to obtain the positional relationship of each data point includes: taking the Euclidean distance to calculate the distance between two data points in the high-dimensional space.

[0020] Furthermore, a preferred method is provided, wherein the projecting of the data distribution structure of the high-dimensional space into the low-dimensional space comprises: minimizing the cross entropy of the data distribution of the high-dimensional space and the low-dimensional space using a cross entropy function.

[0021] Furthermore, a preferred method is provided, wherein each instruction representation is screened according to minimizing the density of instruction data in three-dimensional space, including: clustering instruction data based on Euclidean distance using KMeans method.

[0022] Based on the same inventive concept, the present invention also proposes a LoRA-embedded instruction data mining system, the system comprising:

[0023] A training unit, used to train the ShareGPT dataset in a LoRA manner to obtain an embedded representation of instruction data in the ShareGPT dataset;

[0024] A dimensionality reduction unit, used to reduce the dimension of the embedded representation by using a UMAP algorithm to obtain a representation of each instruction in a three-dimensional space;

[0025] The screening unit is used to screen each instruction representation according to the density of the minimized instruction data in the three-dimensional space to obtain an enhanced data subset.

[0026] Based on the same inventive concept, the present invention also proposes a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes a LoRA-embedded instruction data mining method according to any one of the above.

[0027] Based on the same inventive concept, the present invention also proposes a computer-readable storage medium, which is used to store a computer program, and the computer program executes any one of the above-mentioned LoRA-embedded instruction data mining methods.

[0028] The present invention is beneficial in that:

[0029] The present invention solves the common problems that instruction data generated by models are often highly repetitive, similar instruction data appear repeatedly, and data set homogeneity and redundancy are common.

[0030] The present invention discloses a method for mining instruction data based on LoRA embedding, which reduces the dimension of LoRA embedding through the UMAP algorithm, and maps high-dimensional instruction data to three-dimensional space, thereby reducing the dimension of the data. This not only helps to visualize the data, but also reduces the computational burden of subsequent processing. UMAP and LoRA are both dimensionality reduction methods that maintain data structure. UMAP emphasizes maintaining the local and global structure of data while reducing the dimension, while LoRA focuses on maintaining the local structure. Combining these two methods can better preserve the intrinsic relationship of instruction data. By minimizing the density of instruction data in three-dimensional space, the method screens out instruction data that still maintains the key structure of the data after dimensionality reduction. Such screening can reduce redundant data, improve the effectiveness of data, and obtain more diverse data sets.

[0031] The present invention discloses a method for mining instruction data based on LoRA embedding, which explores the instruction data features contained in the representation according to the different matrix representations of instruction data in the model. The word embedding matrix after the LoRA dimensionality reduction operation obtained after fine-tuning the instruction data is used as the feature representation of the instruction data, and the dimensionality reduction clustering operation is performed on the representation through the UMAP algorithm and the KMeans algorithm, which confirms that the method can extract the features of the instruction data and perform effective classification. This work intuitively represents the instruction data in three-dimensional space, extracts the instruction data through the minimum density principle, obtains a subset of instruction data with strong diversity, and verifies through experiments that fine-tuning the model after the diversity is improved can bring performance improvement. The performance of the model is greatly improved with fewer computing resources.

[0032] The present invention is applied to the field of artificial neural network training. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flow chart of a command data mining method based on LoRA embedding according to implementation mode 1;

[0034] Figure 2 This is a schematic diagram of the LoRA fine-tuning method described in Implementation Example 11;

[0035] Figure 3 This is the instruction data clustering diagram described in Implementation Mode 11. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0037] Implementation method 1, see Figure 1 This embodiment describes a method for mining command data based on LoRA embedding, the method comprising:

[0038] The ShareGPT dataset is trained using LoRA to obtain an embedded representation of the instruction data in the ShareGPT dataset;

[0039] Performing dimensionality reduction on the embedded representation by using a UMAP algorithm to obtain a representation of each instruction in a three-dimensional space;

[0040] Each instruction representation is screened according to minimizing the density of instruction data in three-dimensional space to obtain a data subset with enhanced diversity.

[0041] This implementation method reduces the dimension of LoRA embedding through the UMAP algorithm, mapping the high-dimensional instruction data to three-dimensional space, thereby reducing the dimension of the data. This not only helps to visualize the data, but also reduces the computational burden of subsequent processing. UMAP and LoRA (Local Rademacher Average) are both dimensionality reduction methods that maintain data structure. UMAP emphasizes maintaining the local and global structure of the data while reducing the dimension, while LoRA focuses on maintaining the local structure. Combining these two methods can better preserve the intrinsic relationship of the instruction data. By minimizing the density of instruction data in three-dimensional space, this method screens out instruction data that still maintains the key structure of the data after dimensionality reduction. Such screening can reduce redundant data, improve the effectiveness of data, and obtain more diverse data sets.

[0042] The LoRA described in this embodiment is a dimensionality reduction method for processing high-dimensional data, which measures the complexity of data by local Rademacher complexity, and then maintains the local structure of data during dimensionality reduction. Through LoRA, the instruction data is embedded in a more compact and more clearly expressed space. UMAP is a nonlinear dimensionality reduction algorithm that tries to maintain the global data structure while maintaining the local relationship between data points. The high-dimensional data embedded in LoRA is mapped to a three-dimensional space through UMAP, so that the dimension of the data is greatly reduced. In three-dimensional space, by calculating the density around each instruction data point, it can be determined which instruction data points are in a relatively sparse area after dimensionality reduction. By minimizing these densities, the method can screen out relatively unique and representative instruction data to form an enhanced data subset. By screening out representative instruction data after dimensionality reduction, the method can reduce redundant information in the data set and improve the compactness and effectiveness of the data. Removing highly repetitive, homogeneous and redundant instruction data, the remaining data is more representative, which helps to improve the quality of the data. The reduced-dimensional data set is easier to visualize and analyze, and is also more suitable for modeling and analysis in fields such as machine learning and deep learning.

[0043] Implementation method 2: This implementation method further limits the instruction data mining method based on LoRA embedding described in implementation method 1. The method of training the ShareGPT data set using LoRA includes:

[0044] Fix all parameters of the Llama model;

[0045] The rank decomposition matrix is ​​injected into the embedding layer of the Llama model input using the LoRA method;

[0046] Connect the model with the LoRA-injected rank factorization matrix to the training dataset and fine-tune the embedding layer of the model;

[0047] The ShareGPT dataset is trained based on the fine-tuned model to obtain the embedded representation of the instruction data in the ShareGPT dataset.

[0048] This implementation improves on an existing model (Llama model) without rebuilding the entire model. This reduces training cost and time, especially if the Llama model has already performed well in other fields or tasks. By injecting LoRA into the embedding layer of the model, the characteristics of the input data are better preserved. The expressiveness and performance of the model are further improved. By fine-tuning the entire model, the performance of the model can be further optimized according to the needs of specific tasks and data sets, which helps make the model more suitable for instruction data mining tasks.

[0049] In this embodiment, the parameters of the Llama model are fixed, that is, the weights and structure of the Llama model are no longer trained. This is because the Llama model has been trained in other fields or tasks, and it can be considered that it already has a certain generalization ability. The LoRA method is used to inject the rank decomposition matrix into the embedding layer of the Llama model. This injection method can maintain the local structure of the data through the properties of LoRA while improving the compactness of the data representation. The entire model (including the LoRA-injected part) is connected to the training data set and then fine-tuned. This fine-tuning stage is to adjust the model to adapt to specific instruction data mining tasks. By injecting and fine-tuning LoRA on an existing model, the purpose of this method is to make the model more suitable for specific instruction data mining tasks and improve the performance of the model. Avoiding building a new model from scratch and modifying it based on the existing model can speed up the training process and reduce costs. The LoRA injection method and the fine-tuning process are designed to improve the embedding representation to better capture the characteristics and structure of instruction data, which is expected to improve the quality and efficiency of data mining.

[0050] Implementation method three: This implementation method is a further limitation of the instruction data mining method based on LoRA embedding described in implementation method two. The model injected with the LoRA matrix is ​​trained on the ShareGPT dataset. Specifically, the LoRA dimensionality reduction matrix is ​​trained on the ShareGPT dataset.

[0051] In this implementation, by using the LoRA dimension reduction matrix to train the ShareGPT dataset, the model can better adapt to the characteristics and semantics of this specific dataset. This means that the model will understand and process the instruction data in the ShareGPT dataset more effectively. By training the model more, especially on task-specific data, it is expected that the performance of the model on the ShareGPT dataset will be improved, making it more suitable for instruction data mining tasks.

[0052] The LoRA dimensionality reduction matrix described in this embodiment is a technique for reducing the dimensionality of the model input or embedding layer. It can be used to reduce the dimension of the embedding layer to reduce the computational complexity or increase the generalization performance of the model. This step builds on the previous steps by fine-tuning the model embedding layer, especially on the ShareGPT dataset. During the fine-tuning process, the model parameters can be further adjusted to improve its performance. The main purpose is to optimize the performance of the model by performing deep training on a specific dataset (ShareGPT dataset). This includes training the LoRA dimensionality reduction matrix to better adapt to the characteristics of the dataset. By optimizing the model during the training process of the LoRA dimensionality reduction matrix, it is expected to improve the generalization performance of the model on similar tasks and data. This enables the model to better cope with various instruction data.

[0053] Implementation mode 4: This implementation mode is a further limitation of the instruction data mining method based on LoRA embedding described in Implementation mode 1. The embedded representation is reduced in dimension by the UMAP algorithm to obtain the representation of each instruction in three-dimensional space, including:

[0054] Calculate the manifold structure characteristics of high-dimensional space and obtain the positional relationship of each data point;

[0055] Constructing a data distribution structure in a high-dimensional space according to the positional relationship between the data points;

[0056] The data distribution structure of the high-dimensional space is projected into the low-dimensional space to obtain the representation of each instruction in the three-dimensional space.

[0057] This implementation method captures the complex structure of high-dimensional data by calculating the manifold structure characteristics of the high-dimensional space. High-dimensional data usually contains a potential manifold structure, that is, the nonlinear distribution of data in the high-dimensional space. The role of UMAP here is to calculate the manifold structure characteristics between data points, which includes the proximity relationship and distance of points, so as to reveal the essential structure of data in the high-dimensional space. Further obtaining the positional relationship of each numerical control point helps to understand the relative positional relationship between different points in the high-dimensional data. It helps to identify which points are closer in the high-dimensional space and which points are related, which is very critical for subsequent data dimensionality reduction. Based on the positional relationship of each point, a data distribution structure in a high-dimensional space can be constructed. This structure describes the connection and organization between data points in order to better understand the intrinsic characteristics of high-dimensional data. Ultimately, UMAP will project the data into a low-dimensional space, usually a three-dimensional space, based on the data distribution structure in the high-dimensional space. This low-dimensional representation is easier to visualize and analyze because it reduces the complexity of the data while retaining the essential structure of the data. This helps to mine and understand the potential patterns and characteristics in the data.

[0058] This implementation reduces the dimensionality of high-dimensional data in order to better understand and analyze the data while maintaining the key features of the data. Based on the manifold structure characteristics in high-dimensional space, UMAP can maintain the proximity relationship between data points while reducing the dimensionality, so that the low-dimensional representation better reflects the similarity of the data points. By projecting into three-dimensional space, the data becomes easier to visualize, so that the distribution and characteristics of the data can be better understood.

[0059] Implementation method five. This implementation method is a further limitation of the instruction data mining method based on LoRA embedding described in implementation method four. The method calculates the manifold structure characteristics of the high-dimensional space and obtains the positional relationship of each data point, including: taking the Euclidean distance to calculate the distance between two data points in the high-dimensional space.

[0060] This implementation method calculates the Euclidean distance between data points in the high-dimensional space to understand the distribution and structural characteristics of data in the high-dimensional space. Through the Euclidean distance, the relative distance between different data points can be quantified, and then the positional relationship of the data points can be established, which is helpful for subsequent data analysis, clustering, classification and other tasks.

[0061] Implementation method six. This implementation method is a further limitation of the instruction data mining method based on LoRA embedding described in implementation method four, and the data distribution structure of the high-dimensional space is projected to the low-dimensional space, including: using the cross entropy function to minimize the cross entropy of the data distribution of the high-dimensional space and the low-dimensional space.

[0062] By minimizing the cross entropy between the high-dimensional space and the low-dimensional space, the algorithm can maintain the similarity between data points. This helps to ensure that the low-dimensional representation after dimensionality reduction retains the structure and characteristics of the original data to a certain extent. Since cross entropy is used to compare the data distribution in high-dimensional and low-dimensional space, the results of the algorithm can be more easily interpreted. This makes the data after dimensionality reduction easier to visualize and understand.

[0063] The main purpose of using the cross entropy function in this implementation is to find a mapping method in the high-dimensional space to map the data points from the original high-dimensional space to a low-dimensional space while trying to maintain the similarity between the data points. Specifically, the purpose includes: reducing the dimensionality of high-dimensional data to a low-dimensional space to reduce the complexity of the data, thereby improving the computational efficiency and the feasibility of visualization. Ensure that the structure, clustering and characteristics of the original data can still be retained in the low-dimensional space for more in-depth analysis and understanding.

[0064] Implementation method seven. This implementation method is a further limitation of the instruction data mining method based on LoRA embedding described in implementation method one, and each instruction representation is screened according to minimizing the density of instruction data in three-dimensional space, including: clustering instruction data based on Euclidean distance using the KMeans method.

[0065] This implementation uses the K-Means method to cluster instruction data based on Euclidean distance, and groups similar instruction data points into the same cluster so that instructions within the cluster are similar, while instructions between different clusters are more different. Clustering helps reduce the dimensionality of high-dimensional instruction data, thereby reducing the difficulty of processing complex data. K-Means can help detect outliers because points far away from the centroid may be outliers. By dividing data into different clusters, analysts can more easily understand the inherent structure of the data and identify patterns and regularities in the data set.

[0066] Embodiment 8: A LoRA-embedded instruction data mining system described in this embodiment, the system includes:

[0067] A training unit, used to train the ShareGPT dataset in a LoRA manner to obtain an embedded representation of instruction data in the ShareGPT dataset;

[0068] A dimensionality reduction unit, used to reduce the dimension of the embedded representation by using a UMAP algorithm to obtain a representation of each instruction in a three-dimensional space;

[0069] The screening unit is used to screen each instruction representation according to the density of the minimized instruction data in the three-dimensional space to obtain a data subset with enhanced diversity.

[0070] Implementation method nine, a computer device described in this implementation method, the computer device includes a memory and a processor, the memory stores a computer program, when the processor runs the computer program stored in the memory, the processor executes a LoRA embedded instruction data mining method according to any one of implementation methods one to seven.

[0071] Implementation method ten, a computer-readable storage medium described in this implementation method, wherein the computer-readable storage medium is used to store a computer program, and the computer program executes a LoRA-embedded instruction data mining method described in any one of implementation methods one to seven.

[0072] Implementation Method 11: See Figure 2 and Figure 3 This embodiment provides a specific example of the LoRA embedded instruction data mining method described in the first embodiment, and is also used to explain the second to seventh embodiments, specifically:

[0073] This embodiment proposes a method for filtering instruction data to solve the problem that high data repetitiveness may lead to overfitting and fixed memory mode in model training, which is used to filter a large data set to obtain a more diverse data subset. Figure 1 As shown in the figure, the overall framework is to first train the Llama model on the ShareGPT dataset using the LoRA method, and obtain the embedded representation of the instruction data in the dataset through the LoRA dimension reduction matrix. Then, the high-dimensional representation of the instruction data is reduced by the UMAP algorithm to obtain the representation of each instruction in three-dimensional space. Finally, the dataset is screened by minimizing the density of the instruction data in three-dimensional space to obtain a more diverse dataset.

[0074] In order to perform machine processing on instruction data, this implementation summarizes the use of a large language model to obtain the numerical representation of instruction data. For a large language model, the representation of an instruction data can be obtained through the embedding layer of the model. Embedding refers to the process of mapping data in one space to another space. For the language model in this scheme, this process is the process of mapping the input language to a low-dimensional vector space. The mapped low-dimensional vector can represent the semantics of the word or sentence in the model. The embedding layer converts each word of the input language sequence into a corresponding word embedding. The word embedding is a vector that represents the semantics of the word. In order to subsequently analyze the relationship between instructions, different instruction data first pad short instructions and truncate long instructions to obtain sequences of the same length. After passing through the embedding layer, a sequence of word embedding matrices of equal length will be obtained.

[0075] For a dataset, the embedding layer of a large language model often needs to be further fine-tuned after pre-training to more accurately obtain the semantic representation of the corpus in the dataset. Therefore, it is first necessary to fine-tune the model based on ShareGPT data.

[0076] LoRA is an efficient way to fine-tune the model, which can save a large amount of computing resources consumed during full-parameter fine-tuning of the large language model. Its principle is to add additional network layers to the model, freeze the pre-trained model weight parameters, and only train the parameters of these newly added network layers, thereby reducing the computing resources required during the training process. The process is as Figure 2 shown.

[0077] Specifically, assuming the original parameters of the pre-trained model are W0, the process of updating them is as follows:

[0078] W = W0 + △W

[0079] where W is the new parameter, W0 is the original parameter, and △W is the part that needs to be updated. For full-parameter fine-tuning, the amount of this parameter is the same as W0. That is, assuming Llama-70B needs to be fine-tuned, 70B parameters need to be updated, which requires a high demand for computing resources.

[0080] In the LoRA method, only △W can be fine-tuned. For it can be transformed into the following update:

[0081]

[0082] where r << min(d, k), W0 remains fixed, and the AB matrix is fine-tuned. Compared with the original fine-tuning the number of parameters that need to be changed in this fine-tuning method will be greatly reduced.

[0083] In this embodiment, all parameters of the Llama model are fixed, and the rank decomposition matrix is injected into the embedding layer of the model input in the LoRA manner and fine-tuned. Different from the general method of using the word embedding matrix obtained after passing data through the embedding layer as the representation of the data, this embodiment uses the LoRA embedding representation matrix after dimensionality reduction operation on the input instruction data as the representation of the instruction data. Compared with the traditional method, the data representation obtained by this method has a lower dimension and can more effectively extract the key information of the data.

[0084] After the dimensionality reduction operation, some key features of the data have been extracted, but its dimension is still insufficient for subsequent intuitive analysis. This patent uses the UMAP dimensionality reduction algorithm to further reduce the dimension of the representation of the instruction data to extract key features to obtain a more intuitive representation of different instructions.

[0085] The UMAP algorithm is a nonlinear dimensionality reduction algorithm. The principle of this method is to use manifolds and projection technology to achieve dimensionality reduction. UMAP first calculates the high-dimensional manifold structure features, determines the positional relationship between each point, and thus constructs a high-dimensional data distribution structure. Then it projects them into a low-dimensional space, and then extracts the eigenvalues ​​based on the relative relationship between points in the high-dimensional space, and reconstructs this distance relationship in the low-dimensional space, thereby achieving the effect of clustering and feature extraction.

[0086] Specifically, UMAP uses the traditional Euclidean distance to calculate the distance between two points in high-dimensional space. In order to construct the initial high-dimensional graph, UMAP constructs a weighted graph called a fuzzy simplicial complex, where the edge weight represents the possibility of connecting two points. To determine connectivity, UMAP extends a radius outward from each point and connects points when these radii overlap. The appropriate radius is selected so as not to form too isolated groups or connect all points. UMAP selects the radius based on the distance to the nth nearest neighbor of each point. The graph is then blurred by reducing the possibility of connection as the radius increases. Finally, by stipulating that each point must be connected to at least its nearest neighbor, the local structure is ensured to be balanced with the global structure.

[0087] Thus, an approximate manifold is learned from the high-dimensional space, and then UMAP maps it to the low-dimensional space to make it as similar as possible. First, the minimum distribution of points is controlled to avoid the situation where many points overlap each other in the low-dimensional embedding. Then, a good low-dimensional manifold representation is found that is as similar as possible to the high-dimensional representation. This process uses the cross entropy function to minimize the cross entropy between the high-dimensional and low-dimensional weighted graphs, and finally makes the low-dimensional representation match the high-dimensional representation as much as possible.

[0088] After obtaining the low-dimensional representation, the KMeans method is used to perform clustering based on the Euclidean distance. It is believed that the closer the distance between two targets is, the greater the similarity is. The partitioning scheme of K clusters is iteratively searched to minimize the loss function corresponding to the clustering result. Among them, the loss function can be defined as the sum of squared errors of each sample from the center point of the cluster to which it belongs:

[0089]

[0090] where x i represents the i-th sample, c i For x i The cluster, is the center point corresponding to the cluster, M is the number of samples

[0091] This can give a visual clustering diagram, such as Figure 3As shown, it can be observed that the data can be effectively and intuitively classified and represented, so LoRA's embedding has a certain representation ability for the data. We can use this to filter the instruction data to obtain a more diverse data subset.

[0092] Based on the previous work, an intuitive representation of the instruction data in three-dimensional space has been obtained, and the data has an obvious classification and clustering relationship. This implementation method selects the instruction data according to density in the embedding space of LoRA, so that the density of the data finally selected is the smallest, and the selection of instruction data is completed. Specifically, based on all the instruction data D and the sampled data D', this method selects a new instruction data according to the following formula:

[0093]

[0094] In this way, each time the new data is screened, the sum of the Euclidean distances with the previously selected data is maximized. In this way, the density of the sampled instruction data in the LoRA representation space is minimized, thereby obtaining a more diverse data subset.

[0095] Furthermore, LLaMA-7B is used as the language model in this embodiment, and ShareGPT is used as the instruction data set. LLaMA is an open and efficient large-scale basic language model released by MetaAI. Its data sets are all public data sets, without any customized data sets, which ensures that the work is compatible with open source and reproducible. It has been recognized and adopted by many research institutes through its excellent performance. The ShareGPT data set is data uploaded by users through the data collection API of the ShareGPT plug-in. It is the user's conversation data. It contains conversation samples from different fields, themes, styles and emotions, covering a variety of types such as small talk, Q&A, stories, poetry, lyrics, etc., with high quality, diversity, personalization and emotionality.

[0096] The model performance evaluation uses the MT-bench dataset and AlpacaFarm's automatic evaluation method. AlpacaFarm is a low-cost, high-performance model with feedback close to actual humans. This patent uses this method to evaluate the win rate of the results generated by the model on the AlpacaFarm test set to represent the performance of the model. MT-bench is a multi-round question set used to evaluate the chatbot's ability to conduct multi-round conversations and follow instructions.

[0097] This implementation uses LoRA embedding to filter the ShareGPT dataset according to different scales, and randomly selects a subset of data of the same scale. Based on these data, LLaMA-7B is fine-tuned and performance tested.

[0098] Specifically, this experiment used LoRA representation and random selection in the ShareGPT dataset to select two datasets of 2K and 10K, a total of four datasets. After fine-tuning LLaMA-7B, performance tests were performed on the alpaca-farm and MT-bench test sets. The experimental results are shown in Table 1. The model performance has been greatly improved. It can be concluded that this LoRA-based embedding selection method can effectively improve the diversity of the dataset, and this improvement in diversity has a strong positive impact on the improvement of model training.

[0099] Table 1 Experimental results

[0100]

[0101] The data screening method provided in this embodiment can greatly save the amount of calculation in reducing training data, and at the same time can well improve the performance of the model, which is of great significance in model training and fine-tuning in the era of large models.

[0102] The above further describes in detail the technical solution provided by the present invention in conjunction with the accompanying drawings in order to highlight the advantages and benefits, and is not intended to be a limitation of the present invention. Any modification, combination of implementation modes, improvement and equivalent substitution of the present invention within the spirit and scope of the present invention shall be included in the protection scope of the present invention.

Claims

1. A command data mining method based on LoRA embedding, characterized in that: The method comprises: The ShareGPT dataset is trained using LoRA to obtain the embedded representation of the instruction data in the ShareGPT dataset. The ShareGPT dataset is the data uploaded by the user through the data collection API of the ShareGPT plug-in. It is the user's conversation data and contains conversation samples from different fields, topics, styles and emotions. Performing dimensionality reduction on the embedded representation by using a UMAP algorithm to obtain a representation of each instruction in a three-dimensional space; Each instruction representation is screened according to minimizing the density of instruction data in three-dimensional space to obtain a data subset with enhanced diversity.

2. According to a LoRA embedded instruction data mining method according to claim 1, it is characterized in that: The LoRA method is used to train the ShareGPT dataset, including: Fix all parameters of the Llama model; The rank decomposition matrix is ​​injected into the embedding layer of the Llama model input using the LoRA method; Connect the model with the LoRA-injected rank factorization matrix to the training dataset and fine-tune the embedding layer of the model; The model injected with the LoRA matrix is ​​trained on the ShareGPT dataset to obtain the embedded representation of the instruction data in the ShareGPT dataset.

3. A method for mining command data based on LoRA embedding according to claim 2, characterized in that: The model injected with the LoRA matrix is ​​used to train the ShareGPT dataset, specifically: the LoRA dimensionality reduction matrix is ​​used to train the ShareGPT dataset.

4. According to a LoRA embedded instruction data mining method according to claim 1, it is characterized in that: The step of reducing the dimension of the embedded representation by using the UMAP algorithm to obtain the representation of each instruction in the three-dimensional space includes: Calculate the manifold structure characteristics of high-dimensional space and obtain the positional relationship of each data point; Constructing a data distribution structure in a high-dimensional space according to the positional relationship between the data points; The data distribution structure of the high-dimensional space is projected into the low-dimensional space to obtain the representation of each instruction in the three-dimensional space.

5. According to a LoRA embedded instruction data mining method according to claim 4, it is characterized in that: The calculating of the manifold structure characteristics of the high-dimensional space and obtaining the positional relationship of each data point includes: using the Euclidean distance to calculate the distance between two data points in the high-dimensional space.

6. According to a LoRA embedded instruction data mining method according to claim 4, it is characterized in that: The projecting the data distribution structure of the high-dimensional space to the low-dimensional space includes: minimizing the cross entropy of the data distribution of the high-dimensional space and the low-dimensional space by using a cross entropy function.

7. The method for mining command data based on LoRA embedding according to claim 1, characterized in that: The screening of each instruction representation according to minimizing the density of instruction data in three-dimensional space includes: clustering the instruction data based on Euclidean distance using a KMeans method.

8. A command data mining system based on LoRA embedding, characterized in that: The system comprises: A training unit is used to train the ShareGPT dataset in a LoRA manner to obtain an embedded representation of instruction data in the ShareGPT dataset. The ShareGPT dataset is data uploaded by a user through the data collection API of the ShareGPT plug-in. The data is the user's conversation data and contains conversation samples from different fields, topics, styles, and emotions. A dimensionality reduction unit, used to reduce the dimension of the embedded representation by using a UMAP algorithm to obtain a representation of each instruction in a three-dimensional space; The screening unit is used to screen each instruction representation according to the density of the minimized instruction data in the three-dimensional space to obtain an enhanced data subset.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes a LoRA-embedded instruction data mining method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program executes a LoRA-embedded instruction data mining method according to any one of claims 1 to 7.

Images