Data processing method and apparatus

Abstract

The present application provides a data processing method and apparatus. The method comprises: acquiring a natural language processing (NLP) training data set; inputting the NLP training data set into a first large language model to train the first large language model, the trained first large language model comprising a first language modeling parameter; acquiring the first language modeling parameter; obtaining first text data on the basis of the NLP training data set; and obtaining a first prompt word set on the basis of the first text data, the first language modeling parameter, and a second large language model, wherein the first prompt word set and the first text data are used for training a third large language model, and the first large language model and the second large language model are based on the same model architecture. The method provided by the present application can obtain the first prompt word set of which the data volume is much smaller than that of the NLP training data set. In this way, using the first prompt word set to train the third large language model can improve the speed and accuracy of large language model training.

Description

Data processing methods and apparatus

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411183248.6, filed on August 27, 2024, entitled "Data Processing Method and Apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of artificial intelligence, and more specifically, to a data processing method and apparatus. Background Technology

[0004] Currently, with the maturity of Large Language Model (LLM) technology and the growth of its structure, the demand for training data is also increasing. In order to reduce training costs, data selection or knowledge distillation are generally used to reduce the amount of training data.

[0005] On the one hand, due to the semantic diversity and complexity, context dependence, and long-tail distribution of natural language, it is often difficult to select all representative data, resulting in suboptimal performance of LLMs trained using data selection methods. On the other hand, while knowledge distillation has shown some application potential in computer vision (CV), this is because the pixel values ​​of images are inherently continuous, and adjacent pixel values ​​are often visually similar, making this technique more direct and effective when processing such data. Data in Natural Language Processing (NLP) is essentially a sequence of text, each consisting of a series of words, where each word is encoded as its position in the vocabulary. This encoding method only maps the index of the word, not its inherent meaning. That is, words in adjacent positions in the vocabulary do not necessarily have close semantic relationships. This characteristic makes NLP data inherently discrete, leading to particularly complex methods for achieving knowledge distillation through augmented text data. Summary of the Invention

[0006] This application provides a data processing method and apparatus to improve the speed and accuracy of training large language models.

[0007] In a first aspect, a data processing method is provided, comprising: acquiring a natural language processing (NLP) training dataset; inputting the NLP training dataset into a first large language model to train the first large language model, wherein the trained first large language model includes first language modeling parameters, which are language modeling parameters of the first large language model; acquiring the first language modeling parameters; obtaining first text data based on the NLP training dataset, wherein the first text data is a portion of the text data in the NLP training dataset; and obtaining a first prompt word set based on the first text data, the first language modeling parameters, and the second large language model, wherein the first prompt word set and the first text data are used to train a third large language model, and the first large language model and the second large language model are based on the same model architecture.

[0008] The data processing method provided in this application adjusts the parameters of the second language model based on the parameters of the first language model. The second language model is used to obtain a first set of prompt words with a much smaller data volume than the NLP training dataset. Thus, training the third language model using the first set of prompt words can improve the speed and accuracy of training the large language model.

[0009] In some implementations of the first aspect, obtaining a first set of prompt words based on first text data, first language modeling parameters, and a second large language model includes: inputting the initial value of the first set of prompt words and the first text data into the second large language model, and adjusting the second language modeling parameters according to the loss, wherein the second language modeling parameters are the language modeling parameters of the second large language model, the initial value of the second language modeling parameters is determined according to the first language modeling parameters, and the loss is determined according to the first language modeling parameters and the second language modeling parameters; obtaining the first set of prompt words based on the second large language model.

[0010] For training a typical large language model, such as the first large language model in the aforementioned embodiment, the language modeling learned by the large language model is used to predict the corresponding complete text data based on a cue word sequence and a portion of the text. In this embodiment, the training objective of the second large language model is to reconstruct the cue word sequence corresponding to the text data based on the language modeling parameters of the first large language model and the text data. The second large language model can improve the data quality of the NLP training dataset obtained through knowledge distillation, thereby improving the accuracy of the large language model training.

[0011] In some implementations of the first aspect, adjusting the second language modeling parameters according to the loss includes: inputting the second text data and the second set of prompt words into the second language model, and adjusting the second language modeling parameters according to the loss to update the second set of prompt words; wherein the second text data and the second set of prompt words are selected from the first text data and the first set of prompt words output by the second language model in adjusting the second language modeling parameters, respectively.

[0012] In this implementation, the first set of prompt words is easily reconstructed by the second large language model, which can improve the data quality of the NLP training dataset obtained by knowledge distillation, thereby increasing the training speed of the large language model.

[0013] In some implementations of the first aspect, the loss is determined based on the first language modeling parameters, the second language modeling parameters, the nearest neighbor words of the second cue word set in the first large language model, and the second cue word set.

[0014] In this implementation, the loss can make the obtained first prompt word set as close as possible to its nearest neighbor word in the first large language model, so as to improve the data quality of the first prompt word set and improve the accuracy of training the large language model.

[0015] In some implementations of the first aspect, the method further includes: when the representation space of the first large language model and the representation space of the third large language model are the same, inputting the first set of prompt words and the first text data into the third large language model to train the third large language model.

[0016] In some implementations of the first aspect, the method further includes: when the representation space of the first large language model and the representation space of the third large language model are different: decoding the first prompt word set into natural language text according to the first large language model; encoding the natural language text into a third prompt word set according to the third large language model; and inputting the third prompt word set and the first text data into the third large language model to train the third large language model.

[0017] In both of the above implementation methods, large language models based on different model architectures can be trained using high-quality sets of first prompt words to improve the training speed and accuracy of large language models.

[0018] In some implementations of the first aspect, the first text data is obtained by selecting S text data from F text data in the NLP training dataset based on representative sampling or sampling based on a neural network model.

[0019] In this implementation, representative sampling or sampling based on neural network models can make the first text data include as much information as possible from the NLP training dataset, thereby reducing the amount of data used when training large language models and improving the accuracy of large language model training.

[0020] In some implementations of the first aspect, the value of S / F ranges from 0.02 to 0.1.

[0021] In this implementation, a reasonable data volume setting can ensure that the first text data includes as much information as possible from the NLP training dataset, thereby reducing the amount of data used when training the large language model and improving the accuracy of the large language model training.

[0022] In some implementations of the first aspect, the first language modeling parameters include the parameter trajectory of the expert parameters of the first large language model.

[0023] In this implementation, the parameter trajectory of the expert parameters can reflect the changes in the parameters of language modeling during the training process to a certain extent.

[0024] In a second aspect, a data processing apparatus is provided, including a processing module for performing the method as described in any implementation of the first aspect.

[0025] Thirdly, a computing device cluster is provided, the computing device cluster including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the computing device cluster to perform the method as described in any implementation of the first aspect.

[0026] Fourthly, a computer program product containing instructions is provided, which, when run by a cluster of computing devices, cause the cluster of computing devices to perform the method as described in any implementation of the first aspect.

[0027] Fifthly, a computer-readable storage medium is provided, including computer program instructions that, when executed by a cluster of computing devices, perform the method as described in any implementation of the first aspect. Attached Figure Description

[0028] Figure 1 is a flowchart illustrating the data processing method provided in an embodiment of this application.

[0029] Figure 2 is a flowchart illustrating the data processing method provided in an embodiment of this application.

[0030] Figure 3 is a schematic block diagram of a method for adjusting the parameters of language modeling of a large language model according to an embodiment of this application.

[0031] Figure 4 is a schematic block diagram of a data processing device provided in an embodiment of this application.

[0032] Figure 5 is a schematic block diagram of a computing device provided in an embodiment of this application.

[0033] Figure 6 is a schematic block diagram of a computing device cluster provided in an embodiment of this application.

[0034] Figure 7 is a schematic block diagram of a computing device cluster provided in an embodiment of this application. Detailed Implementation

[0035] First, let's explain the technical terms related to this application.

[0036] In machine learning, especially deep learning, knowledge distillation is a model compression technique involving two main components: a teacher model and a student model. The teacher model is typically a large, complex, and well-trained neural network; the student model is a smaller, simpler model designed to mimic the teacher model's behavior to achieve similar or near-similarity in performance, but with lower computational cost and faster inference speed. During knowledge distillation, the teacher model is first trained on a large dataset until satisfactory performance is achieved. Then, the student model begins training. Generally, the student model can use the same dataset as the teacher model; however, in some cases, the student model may use less data during the knowledge distillation phase than the teacher model did during training. This is because the student model can utilize patterns and features learned by the teacher model from the large dataset as additional guidance data.

[0037] Data selection is a data preprocessing step that involves selecting a subset of data from the original dataset for model training. The goal of data selection is to reduce training time and resource consumption while minimizing the impact of a small amount of data on model performance. Typically, data selection can be achieved through representative sampling, often using stratified sampling or cluster analysis to ensure the sampled data reflects the characteristic distribution of the entire dataset; or through model-based sampling, where a pre-trained model scores the importance of data points, and the data points with the highest scores are selected.

[0038] In the following descriptions and embodiments, the large language model (LLM) may be referred to as a language model, or simply as a model.

[0039] Language modeling is a statistical model used to predict the probability of a series of words or sentences within a text sequence. This model is trained on large-scale text data to learn co-occurrence relationships between words, grammatical rules, and semantic information, thereby generating a probability distribution for a given text sequence and predicting the next word or sentence. Next word prediction (NWP) is a specific task in language modeling and one of the fundamental tasks in LLM training. This task requires the model to predict and generate the most likely next word based on the given text context. Similar to next word prediction, next sentence prediction (NSP) is also a specific task in language modeling; however, it focuses on sentence-level coherence and logical relationships, rather than just word-level prediction. These two tasks complement each other and together constitute an important part of language modeling.

[0040] A vocabulary is a collection of all recognizable words, symbols, or tokens in a large language model. Each word has a unique index in the vocabulary, which is used to establish a mapping between the model's input and output layers.

[0041] A prompt is one or more words used to stimulate or guide a large language model (LLM) to produce a certain type of output or response. In natural language processing (NLP), a prompt can be a question, a topic, the beginning of a dialogue, or any type of instruction. They are typically placed at the beginning or appropriate position of the model's input so that the model can generate or infer accordingly. Prompts can be used in a variety of NLP tasks, including but not limited to text generation, question answering, translation, summarization, and sentiment analysis, helping the model understand the user's intent or needs.

[0042] A prepended prompt is a specific way of using prompts where the prompt is explicitly placed at the very beginning of the input sequence. This pre-positioning helps the model notice the prompt first when processing the input, thus allowing it to better adjust its generation strategy or inference path.

[0043] Enhanced text representation enriches the representation of the original text by integrating additional information, thereby improving the model's ability to understand and predict text. Enhanced text representation typically includes contextual and syntactic information from the input text representation, as well as pre-prompt word representations added before the original text representation. One data point from a set of enhanced text representations is called an enhanced text data point, and an enhanced text data point must include at least one text data point and a set of prompts. It should be understood that language modeling can predict the corresponding text data based on the pre-prompt word sequence in the enhanced text representation.

[0044] The representation space refers to the internal representations used by a model when processing natural language. When a word, phrase, or sentence is input into the model, it is transformed into a multi-dimensional vector (i.e., a representation), which exists in the model's representation space. The dimensions and structure of this space are determined by the model's architecture. The representation space of a language model captures the semantic information of words, syntactic structure, and contextual dependencies, enabling the model to understand and generate coherent text.

[0045] The latent representation space, or simply latent space, is an abstract, multi-dimensional space within a neural network model. It is typically used to capture complex patterns and inherent structures within a dataset. Correspondingly, latent representations are a compact and efficient encoding of the original data. This encoding is often lower in dimensionality than the original data, yet retains its key features and information. Specifically, the latent space maps high-dimensional data to a low-dimensional space through dimensionality reduction, where points represent the main features of the original data. Points in the latent space are usually continuous, and nearby points often correspond to similar instances of the original data. Typically, the length of the cue word vectors is set to match the dimension of the latent space of the neural network model (i.e., the length of the latent representation). This setting not only simplifies computation but also captures the semantic and syntactic characteristics of words, improving the model's generalization ability and prediction accuracy.

[0046] Nearest neighbors: In representation space, the other points that are closest to a given point are called that point's nearest neighbors. For word vectors, nearest neighbors mean the words that are semantically most relevant. The closest words can be found by calculating the distance between word vectors, such as Euclidean distance or cosine similarity. When a synthesized sequence is input into a language model, starting with a specific preceding cue word, the model converts this cue word into its vector representation in the representation space. This vector is then the one with the closest distance to the nearest neighbor in the representation space.

[0047] The concept of expert parameters in LLM (Layered Modeling) comes from the Mixture of Experts (MoE) architecture, a strategy for achieving model parallelism and sparsity in neural networks. In the MoE architecture, the neural network consists of multiple experts, each responsible for processing a portion of the input, rather than the entire network processing all inputs. This significantly reduces computational costs. During training, expert parameters refer to parameters such as weights and biases for each expert layer in the MoE architecture. These parameters are adjusted during training so that each expert specializes in processing a specific type of input, thereby improving the model's efficiency and performance. When an input arrives at a MoE layer, it is routed to the most appropriate expert for processing.

[0048] In LLM (Local Modeling), parameter trajectories refer to the paths along which model parameters change during training. In each iteration, the model parameters are updated based on the gradient of the loss function. This update process forms a continuous path from the initial parameters to the final parameters; this is the parameter trajectory. Parameter trajectories are crucial for understanding the dynamic behavior of the model during training. They help researchers analyze how the model converges, whether overfitting or underfitting exists, and the impact of different hyperparameter settings on model performance. Furthermore, visualizing parameter trajectories allows us to intuitively see which parameters change significantly during training and which change less, which is very helpful for model tuning and training optimization.

[0049] Low-Rank Adaptation (LoRA) is a technique used in machine learning and deep learning for fine-tuning pre-trained models. In natural language processing, pre-trained models typically have a large number of parameters, and directly fine-tuning the entire model on downstream tasks can require significant computational resources and time. LoRA provides a parameter-efficient fine-tuning strategy that modifies only a small subset of parameters in the model, rather than the entire model, thereby reducing computational costs and storage requirements.

[0050] The following section introduces common LLM and NLP training datasets.

[0051] The Open Pre-trained Transformer (OPT) model family is an open-source LLM based on the Transformer architecture. This series of models includes versions of different scales. For example, OPT-1.3B represents a model with approximately 1.3 billion parameters, while OPT-175B is a super-large-scale model with 175 billion parameters.

[0052] The Llama2 model family is an open-source LLM based on the Transformer architecture. This series of models includes versions of different sizes. For example, Llama2-7B represents a model with approximately 700 million parameters, while Llama2-70B is a super-large-scale model with 70 billion parameters.

[0053] Common NLP training datasets include Flan v2, CoT, Dolly, and Open Assistant 1. Flan v2 aims to enhance the model's generalization ability across multiple language tasks, CoT focuses on evaluating the model's reasoning ability, Dolly is designed to improve dialogue understanding and generation, and Open Assistant 1 focuses on creating a versatile AI assistant capable of understanding and executing complex instructions.

[0054] The following section introduces common data processing and data analysis methods in the field of AI.

[0055] Data preprocessing in the AI ​​field mainly includes normalization and standardization. Normalization scales the data to a fixed range, typically between [0,1] or [-1,1]. This ensures that all features have equal importance, preventing some features from having excessively large or small ranges that could negatively impact model training. Standardization transforms the data into a distribution with a mean of 0 and a variance of 1. The most commonly used z-score standardization formula is: z = (x - μ) / σ, where z is the standardized value, x is the original data, μ is the mean of the data, and σ is the standard deviation of the data.

[0056] In the training process of machine learning and deep learning, the loss function is a key metric for evaluating the difference between the model's predictions and the true labels. The goal of the model is to minimize this loss function to improve prediction accuracy and overall model performance. Common loss functions include mean squared error and multi-class cross-entropy loss.

[0057] Mean squared error (MSE) is a commonly used metric for evaluating the predictive performance of a model. It measures the model's predicted values ​​(reconstructed features) from an n-dimensional vector. The average error between the true value (original feature) X and the mean error. MSE has multiple representations, and the specific calculation method is shown in Equation (1).

[0058] MSE can also represent vector A(A1,A2,…,A…). n ) and vector B(B1,B2,…,B n The Euclidean distance between them is shown in equation (2).

[0059] Multiclass cross-entropy loss (MCE) is a commonly used loss function in classification tasks, especially in multi-class classification problems. It measures the difference between the probability distribution predicted by the model and the probability distribution of the true labels. For a classification task Y with N samples and C classes, Y is typically a one-dimensional vector of length N. The model generates the predicted probability distribution for each class. Typically, it's a matrix of shape (N, C), where each row represents a sample and each column represents the predicted probability of a class. Assume Y... ic It is the probability that the true label is class c (e.g., Y). ic It is a one-hot encoding matrix, where the probability of the true class label is 1, and the rest are 0. Let be the probability that the sample predicted by the model belongs to class c. Then the definition of MCE loss is as shown in equation (3).

[0060] During neural network training, the gradient of the loss function L with respect to the network weights w is calculated, denoted as... This gradient vector contains information about the partial derivative of the loss function with respect to each weight. In the backpropagation algorithm, this gradient is used to update the network weights to reduce the value of the loss function, thereby improving the network's prediction performance. For example, when training using the gradient descent algorithm, the weight update rule is shown in equation (4). Where w t+1 It is the updated weight, w t η represents the current weights, and η is the learning rate of the neural network.

[0061] Taking MSE as the loss function as an example, the reconstructed features of the sample We can use f(X,w) tThis indicates that the variable in the loss function L is the weight w. t However, it implicitly includes the sample and the reconstructed features of the sample. L(w t The specific form of ) is shown in equation (5).

[0062] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0063] Currently, with the maturity of large language model technology and the growth of its structure, the demand for training data is also increasing. In order to reduce training costs, data selection or knowledge distillation are generally used to reduce the amount of training data.

[0064] On the one hand, due to the semantic diversity and complexity, context dependence, and long-tail distribution of natural language, it is often difficult to select all representative data, resulting in suboptimal performance of LLMs trained using data selection methods. On the other hand, while knowledge distillation has shown some application potential in computer vision (CV), this is because the pixel values ​​of images are inherently continuous, and adjacent pixel values ​​are often visually similar, making this technique more direct and effective when processing such data. Data in Natural Language Processing (NLP) is essentially a sequence of text, each consisting of a series of words, where each word is encoded as its position in the vocabulary. This encoding method only maps the index of the word, not its inherent meaning. That is, words in adjacent positions in the vocabulary do not necessarily have close semantic relationships. This characteristic makes NLP data inherently discrete, leading to particularly complex methods for achieving knowledge distillation through augmented text data.

[0065] In view of this, embodiments of this application provide a data processing method, the method comprising: acquiring a Natural Language Processing (NLP) training dataset; inputting the NLP training dataset into a first large language model to train the first large language model, the trained first large language model including first language modeling parameters, the first language modeling parameters being language modeling parameters of the first large language model; acquiring the first language modeling parameters; obtaining first text data based on the NLP training dataset, the first text data being a portion of text data in the NLP training dataset; and obtaining a first prompt word set based on the first text data, the first language modeling parameters, and a second large language model, wherein the first prompt word set and the first text data are used to train a third large language model, and the first large language model and the second large language model are based on the same model architecture.

[0066] The data processing method provided in this application adjusts the parameters of the second language model based on the parameters of the first language model. The second language model is used to obtain a first set of prompt words with a much smaller data volume than the NLP training dataset. Thus, training the third language model using the first set of prompt words can improve the speed and accuracy of training the large language model.

[0067] The specific process of the data processing method provided in this application will be described below with reference to the flowchart of the data processing method shown in Figure 1.

[0068] S210, Obtain the NLP training dataset, input the NLP training dataset into the first language model to train the first language model, the trained first language model includes the first language modeling parameters.

[0069] It should be understood that the goal of training is to enable the primary language model to learn an effective representation method that can capture language structure, semantics, and contextual information. This ability is called language modeling, and the parameters of language modeling can be stored through its internal parameter matrix and attention mechanism. The parameters of the primary language model's language modeling can be referred to as the primary language modeling parameters. For example, the primary language model can be used to predict corresponding text data based on cue word sequences. Furthermore, the parameter trajectory of the expert parameters, i.e., the changes in the expert parameters during training, can, to some extent, reflect the changes in the language modeling parameters during training.

[0070] In some embodiments, OPT-1.3B can be selected as the first language model, and an NLP training dataset can be selected as the text data input to the first language model to train the first language model. The NLP training dataset can be at least one of Flan v2, CoT, Dolly, and Open Assistant1.

[0071] In some embodiments, the first large language model includes multiple expert modules, each responsible for handling different types of input. Text data is input into the first large language model, trained for M epochs using LoRA, and multiple sets of first expert parameters are saved. For example, the rank of the LoRA module can be set to 128, the hyperparameter α for weight scaling can be set to 512, and the dropout hyperparameter for preventing overfitting can be set to 0.1.

[0072] In training a neural network model, a training epoch refers to the process of passing the entire training dataset through the model once (including forward and backward propagation). In other words, one epoch means the model has the opportunity to see every sample in the training dataset once. During each training epoch, the model parameters are progressively optimized through multiple batches, ensuring the dataset is completely traversed once. For example, if the training dataset contains 240,000 text samples, and each batch has a sample size of 40, then 6,000 batches are needed to complete one training epoch.

[0073] In the early stages of training (the first or first few epochs), model parameters are typically initialized randomly and may be far from the optimal solution. At this point, the parameter trajectory exhibits large adjustments as gradient descent forces the parameters to move rapidly in directions that significantly reduce the loss function; the learning rate is usually high at this stage to allow the model to quickly explore the parameter space. As training progresses, after several epochs, the model begins to learn patterns in the dataset, the magnitude of parameter adjustments decreases, and the parameter trajectory becomes smoother; if a dynamic learning rate strategy is used, the learning rate may gradually decrease, resulting in more granular parameter updates that help the model approach local minima. At this point, the parameter trajectory may begin to oscillate around a local minimum, indicating that the model is trying to find the minimum point of the loss function. In the convergence phase (the last few epochs), as the model approaches the optimal solution, the changes in the parameter trajectory become very small, and parameter updates almost stagnate. This means that the model has learned most of the information in the dataset.

[0074] S220, obtain the first language modeling parameters.

[0075] In some embodiments, the parameter trajectory of the first expert parameter of the first large language model is used as an example to illustrate the first language modeling parameters, and the parameter trajectory of the first expert parameter of the first large language model is obtained. The parameter trajectory of the first expert parameter is illustrated using one of multiple sets of first expert parameters as an example. Specifically, the first expert parameter can be a parameter trajectory composed of M parameters, where each of the M parameters corresponds to one round of training. In this embodiment, M is a positive integer, which can be 5 to 10, and the sample size of a batch can be 8 to 64 to achieve the best training effect.

[0076] For example, when training using the gradient descent algorithm, the update rule for the first expert parameter is shown in equation (6). Here, L is the loss function, and θ... t+1 These are the updated parameters, θ t Here, is the current parameter, η is the learning rate of the neural network, 1≤t≤M. The loss function L can be either MCE or MSE.

[0077] In some embodiments, the above model training process is run multiple times under the same settings, which include at least the same model architecture, the same hyperparameters, the same initialization, the same training data, and the same number of training epochs. In summary, this embodiment yields multiple parameter trajectories with a first expert parameter length of M.

[0078] S230. Based on the NLP training dataset, the first text data is obtained. The first text data is a part of the text data in the NLP training dataset.

[0079] The first text data is also called the first text representation. This embodiment, in conjunction with the flowchart of the data processing method shown in Figure 2, details the logical relationship between steps S210-220, S230, and S240.

[0080] In some embodiments, the first text representation D sel This data is derived from a portion of the text data in the NLP training dataset. Specifically, the NLP training dataset includes F data points, each of which may be a document, article, sentence, or phrase, and each data point corresponds to a high-dimensional vector in the representation space. If all the text data is converted into a text representation, then that text representation also includes F data points.

[0081] In one possible implementation, the first text representation D sel It includes S data points (also called S samples), and the value of S / F ranges from 0.02 to 0.1, meaning that the first text representation is obtained from 2% to 10% of the text data in the NLP training dataset. Here, S and F are positive integers.

[0082] In this implementation, a reasonable data volume setting can ensure that the first text data includes as much information as possible from the NLP training dataset, thereby reducing the amount of data used when training the large language model and improving the accuracy of the large language model training.

[0083] In one possible implementation, data can be selected from the NLP training dataset through representative sampling or sampling based on a neural network model, thereby selecting S data points from F data points in the NLP training dataset to convert into the first text representation D. sel The specific methods and related technologies are similar and will not be elaborated here.

[0084] In this implementation, representative sampling or sampling based on neural network models can make the first text data include as much information as possible from the NLP training dataset, thereby reducing the amount of data used when training large language models and improving the accuracy of large language model training.

[0085] S240, based on the first text data, the first language modeling parameters, and the second language model, obtain the first set of prompt words.

[0086] The method in this embodiment is a knowledge distillation method. The first large language model is used as the teacher model, and the second large language model is used as the student model. Their parameter configurations and invocation methods may be different, but they are based on the same model architecture. In this embodiment, both the first and second large language models are OPT-1.3B, which ensures their similarity in functionality and performance.

[0087] In some embodiments, the initial value of the first prompt word set and the first text data are input into the second language model, and the second language modeling parameters are adjusted according to the loss. The second language modeling parameters are the language modeling parameters of the second language model. The initial value of the second language modeling parameters is determined according to the first language modeling parameters, and the loss is determined according to the first language modeling parameters and the second language modeling parameters.

[0088] It should be understood that for the training of a typical large language model, such as the training of the first large language model in the aforementioned embodiment, the language modeling learned by the large language model is used to predict the corresponding complete text data based on a cue word sequence and a portion of the text. In this embodiment, the second large language model is used to reconstruct the cue word sequence corresponding to the text data based on the language modeling parameters of the first large language model and the text data. The second large language model can improve the data quality of the NLP training dataset obtained through knowledge distillation, thereby improving the accuracy of the large language model training.

[0089] In this embodiment, the model architecture of the first language modeling parameters and the second language modeling parameters is the same, and the specific values ​​(features) are similar. The first language model and the second language model have similar capabilities. This capability can be used to predict the corresponding complete text data based on a prompt word sequence and a part of the text, and can also reconstruct the prompt word sequence corresponding to the text data based on the text data.

[0090] S241, Based on the first text data and the first set of prompt words, obtain the enhanced text dataset.

[0091] In one possible implementation, the first prompt word set D is obtained by averaging the word vectors of all words in the vocabulary of the first large language model. syn The initial value. The first cue word set D. syn It includes S cue word sequences, each cue word sequence containing K cue words, where K is a positive integer. The first cue word set is also called the first cue word representation.

[0092] In some embodiments, in the first prompt word set D synSelect a batch of cue word sequences from the set, i.e., the second cue word set p. m Select a batch of text data from the first text representation, i.e., the second text data b. m The augmented text dataset h is obtained by concatenating the second set of cue words and the second text data. m The initial values ​​are: The second cue word set includes A cue word sequences from the first cue word set, and the second text data includes A text data items from the first text representation (i.e., the sample size of a batch is A).

[0093] In one possible implementation, the first cue word set D syn It is a tensor of shape (S,K,R), where K can range from 1 to 4, and R is the dimension of the latent representation space of the first major language model. First, in D syn Select a batch of data p m The specific selection method is to randomly select an index (S1, S2, ..., S) from the S prompt word sequence. A Given a sequence of A cue words, then p... m The shape is (A, K, R). Secondly, in the first text representation D... sel Select A samples with the same index from the given list, i.e., indexes (S1, S2, ..., S...). A If there are A samples, then b m The shape is (A, W, R), where W is a constant of D. sel The length of the longest sample among the A selected samples, W, depends on the token partitioning method in the first large language model. Clearly, concatenating p... m and b m The obtained augmented text dataset h m The original feature has a shape of (A, K + W, R), and the splicing process can be represented as h m =concat(p m ,b m ).

[0094] In one possible implementation, the value of A can be in the range of 8 to 64, where A is a positive integer. Subsequent embodiments will be illustrated using M=5, S=16000, K=2, R=2048, and A=16 as examples, but this should not be construed as a limitation on the technical solution of this application.

[0095] In one possible implementation, P augmented text datasets are obtained based on S augmented text data points, where each augmented text data point is included in one augmented text dataset, and P is a positive integer, P = S / A. This method is equivalent to a random sampling method without replacement, ensuring that each augmented text data point has the same number of inputs in subsequent update processes.

[0096] S242, input the second text data and the second set of prompt words into the second language model, and adjust the second language modeling parameters according to the loss.

[0097] This embodiment, in conjunction with the schematic block diagram of the method for adjusting the language modeling parameters of a large language model as shown in Figure 3, details the process of S242.

[0098] Specifically, in the aforementioned embodiments, a total of P (P = 1000 in this embodiment) augmented text datasets are obtained, which can be used to update the second language modeling parameters in Q rounds. Each round of updates uses N augmented text datasets as input samples, and the sample size of each augmented text dataset is A, where the minimum value of Q is P / N and Q is a positive integer.

[0099] In some possible implementations, in the i-th round of updates, N augmented text datasets h are selected. m The second language model is input, and the gradient descent algorithm is used to update the second expert parameters in N mini-batches, as shown in equation (7). This represents the expert parameters updated in the m-th mini-epoch, where 1 ≤ m ≤ N, obtained after one mini-epoch. This is called the second expert parameter, and η is the learning rate of the second largest language model. The loss function represents a small round of update process. In this embodiment, the loss function selected is the cross-entropy loss function MCE.

[0100] In some possible implementations, N is a positive integer, ranging from 4 to 16. For example, if N = 5 and A = 16, then a small round of updates will use 5 augmented text datasets. In this embodiment, the input data is 1000 augmented text datasets, so the minimum value of Q is P / N = 1000 / 5 = 200, thus each augmented text dataset is traversed at least once.

[0101] In some possible implementations, at the beginning of each update round, the initial value of the second expert parameter is determined based on the parameter trajectory of the first expert parameter. Randomly select one parameter trajectory from multiple first-expert parameters. The current and updated parameters of this trajectory in round t are represented as (θ). t ,θ t+1 ), θ t The value is used as The value of t is given. In this embodiment, M=5 is used as an example, and 1≤t≤3 is set because the change in the parameter trajectory during the convergence phase becomes very small, and the parameter update almost stops. For the specific principle, please refer to the aforementioned embodiment, which will not be repeated here.

[0102] In some possible implementations, Q > P / N, such as Q = 2000. After 200 rounds of updates, each augmented text dataset has been input into the second large language model once. In this embodiment, multiple recombined augmented text datasets are obtained according to the random sampling method in S241. Then, the newly obtained 1000 augmented text datasets are input into the second large language model again for another 200 rounds of updates.

[0103] In some embodiments, after each round of updates, the learning rates of the second prompt word set, the second expert parameters, and the second largest language model corresponding to the aforementioned N augmented text datasets are adjusted based on the updated second language modeling parameters. That is, during the adjustment of the second language modeling parameters, the second prompt word set referenced in the first prompt word set is updated; specifically, the second prompt word set is selected from the first prompt word set output by the second largest language model when the second language modeling parameters are adjusted.

[0104] In this embodiment, the first prompt word set is easily reconstructed by the second large language model, which can improve the data quality of the NLP training dataset obtained by knowledge distillation, thereby increasing the training speed of the large language model. Furthermore, various combinations of augmented text datasets can be obtained based on the first text data and the first prompt word set, increasing the number of times the first prompt word set is updated and improving its data quality, thus enhancing the accuracy of the large language model training.

[0105] In some possible implementations, the loss is determined based on the first language modeling parameters, the second language modeling parameters, the nearest neighbor words of the second cue word set in the first large language model, and the second cue word set. As shown in Equation (8), the loss for each round of updates is ε. ds and ε reg A linear combination of ε ds Based on the parameter trajectory of the first expert parameter and the second expert parameter, ε is obtained. reg It is a vector v m sum vector p m The average distance between them, v m p represents the second set of cue words in the first large language model. m The nearest neighbor word, β is a constant.

[0106] In this implementation, the loss can make the obtained first prompt word set as close as possible to its nearest neighbor word in the first large language model, so as to improve the data quality of the first prompt word set and improve the accuracy of training the large language model.

[0107] S243, based on the second language model, obtain the first set of prompt words.

[0108] According to the foregoing embodiments, the second language model is used to reconstruct the prompt word sequence corresponding to the first text data based on the language modeling parameters of the first language model and the first text data. Therefore, the second language model after adjusting the second language modeling parameters can obtain the first prompt word set D based on the first text data. syn .

[0109] S250, train the third language model based on the first set of prompt words and the first text data.

[0110] In some embodiments, the representation space of the first and third language models is the same. Typically, if the first and third language models are based on the same model architecture, their representation spaces are the same; for example, if both the first and third language models are OPT-1.3B.

[0111] In one possible implementation, the first set of prompt words and the first text data are input into a third language model to train the third language model. The language model learned by the third language model is used to predict the corresponding complete text data based on a prompt word sequence and partial text. The specific training process can refer to the method used to train the first language model. In this embodiment, the amount of data required to train the third language model is much smaller than the amount of data required to train the first language model, and the training speed of the third language model is much greater than the training speed of the first language model.

[0112] In some embodiments, the representation spaces of the first and third language models are different. Typically, if the first and third language models are based on different model architectures, their representation spaces will differ; for example, the first language model is OPT-1.3B, and the second language model is Llama2-7B. Specifically, the first set of prompt words can be decoded into natural language text using the first language model, and the natural language text can be encoded into a third set of prompt words using the third language model. The combination of the third set of prompt words and the first text data is then input into the third language model to train it.

[0113] In this implementation, large language models based on different model architectures can be trained using high-quality sets of first cue words to improve the speed and accuracy of training large language models.

[0114] This application also provides a data processing apparatus, as shown in FIG4, comprising:

[0115] The module is used to acquire the Natural Language Processing (NLP) training dataset; input the NLP training dataset into the first language model to train the first language model, the trained first language model includes the first language modeling parameters, which are the language modeling parameters of the first language model; and acquire the first language modeling parameters.

[0116] The processing module is used to obtain first text data from the NLP training dataset, which is a part of the text data in the NLP training dataset; and to obtain a first prompt word set based on the first text data, first language modeling parameters, and second language model. The first prompt word set and the first text data are used to train a third language model, and the first and second language models are based on the same model architecture.

[0117] Both the processing module and the acquisition module can be implemented in software or hardware. For example, the implementation of the processing module will be described below. Similarly, the implementation of the acquisition module can be referenced from that of the processing module.

[0118] As an example of a software functional unit, a processing module may include code running on a computing instance. A computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Furthermore, the aforementioned computing instance may be one or more. For example, a processing module may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ comprising one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0119] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0120] As an example of a hardware functional unit, a processing module may include at least one computing device, such as a server. Alternatively, a processing module may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The aforementioned PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0121] The processing module includes multiple computing devices that can be distributed within the same region or in different regions. Similarly, the processing module includes multiple computing devices that can be distributed within the same Availability Zone (AZ) or in different AZs. Likewise, the processing module includes multiple computing devices that can be distributed within the same Virtual Private Cloud (VPC) or multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0122] This application also provides a computing device 1100. As shown in FIG5, the computing device 1100 includes: a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, the memory 1106, and the communication interface 1108 communicate with each other via the bus 1102. The computing device 1100 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 1100.

[0123] Bus 1102 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 5, but this does not imply that there is only one bus or one type of bus. Bus 1102 can include pathways for transmitting information between various components of computing device 1100 (e.g., memory 1106, processor 1104, communication interface 1108).

[0124] The processor 1104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0125] The memory 1106 may include volatile memory, such as random access memory (RAM). The processor 1104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0126] The memory 1106 stores executable program code, and the processor 1104 executes the executable program code to implement the functions of the aforementioned processing module and acquisition module, thereby realizing the data processing method. That is, the memory 1106 stores instructions for executing the data processing method.

[0127] The communication interface 1108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1100 and other devices or communication networks.

[0128] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0129] As shown in Figure 6, the computing device cluster includes at least one computing device 1100. The memory 1106 of one or more computing devices 1100 in the computing device cluster may store the same instructions for executing data processing methods.

[0130] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may also store partial instructions for executing data processing methods. In other words, a combination of one or more computing devices 1100 can jointly execute instructions for executing data processing methods.

[0131] It should be noted that the memory 1106 in different computing devices 1100 within the computing device cluster can store different instructions, each used to execute a portion of the functions of the data processing device. That is, the instructions stored in the memory 1106 of different computing devices 1100 can implement the functions of one or more modules among the processing module and the acquisition module.

[0132] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 7 illustrates one possible implementation. As shown in Figure 7, two computing devices 1100A and 1100B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 1106 of one or more computing devices 1100 in the computing device cluster can store the same instructions for executing data processing methods.

[0133] It should be understood that the functions of computing device 1100A shown in Figure 7 can also be performed by multiple computing devices 1100. Similarly, the functions of computing device 1100B can also be performed by multiple computing devices 1100.

[0134] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform a data processing method.

[0135] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform a data processing method.

[0136] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, include: Obtain the Natural Language Processing (NLP) training dataset; The NLP training dataset is input into the first large language model to train the first large language model. The trained first large language model includes first language modeling parameters, which are the language modeling parameters of the first large language model. Obtain the modeling parameters of the first language; The first text data is obtained based on the NLP training dataset, and the first text data is a portion of the text data in the NLP training dataset; Based on the first text data, the first language modeling parameters, and the second major language model, a first set of prompt words is obtained. The first set of prompt words and the first text data are used to train a third major language model. The first major language model and the second major language model are based on the same model architecture.

2. The method according to claim 1, characterized in that, The step of obtaining the first set of prompt words based on the first text data, the first language modeling parameters, and the second major language model includes: The initial value of the first prompt word set and the first text data are input into the second large language model, and the second language modeling parameters are adjusted according to the loss. The second language modeling parameters are the language modeling parameters of the second large language model. The initial value of the second language modeling parameters is determined according to the first language modeling parameters, and the loss is determined according to the first language modeling parameters and the second language modeling parameters. Based on the second large language model, the first set of prompt words is obtained.

3. The method according to claim 2, characterized in that, The adjustment of second language modeling parameters based on loss includes: The second text data and the second set of prompt words are input into the second large language model, and the second language modeling parameters are adjusted according to the loss to update the second set of prompt words; wherein the second text data and the second set of prompt words are respectively selected from the first text data and the first set of prompt words output by the second large language model in the adjusted second language modeling parameters.

4. The method according to claim 3, characterized in that, The loss is determined based on the first language modeling parameters, the second language modeling parameters, the nearest neighbor words of the second prompt word set in the first large language model, and the second prompt word set.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: When the representation space of the first large language model and the representation space of the third large language model are the same, the first prompt word set and the first text data are input into the third large language model to train the third large language model.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: When the representation space of the first large language model and the representation space of the third large language model are different: The first set of prompt words is decoded into natural language text based on the first large language model; The natural language text is encoded into a third set of prompt words according to the third language model. The third set of prompt words and the first text data are input into the third language model to train the third language model.

7. The method according to any one of claims 1 to 6, characterized in that, The first text data is obtained by selecting S text data from the F text data in the NLP training dataset based on representative sampling or sampling based on the neural network model.

8. The method according to claim 7, characterized in that, The value of S / F ranges from 0.02 to 0.

1.

9. The method according to any one of claims 1 to 8, characterized in that, The first language modeling parameters include the parameter trajectory of the expert parameters of the first large language model.

10. A data processing apparatus, characterized in that, It includes a processing module for performing the method as described in any one of claims 1 to 9.

11. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1 to 9.

12. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster performs the method as described in any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a cluster of computing devices, perform the method as described in any one of claims 1 to 9.

Images