Model training data generation method and apparatus, and computing device cluster

Abstract

The model training data generation method comprises the steps that first data input by a user is obtained, and the first data is lexical element distribution or text related to a scene to be reasoned by the user; on the basis of distribution of lexical elements in a corpus of the basic model in the first data, a lexical element pool is constructed, the lexical element pool contains the lexical elements in the corpus, the number of each lexical element in the lexical element pool is larger than or equal to the preset number, and the basic model is related to the reasoning scene; training data is obtained on the basis of the lexical unit pool and the basic model, the training data comprises samples and labels, the samples are the hidden states of the n + 1 lexical units and the first n lexical units in the n + 1 lexical units, the labels are the hidden state of the (n + 1) th lexical unit in the n + 1 lexical units, the n + 1 lexical units are selected from the lexical unit pool, and the hidden states of the n + 1 lexical units are obtained by reasoning the n + 1 lexical units through the basic model. According to the method, the problem of privacy leakage possibly caused by using real corpora can be effectively avoided, and meanwhile, the problem that the training data volume is too small due to the fact that the real corpora is too small can be avoided.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a method, device and computing device cluster for generating model training data. Background Art

[0002] With the advancement of AI technology, large language models (LLMs) are widely used in various fields, but their text generation process is expensive and slow. This inefficiency is attributed to the operation rules of autoregressive decoding. The generation of each token requires a forward propagation, which requires access to LLMs with billions to hundreds of billions of parameters, resulting in the slow speed of traditional autoregressive decoding.

[0003] To solve this problem, some institutions jointly released the extrapolation algorithm for greater language-model efficiency (Eagle) technology, which aims to improve the reasoning speed of LLM while ensuring the distribution consistency of the model output text. The neural network model built based on Eagle technology (referred to as "Eagle model") can significantly improve the generation efficiency by extrapolating the hidden state of the LLM hidden layer output by EAGLE.

[0004] However, when training the Eagle model, if public datasets such as share-gpt are used as corpora, the trained model may not perform well because it does not match the model usage scenario. If a corpus corresponding to the scenario is used, when there is not enough corpus for the model usage scenario, the trained model may have problems such as model instability, poor generalization, and overfitting. In addition, in some scenarios, although there is sufficient corpus, the direct use of relevant corpora may involve issues such as personal privacy, that is, the client is unwilling or unable to provide original corpus information involving personal privacy, which will also lead to insufficient training data and affect the model training effect. Therefore, how to construct a training dataset with sufficient sample size to ensure that the trained Eagle model has a good performance effect is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present application provides a model training data generation method, apparatus, computing device cluster, computer storage medium and computer product, which can construct a training data set with sufficient sample size to ensure that the trained Eagle model has a good performance effect.

[0006] In a first aspect, the present application provides a method for generating model training data, comprising: obtaining first data input by a user, wherein the first data is a token distribution or text related to a reasoning scenario required by the user; constructing a token pool based on the distribution of tokens in a corpus of a basic model in the first data, wherein the token pool contains tokens in the corpus, and the number of each token in the token pool is greater than or equal to a preset number, and the basic model is related to the reasoning scenario; obtaining training data based on the token pool and the basic model, wherein the training data includes samples and labels, the samples are n+1 tokens and hidden states of the first n tokens in the n+1 tokens, the labels are hidden states of the n+1th token in the n+1 tokens, the n+1 tokens are selected from the token pool, and the hidden states of the n+1 tokens are obtained by reasoning the n+1 tokens through the basic model.

[0007] In this way, by combining the small amount of data provided by the user and the corpus of the basic model, the distribution of the corpus tokens in the data provided by the user is statistically analyzed to construct a token pool, rather than using real corpus to generate model training samples. This can effectively avoid the privacy leakage problem that may be caused by using real corpus, and at the same time avoid the problem of too little training data due to the small size of the real corpus.

[0008] In a possible implementation, there are multiple training data, and the number of tokens contained in the samples in different training data is different. Since the input sequence is of variable length, the subsequent training data is also of variable length, so that the model trained by the training data can adapt to the reasoning of sequences of different lengths.

[0009] In one possible implementation, the sentences composed of tokens in the training data have no natural language meaning, so that the model trained later can be adapted to the basic model.

[0010] In a possible implementation, constructing a token pool based on the distribution of tokens in the corpus of the basic model in the first data includes: initializing the token pool based on the distribution of tokens in the corpus of the basic model in the first data, wherein, in the initialized token pool, the number of first tokens is the same as the number of first tokens contained in the first data, the number of second tokens is zero, the first token and the second token are both in the corpus, and the first data does not contain the second token; increasing the number of tokens contained in the initialized token pool by a given number to construct a token pool. In this way, a token pool can be constructed.

[0011] In a possible implementation, after obtaining the training data, the method further includes: training an extrapolation algorithm (Eagle) model for improving the efficiency of the language model based on the training data.

[0012] In a second aspect, the present application provides a model training data generation device, comprising: a communication module and a processing module. The communication module is used to obtain first data input by a user, wherein the first data is a word unit (token) distribution or text related to the reasoning scenario required by the user. The processing module is used to construct a token pool based on the distribution of tokens in the corpus of the basic model in the first data, wherein the token pool contains tokens in the corpus, and the number of each token in the token pool is greater than or equal to a preset number, and the basic model is related to the reasoning scenario. The processing module is also used to obtain training data based on the token pool and the basic model, wherein the training data includes samples and labels, the samples are n+1 tokens and the hidden states of the first n tokens in the n+1 tokens, the labels are the hidden states of the n+1th token in the n+1 tokens, the n+1 tokens are selected from the token pool, and the hidden states of the n+1 tokens are obtained by reasoning the n+1 tokens through the basic model.

[0013] In a possible implementation, there are multiple training data, and samples in different training data contain different numbers of tokens.

[0014] In one possible implementation, the sentences composed of tokens in the training data have no natural language meaning.

[0015] In one possible implementation, when the processing module constructs a token pool based on the distribution of tokens in the corpus of the basic model in the first data, it is specifically used to: initialize the token pool based on the distribution of tokens in the corpus of the basic model in the first data, wherein, in the initialized token pool, the number of first tokens is the same as the number of first tokens contained in the first data, the number of second tokens is zero, the first token and the second token are both located in the corpus, and the second token is not contained in the first data; increase the number of tokens contained in the initialized token pool by a given number to construct the token pool.

[0016] In a possible implementation, after obtaining the training data, the processing module is further used to: train the Eagle model based on the training data.

[0017] In a third aspect, the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium, including computer program instructions, when the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect. Exemplarily, the computing device cluster may include one or more computing devices.

[0019] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computing device cluster, enables the computing device cluster to perform the method described in the first aspect or any possible implementation of the first aspect. Exemplarily, the computing device cluster may include one or more computing devices.

[0020] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of a model training process provided in an embodiment of the present application;

[0022] Figure 2 It is a schematic diagram of a technical concept for generating model training data provided by an embodiment of the present application;

[0023] Figure 3 It is a flowchart of a method for generating model training data provided in an embodiment of the present application;

[0024] Figure 4 It is a schematic diagram of a process of generating model training data and training a model provided in an embodiment of the present application;

[0025] Figure 5 It is a structural schematic diagram of a model training data generating device provided in an embodiment of the present application;

[0026] Figure 6 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0027] Figure 7 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0028] Figure 8It is a structural diagram of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0030] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0031] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0032] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0033] First, the relevant technical terms involved in the technical solution provided in this application are introduced.

[0034] (1) Neural network model

[0035] The neural network model is a computing model inspired by the structure of the human brain. It processes information by imitating the connection and interaction of neurons in the human brain. In the neural network model, the network layer is the basic unit of the model, which is a hierarchical structure for processing data. Each network layer can be composed of multiple neurons, which are responsible for receiving input, performing calculations, and passing the results to the next layer. The network layers in the neural network model can include: input layer, hidden layer, and output layer. Among them, the input layer is the interface for the neural network model to receive external data. The hidden layer is the core part of the neural network model. It performs complex mathematical transformations and feature extraction on the data from the input layer to convert the input data into a more abstract and representative feature representation through the connection between multiple neurons and the action of the activation function. The main task of the output layer is to generate the final output of the neural network model based on the output results of the hidden layer.

[0036] (2) Speculative sampling

[0037] Speculative sampling is an algorithm for accelerating the decoding of large language models. It increases the decoding speed by processing and predicting multiple tokens in parallel. Specifically, it uses a smaller but faster draft model to predict the next few tokens, and then uses a larger but slower target model to determine the acceptance of these tokens.

[0038] (3) Base model

[0039] Base models refer to a class of large machine learning models that are trained on large-scale data (usually in a self-supervised or semi-supervised learning manner) to adapt to various downstream tasks. They carry prior knowledge of these large-scale data. Base models have helped to achieve major innovations in the way AI systems are built, such as providing support for chatbots and other user-oriented artificial intelligence. Base models can include some pre-trained language models, visual or multimodal base models, etc. In speculative sampling, base models are used to test the acceptance of tokens generated by the draft model.

[0040] (4) Input ids

[0041] Input identification refers to converting text data into a sequence of numbers that the model can process. In text data preprocessing, the text is first segmented, and then each word or subword is mapped to a unique integer ID. The sequence of these integer IDs is the input ids.

[0042] (5) Hidden States

[0043] Hidden states refer to the outputs of the hidden layers of the model. These hidden states contain contextual information accumulated from previous time steps and are key to the model's ability to remember and process sequence data.

[0044] (6) Extrapolation algorithm for greater language-model efficiency (Eagle)

[0045] Eagle is an extrapolation algorithm that improves the efficiency of language models. Its core idea is to predict and generate efficient language model outputs through extrapolation techniques, thereby speeding up the inference process and reducing computational costs.

[0046] (7) Eagle Model

[0047] The Eagle model is an innovative neural network model built around unique Eagle-related technologies and architectures. The model effectively improves the efficiency and performance of the model through carefully designed mechanisms, such as using Eagle-based extrapolation algorithms in specific layer structures. Its core is to extrapolate the contextual feature vectors of the hidden state output by the hidden layer of the base model, thereby significantly improving the generation efficiency, and experiments have shown that doing so will hardly lose model performance. In some embodiments, the Eagle model can be a single-layer model, that is, an autoregressive head. The Eagle model can predict the next hidden state, and then use the classification head of the original base model to predict the next word. The following is a detailed description of the Eagle model workflow:

[0048] a) Feature Collection: In the forward process of the base model, the Eagle model collects feature vectors from the output of the hidden layer of the base model.

[0049] b) Autoregressive head prediction: The EAGLE model contains a trained lightweight autoregressive head that works with the word embedding layer of the base model to predict the next feature vector from the hidden layer of the base model based on the current collected feature sequence.

[0050] c) Predict the next word using the classification head of the base model: The frozen classification head of the base model is then used to predict the next word.

[0051] (8) Autoregressive Head

[0052] The autoregressive head is a model structure that predicts the current data point by gradually leveraging previously generated parts when generating sequential data. This method is particularly effective when generating continuous data such as text, speech, or images, because it can capture and utilize the temporal dependencies in the sequence, thereby generating more coherent and accurate output.

[0053] Next, the technical solution provided by this application is introduced.

[0054] For example, Figure 1 A schematic diagram of a model training process provided by an embodiment of the present application is shown. Figure 1As shown, when training the Eagle model, first select at least one input sequence containing n+1 tokens from an existing natural language dataset (such as share-gpt, etc.). Then, map each token in the input sequence to a unique integer ID to obtain Input IDs. Next, input the Input IDs to the base model for inference to obtain the hidden state output by the hidden layer of the base model. Among them, a token can have a hidden state, so that n+1 hidden states can be obtained. After that, the input sequence and the hidden states related to the first n tokens in the input sequence are input to the Eagle model to output the predicted hidden state of the last token in the input sequence through the Eagle model. Finally, the hidden state of the last token in the input sequence output by the base model is used as the sample label, and the predicted hidden state output by the Eagle model and the corresponding sample label are calculated through the loss function to obtain the corresponding loss, and the parameters in the Eagle model are updated along the direction of minimizing the loss.

[0055] exist Figure 1 In the training, since the training dataset is an existing public dataset, it is easy for the trained Eagle model to perform poorly due to the mismatch with the model usage scenario.

[0056] Since the token generated by the Eagle model only needs to match its corresponding basic model, it does not need to have real natural language meaning. Therefore, the Eagle model training can be adapted to the basic model. Based on this concept, the present application proposes a method for randomly constructing input ids as input. For this input, the training target of the Eagle model only needs to be consistent with its basic model reasoning to achieve the effect of improving the hit rate of speculative sampling. Among them, before constructing the input ids, the input ids pool can be initialized and noise can be added according to the customer scenario and word frequency (which can be provided by the customer or customized according to the corresponding scenario statistics) so that the input ids pool can be used to construct training data. The following is a detailed introduction to this technical concept.

[0057] For example, Figure 2 FIG. 1 is a schematic diagram showing a technical concept of generating model training data provided by an embodiment of the present application. Figure 2 As shown in the figure, the model training data generation framework includes: basic data acquisition part, Input ids pool construction part and training data generation part.

[0058] The basic data acquisition part is mainly used to obtain the small-scale training text or token distribution provided by the user. That is to say, the user can provide a small amount of real-scenario text or the token distribution statistics completed by them. For example, when the user is unwilling to provide the complete real-scenario text, the user can independently count the token distribution contained in the relevant scenario text, such as: there are 100 "zhuan", 100 "li", 50 "shen", 50 "qing", and so on. In addition, in this part, the user can also provide the basic model on which the Eagle model to be trained depends, so as to obtain the corpus related to the basic model.

[0059] The Input ids pool construction part is mainly used to construct an Input ids pool based on the training text (or token distribution) provided by the user and the corpus on which the basic model depends. Specifically, in this part, an Input ids pool constructor can be used to calculate the proportion of each token in the corpus on which the basic model depends in the data provided by the user according to the user's input, so as to initialize the input ids pool. Among them, the corpus on which the basic model depends can represent the inference scenario required by the user. The initialized input ids pool contains each token in the corpus and the proportion of each token in the data provided by the user. Then, noise can be added to the initialized input ids pool, that is, for each token, a given number is added to the pool to ensure that each token in the pool exists and is not less than the specified quantity. For example, if there are tokens token 1, token 2,..., token 10 in the corpus, and there are 10 token 1 and 5 token 2 in the data provided by the user, then in the initialized input ids pool, the number of token 1 is 10, the number of token 2 is 5, and the number of token 3 to 10 is 0. And in the input ids pool after adding noise, the number of token 1 is 18, the number of token 2 is 13, and the number of token 3 to 10 is 8. In this way, an Input ids pool adapted to the inference scenario required by the user is constructed. Exemplarily, each id in the Input ids pool can be associated with a token. In addition, the Input ids pool can also be understood as a token pool.

[0060] The training data generation part mainly generates a training data set based on the constructed Input ids pool and the basic model. This training data set can be used to train the Eagle model that the user needs to train. Specifically, in this part, a pseudo prompt generator can be used to randomly extract indefinite length tokens from the input ids pool to form prompts to obtain multiple input sequences. Among them, the input sequence may not have natural language meaning. For example, when there are 2 input sequences, the first input sequence may contain 5 tokens and the second input sequence may contain 8 tokens. Since the input sequence is of indefinite length, the subsequent training data is also of indefinite length, so that the model trained by the training data can adapt to the reasoning of sequences of different lengths. Of course, it is also possible to randomly extract fixed-length tokens from the input ids pool to form prompts. The specific situation can be determined according to the actual situation and is not limited here.

[0061] Furthermore, the basic model can be used to perform an inference on each input sequence generated by the pseudo prompt generator to obtain the hidden state corresponding to each input sequence, thereby obtaining the Hidden-states required for training. The process of obtaining the hidden state corresponding to the input sequence can be referred to in the above Figure 1 Finally, the input sequence and the Hidden-states corresponding to the input sequence can be integrated into the samples required for training (i.e., the input of the Eagle model) and sample labels (i.e., targets) through the data integrator for model training. The process of obtaining samples and sample labels can be referred to in the previous Figure 1 The relevant descriptions in will not be repeated here one by one.

[0062] In the above process, the small amount of data provided by the user and the corpus of the basic model are combined to count the distribution of the corpus tokens in the data provided by the user to construct an input ids pool, rather than using real corpus to generate model training samples. This can effectively avoid the privacy leakage problem that may be caused by using real corpus, and at the same time avoid the problem of too little training data caused by too small a real corpus.

[0063] The specific implementation process of the above technical concept is described below. Among them, through the above technical concept, it is possible to use a small amount of real scene data as input and output a sufficient number of pre-training data. The amount of pre-training data may not be limited by the amount of real data, or it may be possible to output a sufficient number of pre-training data by only inputting the token distribution of the required scene.

[0064] For example, Figure 3A flow chart of a model training data generation method provided by an embodiment of the present application is shown. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Exemplarily, the method can be executed by a model training data generation device, wherein the device can be implemented by software and / or hardware, and can be configured in an electronic device or server, but is not limited to, typically, on a server. For ease of description, the following description is based on the server as the execution subject. Figure 3 As shown, the model training data generation method may include the following steps:

[0065] S301. Obtain first data input by a user, wherein the first data is a token distribution or text related to a reasoning scenario required by the user.

[0066] In this embodiment, the user can input the token distribution or text related to the reasoning scenario required by the user on the client supporting the server. After that, the server can obtain the first data input by the user. In addition, the user can also specify the basic model on which the model to be trained depends through the client.

[0067] S302. Based on the distribution of tokens in the corpus of the basic model in the first data, a token pool is constructed, wherein the token pool contains tokens in the corpus, and the number of each token in the token pool is greater than or equal to a preset number, and the basic model is related to the reasoning scenario.

[0068] In this embodiment, the server may count the distribution of tokens in the corpus of the basic model in the first data to initialize the token pool. Among them, the initialized token pool may include tokens in the corpus. In addition, when a token in the corpus is located in the first data, the number of the token in the initialized token pool may be the number of the token in the first data. When a token in the corpus is not located in the first data, the number of the token in the initialized token pool may be zero. Further, noise may be added to the initialized token pool, that is, the number of tokens contained in the initialized token pool is increased by a given number to construct a token pool. Exemplarily, the basic model may be related to the reasoning scenario. In this way, the constructed token pool is related to the reasoning scenario required by the user, so that the training data finally obtained can be adapted to the reasoning scenario. In some embodiments, the token pool may include all the tokens in the corpus, or may include a part of the tokens, which may be determined according to the actual situation and is not limited here.

[0069] S303. Based on the token pool and the basic model, training data is obtained, wherein the training data includes samples and labels, the samples are n+1 tokens and hidden states of the first n tokens in the n+1 tokens, the labels are the hidden states of the n+1th token in the n+1 tokens, the n+1 tokens are selected from the token pool, and the hidden states of the n+1 tokens are obtained by inferring the n+1 tokens through the basic model.

[0070] In this embodiment, after constructing a token pool, multiple tokens can be selected from it to form an input sequence, and the input sequence can be inferred through the basic model to obtain the hidden state of the input sequence. Furthermore, the input sequence and the hidden states of other tokens in the input sequence except the last token can be used as samples, and the hidden state of the last token in the input sequence can be used as a label to obtain a piece of training data. In some embodiments, the obtained training data can be multiple, and in this case, the number of tokens contained in the samples in different training data is different. In other words, a different number of tokens can be selected each time to form an input sequence, so that the subsequent training data is also of indefinite length. In addition, the sentences composed of the tokens in the training data may also have no natural language meaning.

[0071] In this way, by combining the small amount of data provided by the user and the corpus of the basic model, the distribution of the corpus tokens in the data provided by the user is statistically analyzed to construct a token pool, rather than using real corpus to generate model training samples. This can effectively avoid the privacy leakage problem that may be caused by using real corpus, and at the same time avoid the problem of too little training data due to the small size of the real corpus.

[0072] The following example introduces the process of generating data for model training.

[0073] For example, Figure 4As shown, the frequency of each token in the corpus in the data provided by the user is first counted. Then, an input ids pool is constructed based on the frequency and noise is added to the pool. Then, T input ids are randomly selected from the pool as input prompts (i.e., input sequences). Wherein, T is a random number. Next, a basic model inference is performed using the constructed prompt to construct a piece of data for training. Exemplarily, for any prompt of any construction, when the prompt contains n+1 tokens, a basic model is used to perform an inference on it, and the Hidden states f1, f2, ..., fn, fn+1 corresponding to token 1, token 2, ..., token n, token n+1 can be obtained. Then, f1, f2, ..., fn and the input ids corresponding to the input prompt can be used as samples of the model, and fn+1 can be used as a sample label to obtain a piece of training data. After repeating this operation many times, a training data set can be constructed. Wherein, a piece of training data can be obtained in each cycle. Finally, after generating enough training data, you can train the Eagle model.

[0074] It is understandable that the size of the sequence number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. In addition, the various embodiments described above can be combined according to actual conditions, and the combined scheme is still within the scope of protection of the present application. In addition, the Eagle model described above can also be replaced by other models, and the replaced scheme is still within the scope of protection of the present application.

[0075] Next, based on the method in the above embodiment, a model training data generating device provided in an embodiment of the present application is introduced.

[0076] For example, Figure 5 FIG. 1 is a schematic diagram showing the structure of a model training data generating device provided in an embodiment of the present application. Figure 5As shown, the model training data generating device 500 includes: a communication module 501 and a processing module 502. The communication module 501 is used to obtain the first data input by the user, wherein the first data is a token distribution or text related to the reasoning scenario required by the user. The processing module 502 is used to construct a token pool based on the distribution of tokens in the corpus of the basic model in the first data, wherein the token pool contains tokens in the corpus, and the number of each token in the token pool is greater than or equal to the preset number, and the basic model is related to the reasoning scenario. The processing module 502 is also used to obtain training data based on the token pool and the basic model, wherein the training data includes samples and labels, the samples are n+1 tokens and the hidden states of the first n tokens in the n+1 tokens, the labels are the hidden states of the n+1th token in the n+1 tokens, the n+1 tokens are selected from the token pool, and the hidden states of the n+1 tokens are obtained by reasoning the n+1 tokens through the basic model.

[0077] In some embodiments, there are multiple training data, and the number of tokens contained in samples in different training data is different.

[0078] In some embodiments, the sentences composed of tokens in the training data have no natural language meaning.

[0079] In some embodiments, when constructing a token pool based on the distribution of tokens in the corpus of the basic model in the first data, the processing module 502 is specifically used to: initialize the token pool based on the distribution of tokens in the corpus of the basic model in the first data, wherein, in the initialized token pool, the number of first tokens is the same as the number of first tokens contained in the first data, the number of second tokens is zero, both the first token and the second token are located in the corpus, and the second token is not contained in the first data; increase the number of tokens contained in the initialized token pool by a given number to construct the token pool.

[0080] In some embodiments, after obtaining the training data, the processing module 502 is further used to: train the Eagle model based on the training data.

[0081] In some embodiments, Figure 5 The communication module 501 and the processing module 502 shown in the figure can be implemented by software or by hardware. For example, the implementation of the communication module 501 is introduced below by taking the communication module 501 as an example. Similarly, the implementation of the processing module 502 can refer to the implementation of the communication module 501.

[0082] As an example of a software functional unit, the communication module 501 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the communication module 501 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0083] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0084] As an example of a hardware functional unit, the communication module 501 may include at least one computing device, such as a server, etc. Alternatively, the communication module 501 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0085] The multiple computing devices included in the communication module 501 can be distributed in the same region or in different regions. The multiple computing devices included in the communication module 501 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the communication module 501 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0086] It should be noted that, in other embodiments, the communication module 501 can be used to execute any step in the method for generating model training data described in the above embodiment, and the processing module 502 can also be used to execute any step in the method for generating model training data described in the above embodiment. In addition, the steps that the communication module 501 and the processing module 502 are responsible for implementing can also be specified as needed, and the communication module 501 and the processing module 502 can respectively implement different steps in the method for generating model training data described in the above embodiment to achieve Figure 5 The entire functions of the model training data generating device 500 are shown.

[0087] The present application also provides a computing device 600. Figure 6 As shown, computing device 600 includes: bus 602, processor 604, memory 606 and communication interface 608. Processor 604, memory 606 and communication interface 608 communicate through bus 602. Computing device 600 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 600.

[0088] The bus 602 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The bus 604 is represented by only one line, but does not mean that there is only one bus or one type of bus. The bus 604 may include a path for transmitting information between various components of the computing device 600 (eg, the memory 606, the processor 604, and the communication interface 608).

[0089] The processor 604 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0090] The memory 606 may include a volatile memory, such as a random access memory (RAM). The processor 604 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0091] The memory 606 stores executable program codes, and the processor 604 executes the executable program codes to respectively implement the aforementioned Figure 5 The functions of the communication module 501 and the processing module 502 shown in the embodiment are implemented to realize the model training data generation method described in the above embodiment. That is, the memory 606 stores instructions for executing the model training data generation method described in the above embodiment.

[0092] Alternatively, the memory 606 stores executable codes, and the processor 604 executes the executable codes to respectively implement the aforementioned Figure 5 The functions of the model training data generating device 500 shown in the figure are implemented to realize the model training data generating method described in the above embodiment. That is, the memory 606 stores instructions for executing the model training data generating method described in the above embodiment.

[0093] The communication interface 608 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 600 and other devices or communication networks.

[0094] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0095] like Figure 7As shown, the computing device cluster includes at least one computing device 600. The memory 606 in one or more computing devices 600 in the computing device cluster may store the same instructions for executing the model training data generation method described in the above embodiment.

[0096] In some possible implementations, the memory 606 of one or more computing devices 600 in the computing device cluster may also respectively store some instructions for executing the model training data generation method described in the above embodiment. In other words, the combination of one or more computing devices 600 can jointly execute the instructions for executing the model training data generation method described in the above embodiment.

[0097] It should be noted that the memory 606 in different computing devices 600 in the computing device cluster may store different instructions, which are respectively used to execute the aforementioned Figure 5 Part of the functions of the model training data generating device 500 shown. That is, the instructions stored in the memory 606 in different computing devices 600 can implement the functions of one or more modules in the communication module 501 and the processing module 502.

[0098] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Figure 8 A possible implementation is shown. Figure 8 As shown, two computing devices 600A and 600B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 606 in the computing device 600A stores instructions for executing the functions of the communication module 501. At the same time, the memory 606 in the computing device 600B stores instructions for executing the functions of the processing module 502.

[0099] It should be understood that Figure 8 The functions of the computing device 600A shown in FIG. 6A may also be completed by multiple computing devices 600. Similarly, the functions of the computing device 600B may also be completed by multiple computing devices 600.

[0100] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Figure 7 and Figure 8 The connection mode of the computing device cluster is different in that the memory 606 in one or more computing devices 600 in the computing device cluster may store the same instructions for executing the method in the above embodiment.

[0101] In some possible implementations, the memory 606 of one or more computing devices 600 in the computing device cluster may also respectively store some instructions for executing the aforementioned model training data generation method. In other words, the combination of one or more computing devices 600 may jointly execute instructions for executing the aforementioned model training data generation method.

[0102] It should be understood that each step of the above method embodiment can be completed by a hardware-based logic circuit or a software-based instruction in a processor.

[0103] Based on the method in the above embodiment, the embodiment of the present application provides a computer-readable storage medium, including computer program instructions, when the computer program instructions are executed by a computing device cluster including at least one computing device, the computing device cluster executes the method in the above embodiment. Exemplarily, the computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center including one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0104] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product including instructions. When the instructions are executed by a computing device cluster including at least one computing device, the computing device cluster executes the method in the above embodiment.

[0105] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0106] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0107] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0108] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating model training data, characterized in that: The method comprises: Acquire first data input by a user, wherein the first data is a token distribution or text related to a reasoning scenario required by the user; Based on the distribution of tokens in the corpus of the basic model in the first data, a token pool is constructed, wherein the token pool contains tokens in the corpus, and the number of each token in the token pool is greater than or equal to a preset number, and the basic model is related to the reasoning scenario; Based on the token pool and the basic model, training data is obtained, wherein the training data includes samples and labels, the samples are n+1 tokens and hidden states of the first n tokens of the n+1 tokens, the labels are hidden states of the n+1th token of the n+1 tokens, the n+1 tokens are selected from the token pool, and the hidden states of the n+1 tokens are obtained by inferring the n+1 tokens through the basic model.

2. The method according to claim 1, characterized in that: There are multiple training data, and the number of tokens contained in samples in different training data is different.

3. The method according to claim 1 or 2, characterized in that: The sentences composed of tokens in the training data have no natural language meaning.

4. The method according to any one of claims 1 to 3, characterized in that: The distribution of tokens in the corpus based on the basic model in the first data is used to construct a token pool, including: Initializing the token pool based on the distribution of tokens in the corpus of the basic model in the first data, wherein, in the initialized token pool, the number of first tokens is the same as the number of first tokens contained in the first data, the number of second tokens is zero, the first token and the second token are both located in the corpus, and the first data does not contain the second token; The number of tokens contained in the initialized token pool is increased by a given number to construct the token pool.

5. The method according to any one of claims 1 to 4, characterized in that: After getting the training data, it also includes: Based on the training data, an extrapolation algorithm (Eagle) model for improving the efficiency of the language model is trained.

6. A model training data generating device, characterized in that: include: A communication module, configured to obtain first data input by a user, wherein the first data is a token distribution or text related to a reasoning scenario required by the user; A processing module, configured to construct a token pool based on the distribution of tokens in the corpus of the basic model in the first data, wherein the token pool includes tokens in the corpus, and the number of each token in the token pool is greater than or equal to a preset number, and the basic model is related to the reasoning scenario; The processing module is also used to obtain training data based on the token pool and the basic model, wherein the training data includes samples and labels, the samples are n+1 tokens and hidden states of the first n tokens among the n+1 tokens, the labels are hidden states of the n+1th token among the n+1 tokens, the n+1 tokens are selected from the token pool, and the hidden states of the n+1 tokens are obtained by inferring the n+1 tokens through the basic model.

7. The device according to claim 6, characterized in that There are multiple training data, and the number of tokens contained in samples in different training data is different.

8. The device according to claim 6 or 7, characterized in that The sentences composed of tokens in the training data have no natural language meaning.

9. The device according to any one of claims 6 to 8, characterized in that: When the processing module constructs a token pool based on the distribution of tokens in the corpus of the basic model in the first data, it is specifically used to: Initializing the token pool based on the distribution of tokens in the corpus of the basic model in the first data, wherein, in the initialized token pool, the number of first tokens is the same as the number of first tokens contained in the first data, the number of second tokens is zero, the first token and the second token are both located in the corpus, and the first data does not contain the second token; The number of tokens contained in the initialized token pool is increased by a given number to construct the token pool.

10. The device according to any one of claims 6 to 9, characterized in that: After obtaining the training data, the processing module is further used to: Based on the training data, an extrapolation algorithm (Eagle) model for improving the efficiency of the language model is trained.

11. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1-5.

12. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the instructions are executed by a computing device cluster, the computing device cluster enables the computing device cluster to perform the method according to any one of claims 1 to 5, wherein the computing device cluster comprises at least one computing device.

13. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 5, wherein the computing device cluster includes at least one computing device.

Images