Data prediction method and device, electronic equipment and storage medium

Abstract

The invention discloses a data prediction method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a data sequence generated by a large model, and the data sequence comprises a plurality of lexical elements in a word list; determining a target sub-weight matrix from a plurality of sub-weight matrixes corresponding to the vocabulary according to the data sequence, wherein the sub-weight matrixes are obtained by grouping weight matrixes corresponding to the vocabulary; and performing prediction according to the data sequence and the target sub-weight matrix to obtain the next lexical element of the data sequence.

Description

Technical Field

[0001] This invention relates to artificial intelligence technology, and more particularly to a device, an electronic device, and a storage medium. Background Technology

[0002] Large Language Models (LLMs) have excellent natural language processing capabilities, but they have a large number of parameters and a relatively slow generation speed. Therefore, how to improve the generation speed of LLMs is an urgent problem to be solved. Summary of the Invention

[0003] This disclosure provides a data prediction method, including:

[0004] Obtain the data sequence generated by the large model, the data sequence including multiple lexical units in the vocabulary;

[0005] The target sub-weight matrix is ​​determined from multiple sub-weight matrices corresponding to the vocabulary based on the data sequence, wherein the sub-weight matrix is ​​obtained by grouping the weight matrices corresponding to the vocabulary.

[0006] The next word in the data sequence is predicted based on the data sequence and the target sub-weight matrix.

[0007] The process of determining the target sub-weight matrix from multiple sub-weight matrices corresponding to the vocabulary based on the data sequence includes:

[0008] The data sequence is feature-encoded according to the first feature space dimension to obtain a feature vector;

[0009] Determine the correlation between the feature vector and each of the sub-weight matrices;

[0010] The relevance is normalized to obtain the probability of each sub-weight matrix;

[0011] Based on the probability, at least one sub-weight matrix is ​​determined as the target sub-weight matrix.

[0012] Determining the correlation between the feature vector and each of the sub-weight matrices includes:

[0013] Based on the first relation matrix of the first feature space dimension and the second feature space dimension, the feature vector is transformed to obtain the first intermediate feature vector;

[0014] The first intermediate feature vector is enhanced to obtain the second intermediate feature vector;

[0015] Based on the second relation matrix between the second feature space dimension and each sub-weight matrix, the second intermediate feature vector is transformed to obtain the correlation between the feature vector and each sub-weight matrix.

[0016] The first feature space dimension is constructed based on the task objective of predicting lexical units in the large model, and the second feature space dimension is constructed based on the task objective of determining the target sub-weight matrix.

[0017] The process of predicting the next word element of the data sequence based on the data sequence and the target sub-weight matrix includes:

[0018] The feature vector is processed according to the target sub-weight matrix to obtain the generation probability of each candidate word element corresponding to the target sub-weight matrix;

[0019] The fusion probability of the candidate word is determined based on the probability of the target sub-weight matrix and the generation probability of the candidate word.

[0020] The next word in the data sequence is obtained by sampling based on the fusion probability of all candidate words.

[0021] The method further includes:

[0022] The accuracy of the next term is verified by the large model, so that the large model adds the next term to the data sequence when the verification is successful.

[0023] The method further includes:

[0024] The accuracy of the target sub-weight matrix is ​​verified using the large model. If the verification fails, the first relation matrix and the second relation matrix are optimized.

[0025] This disclosure also provides a data prediction apparatus, including:

[0026] The acquisition module is used to acquire the data sequence generated by the large model, the data sequence including multiple lexical units in the vocabulary;

[0027] The routing module is used to determine a target sub-weight matrix from multiple sub-weight matrices corresponding to the vocabulary based on the data sequence, wherein the sub-weight matrix is ​​obtained by grouping the weight matrices corresponding to the vocabulary;

[0028] The prediction module is used to predict the next word in the data sequence based on the data sequence and the target sub-weight matrix.

[0029] This disclosure also provides an electronic device, including at least one processor and at least one processing model capable of running on the processor, the processing model being invoked by a target application to execute the data prediction method described in this disclosure.

[0030] This disclosure also provides a computer-readable storage medium storing a computer program for executing the data prediction method described in this disclosure. Attached Figure Description

[0031] Figure 1 A schematic flowchart of a data prediction method according to an embodiment of the present disclosure is shown;

[0032] Figure 2 A schematic diagram of the structure for determining the correlation of sub-weight matrices according to an embodiment of this disclosure is shown;

[0033] Figure 3 A flowchart illustrating the process of determining the correlation of sub-weight matrices according to an embodiment of this disclosure is shown;

[0034] Figure 4 A schematic diagram of the composition structure of a data prediction device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0035] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0036] To explain this scheme more clearly, we will first explain speculative decoding.

[0037] The generation of large language models (LLMs) is divided into two stages. The first stage is the pre-filling stage, where the user's query is sent into the LLM until the first token is generated. The second stage is the auto-regressive generation stage, in which the LLM generates tokens one by one based on the previously generated content. Speculative decoding is an acceleration technique for the auto-regressive generation stage of LLMs. Its core idea is to reduce the number of calls to the large model through a loop process of small model prediction + large model (LLM) verification (drafting-verification), thereby improving the generation efficiency. Specifically, the small model predicts the tokens that the large model may generate next, and then the large model verifies the guessed results. If the guessed result is correct, the large model retains the token; if the guessed result is incorrect, the large model discards the token and generates a new token.

[0038] In the above speculative decoding scheme, the LM head used by the small model to predict tokens is the same as that of the large model. If the vocabulary used by the large model contains a large number of tokens, it will result in a very large number of parameters used by the small model when predicting tokens using the LM head, directly affecting the prediction speed of the small model and unable to achieve the purpose of fast generation.

[0039] Therefore, in order to improve the speed of the small model's token prediction, this application provides a data prediction method, which is applied to the small model, such as Figure 1 shown, this method includes:

[0040] Operation 101, obtaining the data sequence generated by the large model, where the data sequence includes multiple tokens in the vocabulary.

[0041] The data sequence here refers to the sequence composed of the tokens that the large model has generated in the auto-regressive generation stage. This data sequence can be generated by the large model alone or through small model prediction + large model verification. The present disclosure does not limit this.

[0042] The role of the data sequence is to serve as the context for the small model to predict the next token. For example, the generated data sequence is {I, today, ate,}, which includes 4 consecutive tokens. The small model predicts the next token based on this data sequence.

[0043] The prediction of tokens needs to be based on the vocabulary. Based on the generated data sequence, calculate the probability of each token in the vocabulary as the next token, and sample from it to determine the next token. Therefore, the tokens in the data sequence all come from the vocabulary.

[0044] Operation 102: Determine the target sub-weight matrix from multiple sub-weight matrices corresponding to the vocabulary based on the data sequence. The sub-weight matrix is ​​obtained by grouping the weight matrices corresponding to the vocabulary.

[0045] In the word prediction process, the probability of each word in the vocabulary becoming the next word is calculated based on the generated data sequence. In essence, this involves performing matrix multiplication between the feature vector obtained by preprocessing the data sequence and the weight matrix (i.e., LMhead) corresponding to the vocabulary, and finally obtaining the probability distribution of words in the vocabulary.

[0046] Small models need to read the entire LM head from memory for each prediction. However, when the vocabulary is large (e.g., Llama3 (an open-source large model) has a vocabulary of 128,256 tokens, and its LM head size is 4096 × 128,256), the amount of data in the LM head becomes extremely large. Each time a small model reads the entire LM head, it consumes a significant amount of bandwidth, significantly increasing the time required and directly limiting the prediction speed, thus affecting the overall acceleration effect of speculative decoding.

[0047] To avoid the significant increase in time consumption caused by the small model reading the entire LM head every time, in this example, the LM head is divided into multiple sub-weight matrices. The target sub-weight matrix is ​​selected from these sub-weight matrices for the prediction of the next word, thereby improving prediction speed. In one example, the small and large models can share the same LM head. The LM head of the large model can be directly grouped. When the small model predicts a word, it only reads the target sub-weight matrix from it, while the large model still needs to read the entire LM head when using it. In another example, a separate LM head (the same as the large model) can be saved for the small model and grouped.

[0048] In one example, the LM head can be divided according to a set number, segmenting it along the vocabulary dimension. For instance, taking Llama3 as an example, its vocabulary size is 128256, and the LM head size is 4096×128256. This can be divided into 16 groups, with each group's sub-weight matrix size being 4096×8016 (128256÷16=8016). Each sub-weight matrix then corresponds to 8016 consecutive words in the vocabulary: the first sub-weight matrix corresponds to words from the 1st to the 8016th word, the second sub-weight matrix corresponds to words from the 8017th to the 16022nd word, and so on, until the 16th sub-weight matrix covers the remaining 8016 words (words from the 120241st to the 128256th word).

[0049] Operation 103: Predict the next word in the data sequence based on the data sequence and the target sub-weight matrix.

[0050] Based on the determined target sub-weight matrix, the feature vector of the data sequence after preprocessing can be multiplied with the target sub-weight matrix to obtain the probability distribution of the word corresponding to the target sub-weight matrix. Based on this, sampling can be performed to determine the next word.

[0051] In the above scheme, predicting the next lexical term based on the data sequence already generated by the large model ensures that the prediction of the small model is based on the same lexical logic and semantic coherence as the large model. By grouping the LM head and determining the target sub-weight matrix based on the data sequence, the memory read volume of the small model is reduced from the complete LM head to the target sub-weight matrix. For example, in the example above, the amount of data read in a single run in Llama3 is only 1 / 16 of the original. The significant reduction in memory read volume directly reduces bandwidth usage and data transmission time, improving the prediction speed of the small model. In addition, the small model only uses the target sub-weight matrix for probability calculation. The computational cost of matrix multiplication is proportional to the number of columns in the weight matrix (vocabulary dimension). In the example above, the computational cost of the target sub-weight matrix is ​​only 1 / 16 of the complete LM head. The reduction in computational complexity further shortens the time for a single prediction.

[0052] In one example, the implementation of determining the target sub-weight matrix from multiple sub-weight matrices corresponding to the vocabulary based on the data sequence includes:

[0053] Operation 201: Encode the data sequence according to the first feature space dimension to obtain a feature vector.

[0054] The role of the small model is to predict the output of the large model (the next word). The probability calculation of the large model's prediction of words depends on its hidden layer features (i.e., feature vectors based on its own hidden layer dimensions). By using the hidden layer dimension of the large model (i.e., the first feature space dimension) to encode the data sequence, the encoding result of the small model (i.e., the feature vector obtained by feature encoding the data sequence according to the first feature space dimension) can accurately capture the contextual semantics of the data sequence.

[0055] The first feature space dimension is constructed based on the task objective of predicting words using a large model. For example, in the example above, the vocabulary size of Llama3 is 128256, and the LM head size is 4096×128256, where 4096 is the hidden layer dimension of Llama3. After the data sequence is encoded, each word is encoded as a 4096-dimensional feature vector.

[0056] For example, if the generated sequence is {I, today, ate}, which contains 4 tokens, then each token is mapped to the first feature space dimension (4096 dimensions), obtaining a 4096-dimensional feature vector for the token. The feature encoding results in a 4×4096 feature matrix.

[0057] It should be noted that the feature encoding process can aggregate the semantic features of the previous tokens in the data sequence into the features of the last token (e.g., "ate"), that is, the feature vector corresponding to the last token already contains the accumulated context semantics of the entire data sequence. Therefore, the feature vector of the last token can be used for subsequent calculations.

[0058] Operation 202: Determine the relevance between the feature vector and each sub-weight matrix.

[0059] The higher the relevance between the sub-weight matrix and the feature vector (e.g., the feature vector of the last token in the data sequence), the greater the likelihood that the sub-weight matrix contains the next token. The determination process of this relevance will be illustrated through subsequent examples and will not be elaborated here.

[0060] Operation 203: Normalize the relevance to obtain the probability of each sub-weight matrix.

[0061] Since the numerical ranges of the relevance determined in Operation 202 may vary greatly and cannot directly reflect the importance of the sub-weight matrix, normalization (such as the softmax function) is required. All relevances of the sub-weight matrices can be converted into a probability distribution with a sum of 1 (e.g., the sum of the probabilities of 16 sub-weight matrices is 1), making the probability value of each sub-weight matrix fall within the interval [0,1], intuitively reflecting its association with the current data sequence.

[0062] Operation 204: Determine at least one sub-weight matrix as the target sub-weight matrix according to the probability.

[0063] Based on the probability distribution of the above sub-weight matrices, sampling is performed to determine at least one target sub-weight matrix. The sampling methods here are, for example:

[0064] Top-1 sampling: Directly select the 1 group of sub-weight matrices with the highest probability as the target sub-weight matrix;

[0065] Threshold sampling: Set a probability threshold and select all sub-weight matrices with probabilities exceeding this threshold as the target sub-weight matrices;

[0066] This is only an example. Sampling methods such as random sampling and Top-k sampling can also be used, and the present disclosure does not limit this.

[0067] The above operation 202 for determining the relevance of the feature vector to each sub-weight matrix can be implemented by a router (routing module), which includes a Linear layer (linear layer), an activation function, and a classifier. The implementation process is as Figure 2 and Figure 3 shown, including:

[0068] Operation 2021, according to the first relationship matrix of the first feature space dimension and the second feature space dimension, transforms the feature vector to obtain a first intermediate feature vector.

[0069] This operation is implemented by the Linear layer. A first relationship matrix is configured in the Linear layer, whose row dimension is the same as the dimension of the input feature vector. For example, if the input feature vector is a 4096-dimensional feature vector of the last token "le" in the data sequence, the row dimension of the first relationship matrix is also 4096 (i.e., the first feature space dimension); its column dimension can be set as needed, and its function is to map the input feature vector to a new feature space, that is, the second feature space. Therefore, the column dimension is the second feature space dimension.

[0070] Since the original 4096-dimensional (first feature space dimension) feature vector is generated by the small model based on the first feature space of the large model, and its design goal is to serve for predicting the next token (consistent with the task of the large model). However, the task of the router here has changed to predicting which of the 16 sub-weight matrices is most relevant to the current data sequence, and the prediction target has changed from the token to the sub-weight matrix. Therefore, a new second feature space needs to be constructed to adapt to this task. The second feature space dimension is preset (such as 4096), and its design does not depend on the original feature dimension of the large model, and only needs to satisfy the ability to effectively distinguish the relevance of multiple sub-weight matrices (that is, the second feature space dimension is constructed based on the task goal of determining the target sub-weight matrix). Therefore, it can also be adjusted to other dimensions according to actual needs (such as 8000). The present disclosure does not limit the setting of the second feature space dimension.

[0071] The Linear layer performs a linear transformation through the first relationship matrix, which can map the input feature vector to the second feature space to obtain a first intermediate feature vector.

[0072] Assuming that the second feature space dimension is 8000, the 4096-dimensional feature vector of the last token "le" in the data sequence can be transformed into an 8000-dimensional feature vector (the first intermediate feature vector).

[0073] Operation 2022, perform feature enhancement on the first intermediate feature vector to obtain a second intermediate feature vector.

[0074] This operation can be achieved through an activation function (such as SiLU), which can perform a non-linear transformation on the first intermediate feature vector of the input. For example, for components with larger values ​​in the first intermediate feature vector (i.e., features with higher correlation to the sub-weight matrix), the activation function will amplify their influence and enhance the expression of effective features; for components with smaller values ​​(i.e., features with lower correlation), the activation function will suppress them and reduce the interference of redundant information.

[0075] After feature enhancement by the activation function, the output second intermediate feature vector still maintains the second feature space dimension (e.g., 8000 dimensions), but the semantic discriminativeness of the features is significantly improved, focusing more on key information related to the current data sequence context and weakening irrelevant features.

[0076] Operation 2023: Based on the second relation matrix between the second feature space dimension and each sub-weight matrix, the second intermediate feature vector is transformed to obtain the correlation between the feature vector and each sub-weight matrix.

[0077] This operation is implemented by a classifier, which configures a second relation matrix. Its row dimension is the same as the second feature space dimension (e.g., if the second feature space dimension is set to 8000 in operation 2021, then the row dimension of the second relation matrix is ​​8000). Its column dimension is consistent with the number of sub-weight matrices (e.g., 16 columns for 16 sub-weight matrices), with each column corresponding to the correlation calculation parameter of one sub-weight matrix.

[0078] Matrix multiplication is performed between the second intermediate feature vector and the second relation matrix to transform the second intermediate feature vector. Essentially, this maps the feature vector in the second feature space to the correlation space of the sub-weight matrix. If the second feature space has a dimension of 8000 and the number of sub-weight matrices is 16, then the second relation matrix is ​​an 8000×16 matrix. The element in the i-th row and j-th column of the matrix represents the correlation weight between the i-th dimension of the second intermediate feature vector and the j-th sub-weight matrix (the larger the value, the more significant the influence of that dimension on the correlation of the j-th sub-matrix). In this example, the classifier output is a 16-dimensional vector, where the value of the j-th element is the correlation between the feature vector of the data sequence and the j-th sub-weight matrix.

[0079] In one example, the router may also include a normalization module and a sampling module. The normalization module is used to implement the above operation 203, normalizing the relevance of the sub-weight matrix to obtain the probability of the sub-weight matrix. The sampling module is used to implement the above operation 204, sampling based on the probability distribution of the sub-weight matrix to determine at least one target sub-weight matrix.

[0080] In one example, the router can be obtained through training. During the training process, the large model can be used to verify the accuracy of the target sub-weight matrix determined by the router. In one example, for a set of training data (including multiple data sequences), the large model can be used to directly predict the target sub-weight matrix corresponding to each data sequence in the training data as the label of the data sequence in the training data. During training, the prediction result of the small model is compared with the label to determine the accuracy of the prediction result of this set of training data. If the accuracy meets the standard, the verification passes; if the accuracy does not meet the standard, the verification fails, and the router can be optimized. In another example, if the next token predicted based on the target sub-weight matrix is verified as successful by the large model, it can be determined that the target sub-weight matrix is correct. If it is verified as incorrect by the large model, it can be determined that the target sub-weight matrix is incorrect. Correspondingly, for a set of training data (including multiple data sequences), if the accuracy rate (i.e., accuracy) of the target sub-weight matrix meets the standard, the verification passes; if the accuracy rate does not meet the standard, the verification fails, and the router can be optimized. The verification method here is only an example, and the present application does not limit the verification method of the accuracy rate.

[0081] In this example, optimizing the router means optimizing the parameters in the first relationship matrix and the second relationship matrix. After optimization, training is carried out again until the accuracy rate meets the standard, and the router training is completed.

[0082] In one example, operation 103 predicts the next token of the data sequence according to the data sequence and the target sub-weight matrix, including:

[0083] Processing the feature vector according to the target sub-weight matrix to obtain the generation probability of each candidate token corresponding to the target sub-weight matrix;

[0084] Determining the fusion probability of the candidate tokens according to the probability of the target sub-weight matrix and the generation probability of the candidate tokens;

[0085] Sampling based on the fusion probabilities of all candidate tokens to obtain the next token of the data sequence.

[0086] The target sub-weight matrix is a specific sub-weight matrix selected from the LM head grouping. Taking the dimension of the target sub-weight matrix as 4096×8016, corresponding to 8016 candidate tokens as an example. The small model performs a matrix multiplication operation on the feature vector (4096-dimensional) of the last token "le" of the data sequence and the target sub-weight matrix to obtain the original scores of the corresponding 8016 candidate tokens of the sub-weight matrix, and performs softmax normalization processing on the original scores to obtain the generation probability of each candidate token, indicating the possibility of each token as the next token within the range of this sub-weight matrix.

[0087] The fusion probability is a comprehensive consideration of the reliability of the target sub-weight matrix and the generation probability. The fusion probability = the probability of the target sub-weight matrix × the generation probability of the candidate token.

[0088] Among them, the probability of the target sub-weight matrix reflects the confidence that this sub-weight matrix contains the correct token. For example, if the probability of the target sub-matrix is 0.8 and the generation probability of the candidate token "rice" it contains is 0.3, then the fusion probability of "rice" is 0.8 × 0.3 = 0.24.

[0089] If multiple groups of target sub-weight matrices are determined, the fusion probabilities of the candidate tokens within each group of target sub-weight matrices need to be calculated separately, and finally, the candidate tokens and their fusion probabilities of all groups are summarized.

[0090] Normalize the fusion probabilities of all candidate tokens (ensure the sum is 1), and then use a sampling strategy (such as greedy sampling, TOP-k sampling, etc.) to select the next token.

[0091] The large model verifies the accuracy of this token. In one example, if the token predicted by the large model is the same as the token predicted by the small model, it is considered that the verification passes; otherwise, the verification fails. When the verification passes, the large model adds this token to the data sequence.

[0092] In one example, it is also possible to set the small model to predict multiple tokens at one time, and each token is predicted in the above manner. For example, predict 5 tokens at one time. After predicting the 1st token according to the generated data sequence and adding this token to the data sequence (at this time, the large model has not verified this token, so this data sequence is not the generated data sequence of the large model, but a temporary data sequence of the small model), predict the 2nd token according to this temporary data sequence, and so on until 5 tokens are predicted. The large model can verify these 5 tokens at one time, and this verification follows the parallel verification logic of speculative decoding. If the 1st token is the same as the prediction result of the large model, retain this token and continue to verify the 2nd; if the token at a certain position (such as the 3rd) is inconsistent, it is determined that all the predicted tokens after this position (the 3rd, 4th, and 5th) are invalid, only retain the first 2 consistent tokens and add them to the data sequence, and replace the token at the 3rd position with the token generated by the large model itself.

[0093] To implement the above data prediction method, as Figure 4 shown, the present application also provides a data prediction device, including:

[0094] An acquisition module 10, configured to acquire a data sequence generated by a large model, where the data sequence includes multiple tokens in a vocabulary;

[0095] The routing module 20 is used to determine a target sub-weight matrix from multiple sub-weight matrices corresponding to the vocabulary based on the data sequence, wherein the sub-weight matrix is ​​obtained by grouping the weight matrices corresponding to the vocabulary.

[0096] The prediction module 30 is used to predict the next word in the data sequence based on the data sequence and the target sub-weight matrix.

[0097] When determining the target sub-weight matrix from multiple sub-weight matrices corresponding to the vocabulary based on the data sequence, the routing module 20 is further configured to:

[0098] The data sequence is feature-encoded according to the first feature space dimension to obtain a feature vector;

[0099] Determine the correlation between the feature vector and each of the sub-weight matrices;

[0100] The relevance is normalized to obtain the probability of each sub-weight matrix;

[0101] Based on the probability, at least one sub-weight matrix is ​​determined as the target sub-weight matrix.

[0102] When determining the correlation between the feature vector and each of the sub-weight matrices, the routing module 20 is further configured to:

[0103] Based on the first relation matrix of the first feature space dimension and the second feature space dimension, the feature vector is transformed to obtain the first intermediate feature vector;

[0104] The first intermediate feature vector is enhanced to obtain the second intermediate feature vector;

[0105] Based on the second relation matrix between the second feature space dimension and each sub-weight matrix, the second intermediate feature vector is transformed to obtain the correlation between the feature vector and each sub-weight matrix.

[0106] In one example, the structure of routing module 20 can be found here. Figure 2 As shown.

[0107] The first feature space dimension is constructed based on the task objective of predicting lexical units in the large model, and the second feature space dimension is constructed based on the task objective of determining the target sub-weight matrix.

[0108] When predicting the next word in the data sequence based on the data sequence and the target sub-weight matrix, the prediction module 30 is further configured to:

[0109] The feature vector is processed according to the target sub-weight matrix to obtain the generation probability of each candidate word element corresponding to the target sub-weight matrix;

[0110] The fusion probability of the candidate word is determined based on the probability of the target sub-weight matrix and the generation probability of the candidate word.

[0111] The next word in the data sequence is obtained by sampling based on the fusion probability of all candidate words.

[0112] For example, this disclosure also provides an electronic device including at least one processor and at least one processing model capable of running on said processor, said processing model being invoked by a target application to execute the data prediction method described in embodiments of this disclosure.

[0113] By way of example, the present invention also provides a computer-readable storage medium storing a computer program for performing the above-described data prediction method.

[0114] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0115] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0116] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0117] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0118] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0119] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0120] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0121] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0122] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A data prediction method, comprising: obtaining a data sequence generated by a large model, the data sequence comprising a plurality of word pieces in a vocabulary; determining a target sub-weight matrix from a plurality of sub-weight matrices corresponding to the vocabulary according to the data sequence, the sub-weight matrices being obtained by grouping a weight matrix corresponding to the vocabulary; performing prediction according to the data sequence and the target sub-weight matrix to obtain a next word piece of the data sequence.

2. The method of claim 1, wherein determining the target sub-weight matrix from the plurality of sub-weight matrices corresponding to the vocabulary according to the data sequence comprises: performing feature encoding on the data sequence according to a first feature space dimension to obtain a feature vector; determining a correlation between the feature vector and each of the sub-weight matrices; normalizing the correlation to obtain a probability of each of the sub-weight matrices; and determining at least one of the sub-weight matrices as the target sub-weight matrix according to the probability.

3. The method of claim 2, wherein determining the correlation between the feature vector and each of the sub-weight matrices comprises: performing conversion on the feature vector according to a first relationship matrix between the first feature space dimension and a second feature space dimension to obtain a first intermediate feature vector; performing feature enhancement on the first intermediate feature vector to obtain a second intermediate feature vector; and performing conversion on the second intermediate feature vector according to a second relationship matrix between the second feature space dimension and each of the sub-weight matrices to obtain the correlation between the feature vector and each of the sub-weight matrices.

4. The method of claim 3, wherein the first feature space dimension is constructed based on a task objective of the large model for predicting word pieces, and the second feature space dimension is constructed based on a task objective of determining the target sub-weight matrix.

5. The method of claim 2, wherein performing prediction according to the data sequence and the target sub-weight matrix to obtain the next word piece of the data sequence comprises: processing the feature vector according to the target sub-weight matrix to obtain a generation probability of each candidate word piece corresponding to the target sub-weight matrix; determining a fusion probability of the candidate word piece according to the probability of the target sub-weight matrix and the generation probability of the candidate word piece; and sampling based on the fusion probabilities of all candidate word pieces to obtain the next word piece of the data sequence.

6. The method of claim 1, further comprising: performing accuracy verification on the next word piece by the large model, so that the large model adds the next word piece to the data sequence when the verification is passed.

7. The method of claim 3, further comprising: performing accuracy verification on the target sub-weight matrix by the large model, and optimizing the first relationship matrix and the second relationship matrix if the verification fails.

8. A data prediction device, comprising: an obtaining module configured to obtain a data sequence generated by a large model, the data sequence comprising a plurality of word pieces in a vocabulary; a routing module configured to determine a target sub-weight matrix from a plurality of sub-weight matrices corresponding to the vocabulary according to the data sequence, the sub-weight matrix being obtained by grouping the weight matrix corresponding to the vocabulary; a prediction module configured to predict a next word element of the data sequence according to the data sequence and the target sub-weight matrix. 9.An electronic device comprising at least one processor and at least one processing model capable of running on the processor, the processing model being capable of being invoked by a target application to perform the data prediction method of any one of claims 1-7. 10.A computer readable storage medium, the storage medium storing a computer program for performing the data prediction method of any one of claims 1-7.

Images