Text prediction method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a text prediction method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: generating a candidate word pair set after word pair extension based on an input text and a preset word pair set; determining a plurality of first prediction words in the candidate word pair set based on words in the input text, and generating a plurality of prediction texts based on the input text and the plurality of first prediction words; wherein the plurality of prediction texts comprise the input text and a plurality of texts obtained by splicing the input text and at least one first prediction word; predicting each prediction text in parallel through a large language model to obtain a plurality of second prediction words; wherein one second prediction word corresponds to one prediction text; and determining a target prediction word of the input text based on the plurality of first prediction words and the plurality of second prediction words. Through the method, the speed and efficiency of text prediction are improved.

Description

Technical Field

[0001] The present disclosure relates to the technical fields of artificial intelligence and natural language processing, and in particular to a text prediction method and device, an electronic device, a storage medium, and a program product. Background Art

[0002] With the rapid development of artificial intelligence technology, large language models (LLM) have shown strong capabilities in the field of natural language processing. Generally speaking, the reasoning process of a large language model is a process of iteratively decoding the input text to generate the answer text. For example, the large language model generates a predicted word (token) for each decoding step in sequence based on the input text, splices this predicted word to the predicted text of the previous decoding step, and inputs the spliced ​​predicted text into the large language model again to generate a predicted word for the next decoding step. Repeat the execution until the decoding termination condition is met to generate the answer text.

[0003] However, during the inference process of large language models, a huge amount of parameters need to be loaded into the computing unit of the Graphic Processing Unit (GPU) and migrated from the GPU high-bandwidth memory (HBM) to the cache in order to realize the operation of a predicted word in a decoding step. This leads to a very large memory access overhead and an extremely unbalanced computation-to-memory ratio during the inference process, resulting in very low GPU utilization and limiting the real-time and efficiency of large language model inference. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a text prediction method and device, an electronic device, a storage medium and a program product, which can improve the speed and efficiency of text prediction.

[0005] According to a first aspect of an embodiment of the present disclosure, a text prediction method is provided, comprising:

[0006] Based on the input text and the preset word pair set, generating a candidate word pair set after the word pair is expanded;

[0007] Based on the words in the input text, a plurality of first predicted words are determined in the candidate word pair set, and a plurality of predicted texts are generated based on the input text and the plurality of first predicted words; wherein the plurality of predicted texts include the input text and a plurality of texts obtained by concatenating the input text with at least one first predicted word;

[0008] Predicting each predicted text in parallel through a large language model to obtain multiple second predicted words; wherein one second predicted word corresponds to one predicted text;

[0009] Based on the plurality of first predicted words and the plurality of second predicted words, a target predicted word of the input text is determined.

[0010] In some embodiments, generating a candidate word pair set after word pair expansion based on the input text and the preset word pair set includes:

[0011] Performing word segmentation processing on the input text according to a predetermined word segmentation length to obtain a plurality of input words included in the input text; wherein the predetermined word segmentation length is the word length in the preset word pair set;

[0012] Input words in the input text that are adjacent and satisfy the number of words included in the word pairs in the preset word pair set are combined into input word pairs, and the input word pairs are added to the preset word pair set to obtain the candidate word pair set.

[0013] In some embodiments, the determining a plurality of first predicted words in the candidate word pair set based on the words in the input text, and generating a plurality of predicted texts based on the input text and the plurality of first predicted words, comprises:

[0014] sequentially querying whether there is a target word corresponding to the current query word in the candidate word pair set; wherein the query word in the first query is the last word in the input text;

[0015] In response to the existence of a target word corresponding to the current query word, determining the target word corresponding to the current query word as a first predicted word; wherein the current query word after the first query is the first predicted word obtained in the previous query;

[0016] The input text is taken as a predicted text, and the input text and the multiple first predicted words are spliced ​​multiple times according to the prediction order of the first predicted words to obtain predicted texts other than the input text; wherein the predicted text obtained by the first splicing is obtained by splicing the input text and the first predicted word obtained by the first prediction, and the predicted text obtained by each splicing after the first splicing is obtained by splicing the predicted text obtained by the previous splicing and a first predicted word to be spliced ​​determined according to the prediction order.

[0017] In some embodiments, the method further comprises:

[0018] In response to the query number threshold not being set and the target word corresponding to the current query word not existing, the first prediction word is determined based on the target words obtained by the query and then the determination of the first prediction word is terminated.

[0019] In some embodiments, the method further comprises:

[0020] In response to setting the query number threshold and the current query not reaching the query number threshold, if there is no target word corresponding to the current query word, a word in the candidate word pair set is randomly selected to be determined as the first predicted word.

[0021] In some embodiments, the first predicted word determined for the first time among the plurality of first predicted words is obtained by querying the candidate word pair set based on the input text, and the first predicted word determined after the first time among the plurality of first predicted words is obtained by querying the candidate word pair set based on the first predicted word determined last time; the second predicted word determined for the first time among the plurality of second predicted words is obtained by using the large language model based on the input text, and the second predicted word determined after the first time among the plurality of second predicted words is obtained by using the large language model after the input text is sequentially accumulated based on the obtained second predicted words;

[0022] The determining, based on the plurality of first predicted words and the plurality of second predicted words, a target predicted word of the input text comprises:

[0023] Each first predicted word is compared with the second predicted word in a prediction order of the predicted words, and in response to the first predicted words and the second predicted words obtained in the same prediction batch being the same, the same first predicted word or second predicted word is determined as a target predicted word corresponding to the input text.

[0024] In some embodiments, the method further comprises:

[0025] Each first predicted word is compared with the second predicted word in the prediction order of the predicted words. In response to the first predicted words and the second predicted words obtained in the same batch being different, the second predicted word and the word at the end of the predicted text corresponding to the second predicted word are combined into a predicted word pair and updated to the candidate word pair set.

[0026] In some embodiments, in response to the first predicted word and the second predicted word obtained in the same batch being different, combining the second predicted word and the word at the end of the predicted text corresponding to the second predicted word into a predicted word pair and updating the pair to the candidate word pair set includes:

[0027] In response to the first predicted word and the second predicted word obtained in the same batch being different and satisfying a preset update condition, the second predicted word and the word at the end of the predicted text corresponding to the second predicted word are combined into a predicted word pair and updated to the candidate word pair set; wherein satisfying the preset update condition includes: meeting a preset update frequency, or reaching a preset update time.

[0028] In some embodiments, the method further comprises:

[0029] Based on the determined order of each target prediction word, each target prediction word is sequentially concatenated to the end of the input text to generate a response text;

[0030] In response to the length of the answer text obtained by any splicing being greater than a preset text length threshold and / or the target prediction word of any splicing containing a terminator, the target prediction word is terminated and the answer text obtained by the previous splicing is output.

[0031] According to a second aspect of an embodiment of the present disclosure, a text prediction device is provided, including:

[0032] A first generating module, configured to generate a candidate word pair set after word pair expansion based on an input text and a preset word pair set;

[0033] a second generating module, configured to determine a plurality of first predicted words in the candidate word pair set based on the words in the input text, and generate a plurality of predicted texts based on the input text and the plurality of first predicted words; wherein the plurality of predicted texts include the input text and a plurality of texts obtained by concatenating the input text with at least one first predicted word;

[0034] A third generation module is used to predict each predicted text in parallel through a large language model to obtain a plurality of second predicted words; wherein one second predicted word corresponds to one predicted text;

[0035] The first determination module is configured to determine a target predicted word of the input text based on the plurality of first predicted words and the plurality of second predicted words.

[0036] In some embodiments, the first generation module is further used to segment the input text according to a predetermined segmentation length to obtain multiple input words included in the input text; wherein the predetermined segmentation length is the word length in the preset word pair set; adjacent input words in the input text that meet the number of words included in the word pairs in the preset word pair set are formed into input word pairs, and the input word pairs are added to the preset word pair set to obtain the candidate word pair set.

[0037] In some embodiments, the second generation module is also used to query in sequence in the candidate word pair set whether there is a target word corresponding to the current query word; wherein the query word in the first query is the last word of the input text; in response to the existence of a target word corresponding to the current query word, the target word corresponding to the current query word is determined as a first predicted word; wherein the current query word after the first query is the first predicted word obtained in the previous query; the input text is used as a predicted text, and the input text and the multiple first predicted words are spliced ​​multiple times in the predicted order of the first predicted words to obtain each predicted text other than the input text; wherein the predicted text obtained by the first splicing is obtained by splicing the input text and the first predicted word obtained by the first prediction, and the predicted text obtained by each splicing after the first splicing is obtained by splicing the predicted text obtained by the previous splicing and a first predicted word to be spliced ​​determined according to the predicted order.

[0038] In some embodiments, the apparatus further comprises:

[0039] The second determination module is configured to, in response to the query number threshold not being set and the target word corresponding to the current query word not existing, determine the plurality of first prediction words based on the target words obtained by the query and then terminate the determination of the first prediction word.

[0040] In some embodiments, the apparatus further comprises:

[0041] The third determination module is used to randomly select a word in the candidate word pair set as the first predicted word in response to the query number threshold being set and the current query not reaching the query number threshold and if there is no target word corresponding to the current query word.

[0042] In some embodiments, the first predicted word determined for the first time among the plurality of first predicted words is obtained by querying the candidate word pair set based on the input text, and the first predicted word determined after the first time among the plurality of first predicted words is obtained by querying the candidate word pair set based on the first predicted word determined last time; the second predicted word determined for the first time among the plurality of second predicted words is obtained by using the large language model based on the input text, and the second predicted word determined after the first time among the plurality of second predicted words is obtained by using the large language model after the input text is sequentially accumulated based on the obtained second predicted words;

[0043] The first determination module is further used to compare each first predicted word with the second predicted word according to the prediction order of the predicted words, and in response to the first predicted words and the second predicted words obtained in the same prediction batch being the same, determine the same first predicted word or second predicted word as a target predicted word corresponding to the input text.

[0044] In some embodiments, the apparatus further comprises:

[0045] A comparison and update module is used to compare each first predicted word with the second predicted word in the prediction order of the predicted words, and in response to the first predicted words and the second predicted words obtained in the same batch being different, the second predicted word and the word at the end of the predicted text corresponding to the second predicted word are combined into a predicted word pair and updated to the candidate word pair set.

[0046] In some embodiments, the comparison and update module is also used to combine the second predicted word and the word at the end of the predicted text corresponding to the second predicted word into a predicted word pair and update it to the candidate word pair set in response to the first predicted word and the second predicted word obtained in the same batch being different and satisfying a preset update condition; wherein satisfying the preset update condition includes: meeting a preset update frequency, or reaching a preset update time.

[0047] In some embodiments, the apparatus further comprises:

[0048] The fourth generation module is used to sequentially splice each target prediction word to the end of the input text based on the determined order of each target prediction word to generate a response text; in response to the length of the response text obtained by any splicing being greater than a preset text length threshold and / or the target prediction word of any splicing contains a terminator, stop splicing the target prediction word and output the response text obtained by the previous splicing.

[0049] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0050] processor;

[0051] Memory for storing computer programs or instructions;

[0052] The processor executes the computer program or instructions to implement the steps of the method described in the first aspect above.

[0053] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, wherein the storage medium stores a computer program or instructions. When the computer program or instructions in the storage medium are executed by a processor, the steps of the method described in the first aspect above are implemented.

[0054] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program or instructions, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0055] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0056] In the disclosed embodiment, the electronic device can directly predict multiple first predicted words using a candidate word pair set. On the one hand, multiple possible first predicted words in the future can be predicted in one decoding step of the reasoning process, so that multiple second predicted words can be predicted in parallel for multiple possible predicted texts in one decoding step, and then the multiple first predicted words can be verified in one decoding step, so that as many target predicted words in multiple future decoding steps as possible can be determined in one decoding step, thereby increasing the decoding step length of a single decoding step, reducing the number of decoding steps, thereby reducing the number of times a large number of parameters are loaded or migrated on a GPU, and improving the speed and efficiency of text prediction as a whole; on the other hand, the electronic device adopts By replacing the small model with a set of candidate word pairs, the "inference" process does not need to introduce a small model with high computational complexity, which reduces the computational complexity of "inference", improves the timeliness of reasoning, and increases the applicable scenarios of the text prediction method (for example, it is more applicable to distributed deployment scenarios). On the other hand, the candidate word pair set is generated by expanding the input text and the preset word pair set. The implementation method of the candidate word pair set does not require complex neural network training, which reduces the technical complexity and implementation difficulty of "inference", thereby reducing the difficulty of obtaining the "inference" small model, increasing the scope of application of the text prediction method, and facilitating a balance between accuracy and deployment convenience.

[0057] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0059] Figure 1 is a flowchart of a text prediction method according to an exemplary embodiment;

[0060] Figure 2 is a schematic diagram of a text prediction process provided by an embodiment of the present disclosure;

[0061] Figure 3 is a block diagram of a text prediction device according to an exemplary embodiment;

[0062] Figure 4 The present invention is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0063] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0064] In order to solve the problem that the reasoning process of large language models is limited by the aforementioned hardware resources, especially the memory bandwidth, the relevant technology introduces the speculative decoding technology to realize the reasoning acceleration of large language models. Speculative decoding is a "guess first and then verify" decoding algorithm. It efficiently "guesses" the predicted words that may be generated in multiple future decoding steps in a single decoding step, and verifies them through the target large language model (Target LLM) to generate the decoding result of the current decoding step. This method can theoretically ensure the consistency of the decoding result and the autoregressive decoding result of the target large language model.

[0065] In the related art, a small model (Smaller LM) of the same series as the target large language model is usually used for "inference". The small model is trained with a model structure, word segmentation method, training corpus and training process similar to the target large language model. Therefore, there is a certain "behavioral similarity" between the small model and the target large language model, which enables the small model to be used as an efficient "inference" model. For example, for the reasoning acceleration of the large language model 1 (OPT-70B), the small model 1 (OPT-125M) of the same series can be used for inference; for another example, for the large language model 2 (T5-XXL), the small model 2 (T5-small) of the same series can be used for inference. The advantage of this method in the related art is that it can directly utilize existing model resources without the need for additional training.

[0066] However, in the related art, there are also some disadvantages in using a small model as the "inference" model. First, introducing an additional small model in the "inference" stage will increase the computational complexity of the reasoning process, especially in distributed deployment scenarios. This additional computational burden may become a bottleneck for the system and increase the delay time of reasoning. Second, small models are difficult to obtain. In practical applications, not all large language models have small models of the same series. For example, the large language model 3 (LLaMA-7B) does not have a directly available small model of the same series, which limits the scope of application of the inference decoding technology. In addition, when there is no small model of the same series as the large language model, the target large language model itself is usually used for "inference". However, this inference process requires the introduction of multiple additional feedforward neural networks (FFN Heads) or training processes, which increases the complexity of the technology and the difficulty of implementation. In addition, it is difficult to balance the "inference accuracy" and "convenience of deployment" of this method.

[0067] In this regard, the present disclosure discloses a text prediction method. Figure 1 is a flowchart of a text prediction method according to an exemplary embodiment. Figure 1 As shown, the method mainly includes the following steps:

[0068] S101, generating a candidate word pair set after word pair expansion based on input text and a preset word pair set;

[0069] S102: Based on the words in the input text, determine a plurality of first predicted words in the candidate word pair set, and generate a plurality of predicted texts based on the input text and the plurality of first predicted words; wherein the plurality of predicted texts include the input text and a plurality of texts obtained by concatenating the input text with at least one first predicted word;

[0070] S103, predicting each predicted text in parallel using a large language model to obtain a plurality of second predicted words; wherein one second predicted word corresponds to one predicted text;

[0071] S104: Determine a target predicted word of the input text based on the multiple first predicted words and the multiple second predicted words.

[0072] The text prediction method provided in the embodiments of the present disclosure may be executed by an electronic device, wherein the electronic device may be a terminal device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, or a server device such as a server cluster or a cloud server. In some possible implementations, the text prediction method may be implemented by a processor calling computer-readable instructions stored in a memory.

[0073] In the disclosed embodiments, the text prediction method is applied to a natural language processing model. The large language model in the text prediction method is usually a deep learning model with a large number of parameters. In the field of natural language processing, language patterns, grammar and semantics are learned by processing a large amount of text data, so as to understand and generate human language, including but not limited to text generation, question answering, text classification, etc. In the disclosed embodiments, there are many types of large language models, which can be general large language models suitable for various natural language processing tasks, such as the generative pre-trained model series (Generative Pre-trained Transformer, GPT), Wenxinyiyan, etc.; or they can be large language models in specific fields, which have higher accuracy and professionalism in the corresponding fields, such as large language models trained for medical, legal, financial and other fields.

[0074] In step S101, the preset word pair is composed of two words with an associated relationship, which may be a semantically continuous word pair, such as "I, love", "love, you"; or a logically associated word pair, such as "I, good". In the disclosed embodiment, the electronic device may pre-collect predetermined texts, and perform word segmentation, frequency statistics and other processing based on a large amount of predetermined texts to obtain a plurality of word pairs with an associated relationship to generate a preset word pair set; wherein the predetermined text may be a predetermined text formed based on historical prediction data of a large language prediction model; or a predetermined text formed based on specific domain knowledge or a corpus of a specific prediction task.

[0075] In the disclosed embodiment, the input text is the starting text that the target large language model needs to perform inference decoding, which can be text data during intelligent search; or text data obtained by converting voice data during intelligent dialogue. In the disclosed embodiment, there are multiple ways for the electronic device to generate a candidate word pair set after word pair expansion based on the input text and the preset word pair set. It can be to select a preset word pair set related to the input text field to generate a candidate word pair set with a high correlation with the input text; or it can be to select a preset word pair set corresponding to a specific application scenario to generate a candidate word pair set suitable for a specific application scenario. In the disclosed embodiment, word pair expansion is performed based on the input text and the preset word pair set, and the obtained candidate word pair set is conducive to improving the accuracy of predictive word guessing in related fields or specific application scenarios.

[0076] In the disclosed embodiment, the electronic device may use an N-gram algorithm to guide the generation of a candidate word pair set, and may use the N-gram algorithm to segment and perform probability statistics on the input text and the predetermined text respectively, and determine N consecutive word pairs in the input text or the predetermined text according to the probability of the words appearing together after segmentation, thereby generating a candidate word pair set. It should be noted that the N-gram algorithm considers that the probability of the Nth word appearing is only related to the first N-1 words, which is conducive to simplifying the complex dependency relationship between words in natural language.

[0077] In the disclosed embodiment, the electronic device may use different computer resources to calculate the generation of the first predicted word and the prediction of the second predicted word, respectively. The generation of the candidate word pair set and the determination of the first predicted word may be calculated using a central processing unit (CPU); the prediction calculation of the second predicted word may be calculated using a tensor processing unit (TPU). In the disclosed embodiment, during the text prediction process of the electronic device, the first predicted word and the second predicted word rely on different hardware resources for prediction, which reduces the occurrence of computer resource grabbing and improves the balance of the electronic device when running the text prediction method.

[0078] In the disclosed embodiment, when the electronic device generates a candidate word pair set, the word length of the word pair in the candidate word pair set is set according to the text characteristics of the input text, and can also be set according to the prediction accuracy. In the disclosed embodiment, the electronic device can use different N values ​​in the N-gram algorithm to generate multiple candidate word pair sets including different numbers of words, so that in step S102, a suitable candidate word pair set can be selected as needed to generate the first predicted word, which can better capture the word order information of different input texts and improve the flexibility of decoding.

[0079] In step S102, the electronic device may determine multiple first prediction words in the candidate word pair set based on the words in the input text in various ways. In some embodiments, the electronic device may query a first prediction word in the candidate word pair set that is continuous with the word at the end of the input text, and continue to query a first prediction word in the candidate word pair set that is continuous with the word at the end of the input text in the current time after the first query is completed; wherein the length of the word at the end of the input text may be set as needed, and may be a single word with a length of 1, such as "you"; or may be multiple words with a length greater than 1, such as "love you". In other embodiments, a prediction tree may be introduced in the process of determining the first prediction word, and multiple first prediction words in the candidate word pair set that are continuous with the word at the end of the input text may be query based on the word at the end of the input text, and multiple first prediction words in the candidate word pair set that are continuous with the words of the first prediction words in the previous time may be queryed in the current time after the first query is completed.

[0080] In the disclosed embodiment, the electronic device generates multiple predicted texts based on the input text and multiple first predicted words. The input text may be directly used as a predicted text; or the input text may be concatenated with one or more first predicted words to obtain multiple predicted texts. In the disclosed embodiment, the process of the electronic device determining the first predicted word and generating the predicted text may adopt multiple reasoning modes. It may be a dynamic batch reasoning mode. When the corresponding target word cannot be found in the candidate word pair set, the determination of the first predicted word is terminated and the corresponding multiple batches of predicted texts are generated; or it may be a fixed batch reasoning mode. When the number of the first predicted words obtained by the query meets the requirement of generating a preset batch of predicted texts, the determination of the first predicted word is terminated and the corresponding multiple batches of predicted texts are generated.

[0081] In step S103, the electronic device predicts each predicted text in parallel through the large language model. There are many ways to obtain multiple second predicted words. The predicted text can be predicted using a general large language model; or the specific field corresponding to the input text can be determined based on the input text, and the predicted text can be predicted using the large language model of the specific field. In the embodiment of the present disclosure, when the electronic device predicts the predicted text in parallel through the large language model, all the predicted texts can be directly processed in parallel; or the predicted texts can be grouped and then each group of predicted texts can be processed in parallel.

[0082] As described above, in the embodiment of the present disclosure, the electronic device first determines a first predicted word for an input text query, one first predicted word corresponds to one input text, and one second predicted word also corresponds to one predicted text. Since the predicted text includes the input text, there are corresponding first predicted words and second predicted words for the input text; for predicted text other than the input text, the input text and multiple first predicted words are concatenated in sequence according to the prediction order of the first predicted words. For the same predicted text other than the input text, there are also corresponding first predicted words and second predicted words.

[0083] In step S104, the electronic device determines a target predicted word for the input text based on the multiple first predicted words and the multiple second predicted words. The determination may be made using the first predicted word and the second predicted word corresponding to the same predicted text. If the first predicted word is the same as the second predicted word, the first predicted word is determined as a target predicted word corresponding to the input text. If the first predicted word is different from the second predicted word, the predicted word pair formed by combining the word at the end of the predicted text with the second predicted word is updated to the candidate word pair set.

[0084] In the disclosed embodiment, the electronic device may update the candidate word pair set when the preset update frequency is met; or may update the candidate word pair set when the preset update time is reached. In the disclosed embodiment, the electronic device may concatenate the target predicted word to the end of the input text to generate the answer text, and repeatedly perform the above steps S102 to S104 until the preset decoding termination condition is reached, stop decoding and output the answer text; wherein the decoding termination condition includes: the length of the decoded answer text is greater than the preset text length threshold or the target predicted word contains a terminator, etc.

[0085] In the disclosed embodiment, the electronic device can directly predict multiple first predicted words using a candidate word pair set. On the one hand, multiple possible first predicted words in the future can be predicted in one decoding step of the reasoning process, so that multiple second predicted words can be predicted in parallel for multiple possible predicted texts in one decoding step, and then the multiple first predicted words can be verified in one decoding step, so that as many target predicted words in multiple future decoding steps as possible can be determined in one decoding step, thereby increasing the decoding step length of a single decoding step, reducing the number of decoding steps, thereby reducing the number of times a large number of parameters are loaded or migrated on a GPU, and improving the speed and efficiency of text prediction as a whole; on the other hand, the electronic device adopts By replacing the small model with a set of candidate word pairs, the "inference" process does not need to introduce a small model with high computational complexity, which reduces the computational complexity of "inference", improves the timeliness of reasoning, and increases the applicable scenarios of the text prediction method (for example, it is more applicable to distributed deployment scenarios). On the other hand, the candidate word pair set is generated by expanding the input text and the preset word pair set. The implementation method of the candidate word pair set does not require complex neural network training, which reduces the technical complexity and implementation difficulty of "inference", thereby reducing the difficulty of obtaining the "inference" small model, increasing the scope of application of the text prediction method, and facilitating a balance between accuracy and deployment convenience.

[0086] In some embodiments, generating a candidate word pair set after word pair expansion based on the input text and the preset word pair set includes:

[0087] Performing word segmentation processing on the input text according to a predetermined word segmentation length to obtain a plurality of input words included in the input text; wherein the predetermined word segmentation length is the word length in the preset word pair set;

[0088] Input words in the input text that are adjacent and satisfy the number of words included in the word pairs in the preset word pair set are combined into input word pairs, and the input word pairs are added to the preset word pair set to obtain the candidate word pair set.

[0089] As mentioned above, in the embodiment of the present disclosure, the electronic device can use the N-gram algorithm to generate candidate word pair sets with different numbers of words. The length of the N-gram can be set as needed, and the present disclosure does not limit it. Exemplarily, N=2 is used for illustration. When N=2, the electronic device obtains a preset word pair set including word pairs such as "I, love" and "love, you" based on a large number of pre-collected predetermined texts.

[0090] In an embodiment of the present disclosure, the predetermined word segmentation length is the same as the word length in the preset word pair set, and the electronic device performs word segmentation processing on the input text according to the predetermined word segmentation length. For example, assuming that the input text is "please tell me", the electronic device performs word segmentation processing on "please tell me" according to the word length in the preset word pair set (for example, the word length is 1), and obtains multiple input words such as "please", "tell", "sue", and "I", and combines adjacent input words according to the number of words being 2 (N=2) to obtain input word pairs such as "please, tell", "tell, sue", and "sue, me", and adds the input word pairs to the preset word pair set to obtain a candidate word pair set consisting of "please, tell", "tell, sue", "sue, me", "I, love" and "love, you".

[0091] In the disclosed embodiment, the electronic device adds the input word pairs generated by the input text to the candidate word pair set obtained by the preset word pair set, which can improve the correlation between the candidate word pair set and the input text, so that the candidate word pair set can better adapt to different types of input text, thereby improving the prediction accuracy of the first predicted word when determining the first predicted word based on the input text and the candidate word pair set.

[0092] In some embodiments, the determining a plurality of first predicted words in the candidate word pair set based on the words in the input text, and generating a plurality of predicted texts based on the input text and the plurality of first predicted words, comprises:

[0093] sequentially querying whether there is a target word corresponding to the current query word in the candidate word pair set; wherein the query word in the first query is the last word in the input text;

[0094] In response to the existence of a target word corresponding to the current query word, determining the target word corresponding to the current query word as a first predicted word; wherein the current query word after the first query is the first predicted word obtained in the previous query;

[0095] The input text is taken as a predicted text, and the input text and the multiple first predicted words are spliced ​​multiple times according to the prediction order of the first predicted words to obtain predicted texts other than the input text; wherein the predicted text obtained by the first splicing is obtained by splicing the input text and the first predicted word obtained by the first prediction, and the predicted text obtained by each splicing after the first splicing is obtained by splicing the predicted text obtained by the previous splicing and a first predicted word to be spliced ​​determined according to the prediction order.

[0096] Figure 2 is a schematic diagram of a text prediction process provided by an embodiment of the present disclosure, such as Figure 2 As shown, the text prediction process includes the following steps:

[0097] S201, generating a candidate word pair set based on a common word pair set and an input text;

[0098] S202, generating a first predicted word and a predicted text according to the current input text and the candidate word pair set;

[0099] S203, the first predicted word is combined with the initial input text as a multi-batch input and decoded in parallel to obtain a second predicted word;

[0100] S204, comparing the generated second predicted word and the first predicted word to see if they are the same; if so, executing step S205; if not, executing step S206;

[0101] S205, adopting the first predicted word;

[0102] S206, updating the candidate word pair set;

[0103] S207, determine whether the termination condition is met; if so, end text prediction; if not, execute step S202.

[0104] In step S201, the public word pair set is the preset word pair set in the embodiment of the present disclosure, and the candidate word pair set can be expanded and generated using a 2-gram algorithm. The specific method can refer to the above-mentioned related description.

[0105] In step S202, the electronic device may predict multiple first predicted words and generate multiple predicted texts based on the input text and the candidate word pair set. In the disclosed embodiment, the electronic device may sequentially query whether there is a target word corresponding to the current query word in the candidate word pair set, wherein the query word in the first query is the last word in the input text, and the current query word after the first query is the first predicted word obtained in the previous query, and the number of target words may be one or more. In some embodiments, if there are multiple target words corresponding to the current query word, the multiple target words corresponding to the current query word are all determined as first predicted words, and the target words of each first predicted word may be queried in the candidate word pair set based on the multiple first predicted words in the next query process.

[0106] In an embodiment of the present disclosure, the electronic device uses the input text as a predicted text, and splices the input text with multiple first predicted words multiple times according to the predicted order of the first predicted words to obtain predicted texts other than the input text. The predicted text obtained by the first splicing is obtained by splicing the input text with the first predicted word, and the predicted text obtained by each splicing after the first splicing is obtained by splicing the predicted text obtained by the previous splicing with a first predicted word to be spliced ​​determined in the predicted order. In some embodiments, if there are multiple first predicted words in the current splicing process, the multiple first predicted words are spliced ​​with the previous predicted text, and multiple predicted texts corresponding to the number of first predicted words are in the current splicing.

[0107] Exemplarily, the last word of the input text is "I", and the target word first queried in the candidate word pair set is "love", then "love" is determined as the first predicted word for the first time; after the first query, the queried target word "you" can be used as the first predicted word for the current time based on the first predicted word "love" of the previous time; in the process of splicing predicted texts, the input text "please tell me" and the first predicted word "love" predicted for the first time are spliced ​​to obtain a predicted text of "please tell me love"; after the first splicing, the previous spliced ​​text "please tell me love" and the first predicted word "you" predicted for the second time are used to splice to obtain a predicted text of "please tell me I love".

[0108] Exemplarily, the electronic device may introduce a prediction tree mode to generate a first prediction word and a prediction text. For the input text "please tell me", the first prediction word for the first time may be "love" or "do", and the first prediction words after the first time may be "you" and "what". Based on the input text and multiple first prediction words, four batches of prediction texts are generated: "Please tell me love", "Please tell me do", "Please tell me you love", and "Please tell me what to do".

[0109] In step S203, the electronic device may use the multiple batches of predicted texts as inputs of the large language model, and use the large language model to decode the multiple batches of predicted texts to obtain the second predicted words of each batch;

[0110] In step S204, the electronic device may compare whether the first predicted words and the second predicted words predicted in each batch are the same;

[0111] In step S205, the electronic device determines the first predicted word as a target predicted word;

[0112] In step S206, the electronic device may update the candidate word pair set based on the second predicted word predicted by the large language model;

[0113] In step S207, when the electronic device determines that the termination condition is met, the text prediction is terminated and the answer text obtained by concatenating the input text and the target predicted word is output.

[0114] In the disclosed embodiment, the electronic device generates multiple predicted texts based on the input text and multiple first predicted words, so that a large language model can perform parallel prediction on the multiple predicted texts to obtain multiple second predicted words. The number of second predicted words predicted in a single decoding step increases. On the one hand, more first predicted words can be verified based on the second predicted words in a single decoding step, and more target predicted words can be generated, thereby improving the text prediction efficiency. On the other hand, when a single decoding step can generate more predicted words, the number of decoding steps can be shortened, thereby reducing the number of times a large number of parameters are loaded or migrated on the GPU, balancing the calculation memory access ratio, and improving the GPU utilization rate.

[0115] In some embodiments, the method further comprises:

[0116] In response to the query number threshold not being set and the target word corresponding to the current query word not existing, the first prediction word is determined based on the target words obtained by the query and then the determination of the first prediction word is terminated.

[0117] In the disclosed embodiment, the electronic device may generate the first predicted word by a dynamic batch reasoning method. If the query number threshold for querying the target word in the candidate word pair set is not set, if there is no corresponding target word for the current query word, the query of the target word is terminated, and all the queried target words are output in order as the first target word. For example, if the query word is "you", and the target word corresponding to "you" cannot be found in the candidate word pair set, the query in the candidate word pair set is stopped, and multiple first predicted words "I", "love", and "you" are determined.

[0118] In the disclosed embodiment, the electronic device may generate the first predicted word by a dynamic batch reasoning method. If the target word cannot be found in the candidate word pair set, that is, there is no target word with a high correlation with the query word, the number of invalid queries may be reduced and the prediction efficiency of the first predicted word may be improved.

[0119] In some embodiments, the method further comprises:

[0120] In response to setting the query number threshold and the current query not reaching the query number threshold, if there is no target word corresponding to the current query word, a word in the candidate word pair set is randomly selected to be determined as the first predicted word.

[0121] In the embodiments of the present disclosure, the electronic device may adopt a fixed batch inference method to generate the first predicted word. By setting a query count threshold, the query count threshold can be adaptively set according to the number of batches in a fixed batch. For example, if 5 batches of predicted texts are required, the query count threshold is set to 4 times. Thus, the input text and 4 first predicted words can generate 5 batches of predicted texts.

[0122] In the embodiments of the present disclosure, if the current query count has not reached the query count threshold, but there is no corresponding target word for the current query word, a word can be randomly selected from the candidate word pair set as a first predicted word. In the embodiments of the present disclosure, when the current query count reaches the query count threshold, the query for the target word is terminated, and all the queried target words are output in order as the first target words. For example, if the target word corresponding to "你" cannot be found in the candidate word pair set, a "告" is randomly selected as the first predicted word, and multiple first predicted words "我", "爱", "你", "告" are determined.

[0123] In the embodiments of the present disclosure, the electronic device may adopt a fixed batch inference method to generate the first predicted word. In the case where the target word cannot be found in the candidate word pair set, a sufficient number of first predicted words are still output to generate a sufficient number of batches of predicted texts, thereby increasing the number of predicted texts that the large language model can process in parallel and improving the prediction efficiency of the predicted texts.

[0124] In some embodiments, the first predicted word determined for the first time among the multiple first predicted words is obtained by querying the candidate word pair set based on the input text, and the first predicted words determined after the first time among the multiple first predicted words are obtained by querying the candidate word pair set based on the previously determined first predicted word; the first second predicted word determined among the multiple second predicted words is obtained by using the large language model based on the input text, and the second predicted words determined after the first time among the multiple second predicted words are obtained by using the large language model after the input text has sequentially accumulated based on the already obtained second predicted words;

[0125] Determining the target predicted word of the input text based on the multiple first predicted words and the multiple second predicted words includes:

[0126] Comparing each first predicted word with the second predicted word according to the prediction order of the predicted words. In response to the first predicted word and the second predicted word obtained in the same prediction batch being the same, the same first predicted word or second predicted word is determined as a target predicted word corresponding to the input text.

[0127] In the disclosed embodiment, the electronic device may perform multiple batches of predictions in the process of determining multiple first predicted words using a candidate word pair set. The first predicted word determined for the first time may be a target word obtained by searching the candidate word pair set based on a word of a first predetermined length at the end of an input text; the first predicted word determined after the first time may be obtained by searching the candidate word pair set based on the first predicted word determined last time.

[0128] In the disclosed embodiment, the electronic device may perform multiple batches of predictions in the process of determining multiple second predicted words using a large language model. The second predicted words determined for the first time may be obtained by inputting the input text into the large language model and predicting it; the second predicted words determined after the first time may be obtained by inputting the text after the second predicted words that have been accumulated in sequence into the input text into the large language model and predicting it.

[0129] In the disclosed embodiment, the electronic device compares each first predicted word with the second predicted word in the prediction order of the predicted words, and may compare the first predicted words and the second predicted words of the same batch. If the first predicted word and the second predicted word obtained in the same prediction batch are the same, then the first predicted word of the predicted text is considered to be accurate, and the same first predicted word or the second predicted word is determined as a target predicted word corresponding to the input text; if the first predicted word and the second predicted word obtained in the same prediction batch are different, then the first predicted word of the predicted text is considered to be inaccurate, and the next batch of predicted words may be compared; wherein the first predicted words and the second predicted words of the same batch correspond to the same content to be predicted.

[0130] In the disclosed embodiment, the electronic device generates multiple batches of first predicted words through a set of candidate word pairs, uses multiple batches of second predicted words generated by a large language model, compares the first predicted words of the same batch with the second predicted words, verifies the accuracy of the first predicted words, and verifies and determines the possible first predicted words for multiple future decoding steps in one decoding step, increases the decoding step length of a single decoding step, reduces the number of decoding steps, thereby reducing the number of times a huge amount of parameters are loaded or migrated on the GPU, and overall improves the speed and efficiency of text prediction.

[0131] In some embodiments, the method further comprises:

[0132] Each first predicted word is compared with the second predicted word in the prediction order of the predicted words. In response to the first predicted words and the second predicted words obtained in the same batch being different, the second predicted word and the word at the end of the predicted text corresponding to the second predicted word are combined into a predicted word pair and updated to the candidate word pair set.

[0133] In the embodiment of the present disclosure, when the first predicted word and the second predicted word obtained in the same batch are different, the electronic device considers that the second predicted word is accurate, and combines the second predicted word and the word at the end of the predicted text corresponding to the second predicted word into a predicted word pair. There are multiple ways to update the candidate word pair set. The predicted word pair can be added to the candidate word pair set in real time when the difference is determined; or the predicted word pair can be temporarily stored, and the candidate word pair can be updated when the preset update conditions are met.

[0134] In the disclosed embodiment, the electronic device dynamically updates the candidate word pair set according to the actual output (second predicted word) during the large language model decoding process, so that the candidate word pair set can gradually adapt to the context of the input text and generate a more accurate first predicted word.

[0135] In some embodiments, in response to the first predicted word and the second predicted word obtained in the same batch being different, combining the second predicted word and the word at the end of the predicted text corresponding to the second predicted word into a predicted word pair and updating the pair to the candidate word pair set includes:

[0136] In response to the first predicted word and the second predicted word obtained in the same batch being different and satisfying a preset update condition, the second predicted word and the word at the end of the predicted text corresponding to the second predicted word are combined into a predicted word pair and updated to the candidate word pair set; wherein satisfying the preset update condition includes: meeting a preset update frequency, or reaching a preset update time.

[0137] In the real-time example disclosed in the present invention, the preset update frequency and the preset update time can be set to match the complexity of the prediction task and the field to which the prediction task belongs. For example, for prediction tasks with fast processing speed, the update frequency is set to every half hour; for prediction tasks with high processing accuracy, the update frequency is set to every 4 hours, etc.; for another example, for prediction tasks in the financial field, the update time is set to non-business time at night; for prediction tasks in the education field, the update time is set to a time with low visit volume, etc.

[0138] In the disclosed embodiment, the electronic device updates the candidate word pair set when a preset update condition is met, so as to improve the stability of the prediction method while improving the accuracy of the candidate word pair set.

[0139] In some embodiments, the method further comprises:

[0140] Based on the determined order of each target prediction word, each target prediction word is sequentially concatenated to the end of the input text to generate a response text;

[0141] In response to the length of the answer text obtained by any splicing being greater than a preset text length threshold and / or the target prediction word of any splicing containing a terminator, the target prediction word is terminated and the answer text obtained by the previous splicing is output.

[0142] In the disclosed embodiment, after completing the verification of multiple batches of prediction words of the current decoding step and obtaining the target prediction word of the current decoding step, the electronic device can continue to generate the first prediction word and the second prediction word of the next decoding step and determine the target prediction word corresponding to the next decoding step, and sequentially splice the target prediction words of each decoding step to the end of the input text to obtain the answer text. If the length of the answer text is greater than a preset text length threshold (for example, 200 characters), and / or the target prediction word contains a terminator, the text prediction can be ended and the answer text can be output.

[0143] In the disclosed embodiments, the electronic device performs text prediction based on a preset text length threshold, or stops prediction when a terminator is predicted, which can improve the effectiveness of the answer text, reduce the occurrence of invalid predictions, and save computer resources.

[0144] Figure 3 FIG. 1 is a block diagram of a text prediction device according to an exemplary embodiment. Figure 3 As shown, the device 300 mainly includes:

[0145] A first generating module 301 is used to generate a candidate word pair set after word pair expansion based on the input text and the preset word pair set;

[0146] A second generating module 302 is configured to determine a plurality of first predicted words in the candidate word pair set based on the words in the input text, and generate a plurality of predicted texts based on the input text and the plurality of first predicted words; wherein the plurality of predicted texts include the input text and a plurality of texts obtained by concatenating the input text with at least one first predicted word;

[0147] The third generation module 303 is used to predict each predicted text in parallel through the large language model to obtain multiple second predicted words; wherein one second predicted word corresponds to one predicted text;

[0148] The first determination module 304 is configured to determine a target predicted word of the input text based on the plurality of first predicted words and the plurality of second predicted words.

[0149] In some embodiments, the first generation module 301 is further used to segment the input text according to a predetermined segmentation length to obtain multiple input words included in the input text; wherein the predetermined segmentation length is the word length in the preset word pair set; adjacent input words in the input text that meet the number of words included in the word pairs in the preset word pair set are combined into input word pairs, and the input word pairs are added to the preset word pair set to obtain the candidate word pair set.

[0150] In some embodiments, the second generation module 302 is further used to query in sequence in the candidate word pair set whether there is a target word corresponding to the current query word; wherein the query word at the first query is the last word of the input text; in response to the existence of a target word corresponding to the current query word, the target word corresponding to the current query word is determined as a first predicted word; wherein the current query word after the first query is the first predicted word obtained in the previous query; the input text is used as a predicted text, and the input text and the multiple first predicted words are spliced ​​multiple times in the predicted order of the first predicted words to obtain each predicted text other than the input text; wherein the predicted text obtained by the first splicing is obtained by splicing the input text and the first predicted word obtained by the first prediction, and the predicted text obtained by each splicing after the first splicing is obtained by splicing the predicted text obtained by the previous splicing and a first predicted word to be spliced ​​determined according to the predicted order.

[0151] In some embodiments, the apparatus further comprises:

[0152] The second determination module is configured to, in response to the query number threshold not being set and the target word corresponding to the current query word not existing, determine the plurality of first prediction words based on the target words obtained by the query and then terminate the determination of the first prediction word.

[0153] In some embodiments, the apparatus further comprises:

[0154] The third determination module is used to randomly select a word in the candidate word pair set as the first predicted word in response to the query number threshold being set and the current query not reaching the query number threshold and if there is no target word corresponding to the current query word.

[0155] In some embodiments, the first predicted word determined for the first time among the plurality of first predicted words is obtained by querying the candidate word pair set based on the input text, and the first predicted word determined after the first time among the plurality of first predicted words is obtained by querying the candidate word pair set based on the first predicted word determined last time; the second predicted word determined for the first time among the plurality of second predicted words is obtained by using the large language model based on the input text, and the second predicted word determined after the first time among the plurality of second predicted words is obtained by using the large language model after the input text is sequentially accumulated based on the obtained second predicted words;

[0156] The first determination module 304 is further configured to compare each first predicted word with the second predicted word according to the prediction order of the predicted words, and in response to the first predicted word and the second predicted word obtained in the same prediction batch being the same, determine the same first predicted word or second predicted word as a target predicted word corresponding to the input text.

[0157] In some embodiments, the apparatus further comprises:

[0158] A comparison and update module is used to compare each first predicted word with the second predicted word in the prediction order of the predicted words, and in response to the first predicted words and the second predicted words obtained in the same batch being different, the second predicted word and the word at the end of the predicted text corresponding to the second predicted word are combined into a predicted word pair and updated to the candidate word pair set.

[0159] In some embodiments, the comparison and update module is also used to combine the second predicted word and the word at the end of the predicted text corresponding to the second predicted word into a predicted word pair and update it to the candidate word pair set in response to the first predicted word and the second predicted word obtained in the same batch being different and satisfying a preset update condition; wherein satisfying the preset update condition includes: meeting a preset update frequency, or reaching a preset update time.

[0160] In some embodiments, the apparatus further comprises:

[0161] The fourth generation module is used to sequentially splice each target prediction word to the end of the input text based on the determined order of each target prediction word to generate a response text; in response to the length of the response text obtained by any splicing being greater than a preset text length threshold and / or the target prediction word of any splicing contains a terminator, stop splicing the target prediction word and output the response text obtained by the previous splicing.

[0162] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0163] Figure 44 is a block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 400 may be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, a server, etc.

[0164] Reference Figure 4 , the electronic device 400 may include one or more of the following components: a processing component 402 , a memory 404 , a power component 406 , a multimedia component 408 , an audio component 410 , an input / output (I / O) interface 412 , a sensor component 414 , and a communication component 416 .

[0165] The processing component 402 generally controls the overall operation of the electronic device 400, such as operations associated with at least one of display, phone calls, data communications, camera operations, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 402 may include one or more modules to facilitate the interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate the interaction between the multimedia component 408 and the processing component 402.

[0166] The memory 404 is configured to store various types of data to support operations on the electronic device 400. Examples of such data include at least one of the following: instructions for any application or method operating on the electronic device 400, contact data, phone book data, messages, pictures, and videos. The memory 404 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0167] The power supply component 406 provides power to various components of the electronic device 400. The power supply component 406 may include at least one of the following: a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 400.

[0168] The multimedia component 408 includes a screen that provides an output interface between the electronic device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 408 includes a front camera and / or a rear camera. When the electronic device 400 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0169] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC), and when the electronic device 400 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 404 or sent via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

[0170] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0171] The sensor assembly 414 includes one or more sensors for providing various aspects of status assessment for the electronic device 400. For example, the sensor assembly 414 can detect the open / closed state of the electronic device 400, the relative positioning of the components, such as the display and keypad of the electronic device 400, and the sensor assembly 414 can also detect the position change of the electronic device 400 or a component in the electronic device 400, the presence or absence of contact between the user and the electronic device 400, the orientation or acceleration / deceleration of the electronic device 400, and the temperature change of the electronic device 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 may also include a light sensor, such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 may also include, but is not limited to, at least one of the following: an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, and a temperature sensor.

[0172] The communication component 416 is configured to facilitate communication between the electronic device 400 and other devices in a wired or wireless manner. The electronic device 400 can access a wireless network based on a communication standard, such as Wi-Fi, 4G, 5G, or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (Ultra Wide Band, UWB) technology, Bluetooth (BT) technology and other technologies.

[0173] In an exemplary embodiment, the electronic device 400 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components.

[0174] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including executable instructions or a computer program, which can be executed by a processor 420 of an electronic device 400 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0175] A non-temporary computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute any one of the above-mentioned text prediction methods of the embodiments of the present disclosure.

[0176] The embodiment of the present disclosure provides a computer program product, which includes: a computer program or executable instructions, which are stored in a computer-readable storage medium. The processor of the computer device reads the computer program or executable instructions from the computer-readable storage medium, and the processor executes the computer program or executable instructions, so that the computer device executes any one of the above-mentioned text prediction methods of the embodiment of the present disclosure.

[0177] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0178] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A text prediction method, characterized in that: include: Based on the input text and the preset word pair set, generating a candidate word pair set after the word pair is expanded; Based on the words in the input text, a plurality of first predicted words are determined in the candidate word pair set, and a plurality of predicted texts are generated based on the input text and the plurality of first predicted words; wherein the plurality of predicted texts include the input text and a plurality of texts obtained by concatenating the input text with at least one first predicted word; Predicting each predicted text in parallel through a large language model to obtain multiple second predicted words; wherein one second predicted word corresponds to one predicted text; Based on the plurality of first predicted words and the plurality of second predicted words, a target predicted word of the input text is determined.

2. The text prediction method according to claim 1, characterized in that: The step of generating a candidate word pair set after word pair expansion based on the input text and the preset word pair set includes: Performing word segmentation processing on the input text according to a predetermined word segmentation length to obtain a plurality of input words included in the input text; wherein the predetermined word segmentation length is the word length in the preset word pair set; Input words in the input text that are adjacent and satisfy the number of words included in the word pairs in the preset word pair set are combined into input word pairs, and the input word pairs are added to the preset word pair set to obtain the candidate word pair set.

3. The text prediction method according to claim 1, characterized in that: The step of determining a plurality of first predicted words in the candidate word pair set based on the words in the input text, and generating a plurality of predicted texts based on the input text and the plurality of first predicted words includes: sequentially querying whether there is a target word corresponding to the current query word in the candidate word pair set; wherein the query word in the first query is the last word in the input text; In response to the existence of a target word corresponding to the current query word, determining the target word corresponding to the current query word as a first predicted word; wherein the current query word after the first query is the first predicted word obtained in the previous query; The input text is taken as a predicted text, and the input text and the multiple first predicted words are spliced ​​multiple times according to the prediction order of the first predicted words to obtain predicted texts other than the input text; wherein the predicted text obtained by the first splicing is obtained by splicing the input text and the first predicted word obtained by the first prediction, and the predicted text obtained by each splicing after the first splicing is obtained by splicing the predicted text obtained by the previous splicing and a first predicted word to be spliced ​​determined according to the prediction order.

4. The text prediction method according to claim 3, characterized in that: The method further comprises: In response to the query number threshold not being set and the target word corresponding to the current query word not existing, the first prediction word is determined based on the target words obtained by the query and then the determination of the first prediction word is terminated.

5. The text prediction method according to claim 4, characterized in that: The method further comprises: In response to setting the query number threshold and the current query not reaching the query number threshold, if there is no target word corresponding to the current query word, a word in the candidate word pair set is randomly selected to be determined as the first predicted word.

6. The text prediction method according to claim 1, characterized in that: The first predicted word determined for the first time among the plurality of first predicted words is obtained by searching the candidate word pair set based on the input text, and the first predicted word determined after the first time among the plurality of first predicted words is obtained by searching the candidate word pair set based on the first predicted word determined last time; The first determined second prediction word among the plurality of second prediction words is obtained by using the large language model based on the input text, and the second prediction words determined after the first one among the plurality of second prediction words are obtained by using the large language model after sequentially accumulating the obtained second prediction words based on the input text; The determining, based on the plurality of first predicted words and the plurality of second predicted words, a target predicted word of the input text comprises: Each first predicted word is compared with the second predicted word in a prediction order of the predicted words, and in response to the first predicted words and the second predicted words obtained in the same prediction batch being the same, the same first predicted word or second predicted word is determined as a target predicted word corresponding to the input text.

7. The text prediction method according to claim 6, characterized in that: The method further comprises: Each first predicted word is compared with the second predicted word in the prediction order of the predicted words. In response to the first predicted words and the second predicted words obtained in the same batch being different, the second predicted word and the word at the end of the predicted text corresponding to the second predicted word are combined into a predicted word pair and updated to the candidate word pair set.

8. The text prediction method according to claim 7, characterized in that: In response to the first predicted word and the second predicted word obtained in the same batch being different, combining the second predicted word and the word at the end of the predicted text corresponding to the second predicted word into a predicted word pair and updating the pair to the candidate word pair set, comprising: In response to the first predicted word and the second predicted word obtained in the same batch being different and satisfying a preset update condition, the second predicted word and the word at the end of the predicted text corresponding to the second predicted word are combined into a predicted word pair and updated to the candidate word pair set; wherein satisfying the preset update condition includes: meeting a preset update frequency, or reaching a preset update time.

9. The text prediction method according to claim 1, characterized in that: The method further comprises: Based on the determined order of each target prediction word, each target prediction word is sequentially concatenated to the end of the input text to generate a response text; In response to the length of the answer text obtained by any splicing being greater than a preset text length threshold and / or the target prediction word of any splicing containing a terminator, the target prediction word is terminated and the answer text obtained by the previous splicing is output.

10. A text prediction device, characterized in that: include: A first generating module, configured to generate a candidate word pair set after word pair expansion based on an input text and a preset word pair set; a second generating module, configured to determine a plurality of first predicted words in the candidate word pair set based on the words in the input text, and generate a plurality of predicted texts based on the input text and the plurality of first predicted words; wherein the plurality of predicted texts include the input text and a plurality of texts obtained by concatenating the input text with at least one first predicted word; A third generation module is used to predict each predicted text in parallel through a large language model to obtain a plurality of second predicted words; wherein one second predicted word corresponds to one predicted text; The first determination module is configured to determine a target predicted word of the input text based on the plurality of first predicted words and the plurality of second predicted words.

11. An electronic device, characterized in that: include: processor; Memory for storing computer programs or instructions; The processor executes the computer program or instructions to implement the steps of the method according to any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing a computer program or instruction, characterized in that: When the computer program or instructions in the storage medium are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, comprising a computer program or instructions, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images