Prediction method and device for following data, electronic equipment and storage medium

Abstract

The invention provides a method and a device for predicting following text data, electronic equipment and a storage medium. The method comprises the following steps of: obtaining previous text data; inputting the previous data into a next prediction model in the terminal equipment to obtain a next prediction result output by the next prediction model; the following prediction model is obtained by cutting an original dictionary of an embedded layer based on reserved data on the basis of a large language model, and the reserved data comprises entries of a target language and byte order split data corresponding to the entries of the target language. According to the method, the lightweight designed following prediction model is deployed in the terminal equipment, and efficient operation can be realized under the condition of limited hardware resources of the terminal equipment while following prediction is performed on the preceding data of the terminal equipment. According to the technical scheme, the previous text data input in the terminal equipment is only processed in the local terminal equipment and does not need to be uploaded to the cloud server, so that the privacy disclosure risk is greatly reduced, and the privacy security of a user in the terminal equipment is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and storage medium for predicting context data. Background Art

[0002] The existing methods for predicting the input content of the terminal device are based on pre-storing a certain amount of context-context pairs in the cloud server. After receiving the context data from the terminal device, the cloud server uses full or partial content matching to search for matching contexts in these context pairs, extract the corresponding context as recommended content, and feed it back to the mobile terminal, which is then displayed to the user for selection.

[0003] The existing methods rely on the implementation of cloud servers, which requires uploading the text data entered in the terminal device to the cloud server, posing a potential risk of privacy leakage. Summary of the invention

[0004] The present invention provides a method, device, electronic device and storage medium for predicting the following text data, so as to solve the problem of input content prediction of terminal devices in the prior art, which requires uploading text data input in the terminal device to a cloud server, and has the defect of privacy leakage, and ensure the privacy security of the terminal device.

[0005] The present invention provides a method for predicting the following data, comprising the following steps: Get the above data; Inputting the above context data into a context prediction model deployed on a terminal device to obtain a context prediction result output by the context prediction model, wherein the context prediction model is trained based on the above context sample data and context labels corresponding to the above sample data; The following context prediction model is obtained by trimming the original dictionary of the embedding layer based on the retained data on the basis of the large language model, and the retained data includes the entries of the target language and the byte order splitting data corresponding to the entries of the target language.

[0006] According to a method for predicting following data provided by the present invention, the process of determining the retained data includes: Based on the target language, multiple entries in the original dictionary are screened to determine the entries in the target language in the original dictionary; The coded data corresponding to the entry of the target language is split to determine the byte order split data corresponding to the entry of the target language.

[0007] According to a method for predicting the following text data provided by the present invention, the method of screening multiple entries in the original dictionary based on the target language to determine the entries in the target language in the original dictionary includes: Based on the byte pair encoding BPE algorithm, the original dictionary is decoded to obtain a decoded dictionary; The data of the target language in the decoding dictionary is recognized to obtain the entries of the target language.

[0008] According to a method for predicting the following text data provided by the present invention, when the target language is Chinese, the data of the target language in the decoding dictionary is identified to obtain the entry of the target language, including: Based on regular expressions, matching Chinese characters in the decoding dictionary to identify Chinese data in the decoding dictionary; The Chinese data recognized in the decoding dictionary is extracted to obtain entries in the Chinese language.

[0009] According to a method for predicting subsequent data provided by the present invention, after obtaining the subsequent prediction result output by the subsequent prediction model, the method further includes: Determining a plurality of following prediction data included in the following prediction result; The multiple subsequent prediction data are returned to the input interface of the terminal device.

[0010] According to a method for predicting subsequent text data provided by the present invention, returning the plurality of subsequent text prediction data to the input interface of the terminal device includes: Determining the usage frequency of each context prediction data based on the historical context data input into the terminal device, and determining the sorting priority of the multiple context prediction data based on the usage frequency; Based on the sorting priority, the plurality of subsequent prediction data are sorted, and the sorted subsequent prediction data are displayed on an input interface of the terminal device.

[0011] According to a method for predicting context data provided by the present invention, before inputting the above context data into a context prediction model deployed in a terminal device, the method further includes: The model parameters of the following prediction model are quantized from a first precision to a second precision, wherein a numerical range of the first precision is greater than a numerical range of the second precision.

[0012] The present invention also provides a prediction device for the following data, comprising the following modules: A data acquisition module, used to acquire the above data; A prediction module, used for inputting the above context data into a context prediction model deployed on a terminal device to obtain a context prediction result output by the context prediction model, wherein the context prediction model is trained based on the above context sample data and context labels corresponding to the above context sample data; The following context prediction model is obtained by trimming the original dictionary of the embedding layer based on the retained data on the basis of the large language model, and the retained data includes the entries of the target language and the byte order splitting data corresponding to the entries of the target language.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, the method for predicting the following data as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for predicting the following data as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for predicting subsequent data.

[0016] The method, device, electronic device and storage medium for predicting the following data provided by the present invention obtain a following prediction model by trimming the original dictionary in the embedding layer based on a large language model. A lightweight design of the following prediction model is achieved. The lightweight following prediction model is deployed in a terminal device, and while realizing the following prediction process for the preceding data of the terminal device, it can run efficiently under the limited hardware resources of the terminal device. The preceding data input into the terminal device is only processed in the local terminal device, without uploading to the cloud server, which greatly reduces the risk of privacy leakage and ensures the personal information security and input content security of users in the terminal device. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a schematic diagram of the flow chart of the method for predicting the following data provided by the present invention.

[0019] Figure 2 It is a schematic diagram of the entry recognition and retention process provided by the present invention.

[0020] Figure 3 It is a schematic diagram of dictionary cutting provided by the present invention.

[0021] Figure 4 It is a schematic diagram of the structure of the prediction device for the following data provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] With the popularization of terminal devices such as smart phones and the development of mobile Internet, the scenarios of text input on terminal devices are increasing. In order to improve user input efficiency, providing intelligent input assistance functions has become a trend.

[0025] Large language models have demonstrated powerful capabilities in natural language processing tasks, such as text generation and semantic understanding. However, large language models usually have large parameters and complex structures, and require extremely high computing and storage resources. They are difficult to run directly and efficiently under the limited hardware conditions of mobile devices such as mobile phones, resulting in the inability to provide users with real-time and convenient input prediction services locally. Most of the input predictions in related methods rely on cloud servers for calculations, but there are problems such as network latency and privacy leakage, which limits the widespread application of input prediction functions in terminal devices and the improvement of user experience.

[0026] Specifically, in the related methods, the input content prediction in the input method product is implemented based on the cloud server, and a certain amount of context-context pairs are stored in the cloud server in advance. After the cloud server receives the context content from the terminal device, it uses full or partial content matching to search for the matching context in these corpus pairs, and extracts the corresponding context as recommended content and feeds it back to the terminal device, and then displays it to the user for selection. The whole process is highly dependent on the corpus resources and network connection of the cloud server, and its recommendation logic is only based on the existing corpus matching, lacking in-depth mining and flexible application of the intrinsic logic of the language.

[0027] Another implementation method in the related method is to upload the previous text data entered by the user in the terminal device to the cloud server. The cloud server uses the large language model deployed on it to analyze and process the previous text input by the user, and then returns the predicted following text result to the terminal device for display to the user.

[0028] The shortcomings of the related methods are: Strong network dependence: It relies on cloud servers for computing. When the network signal is poor or interrupted, the input prediction function cannot be provided normally, which seriously affects the user experience. Privacy and security risks: The data entered by users is uploaded to the cloud, which poses a potential risk of privacy leakage and may involve sensitive content such as users' personal information and chat records; Response delay: Data transmission and cloud computing processes will produce certain delays, making it impossible to achieve instant input prediction feedback, especially in some scenarios with high real-time requirements (such as quick replies in instant messaging).

[0029] In view of the defects in the related methods, the present invention provides a method for predicting the following data: Figure 1 It is a flow chart of the method for predicting the following data provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 110, obtaining the above data; Step 120, inputting the above context data into a context prediction model deployed on a terminal device to obtain a context prediction result output by the context prediction model, wherein the context prediction model is trained based on the above context sample data and context labels corresponding to the above context sample data; The following context prediction model is obtained by trimming the original dictionary of the embedding layer based on the retained data on the basis of the large language model, and the retained data includes the entries of the target language and the byte order splitting data corresponding to the entries of the target language.

[0030] The following takes the terminal device executing the prediction method of the following data provided by the present invention as an example to describe the technical solution of the present invention in detail.

[0031] In step 110, the above data is acquired.

[0032] It should be noted that the preceding data corresponds to the following data, and together they constitute a complete expression of meaning. Specifically, the preceding data refers to all relevant data before a certain data point. Correspondingly, the following data refers to all relevant data after a certain data point. Among them, data points can be punctuation marks, words, sentences, paragraphs, etc. For example, for a sentence "Hello, world", the sentence can be divided into two parts: preceding data and following data based on the comma in the middle. Among them, the preceding data is "Hello" and the following data is "World".

[0033] Specifically, the above data may be data input by the user in the process of using the terminal device. After the user inputs the above data in the terminal device, the above data input in the terminal device is extracted. The acquired above data may be the above data input by the user in the process of inputting text based on the input method in the terminal device.

[0034] Specifically, the acquisition of the above data may be based on a preset program, reading the data input by the user line by line or obtaining the user input in the text box through an event listener to obtain the above data input in the terminal device.

[0035] Among them, the terminal device refers to an electronic device that can run an interactive command-line interface (CLI) or a graphical user interface (GUI) and interact directly with the user.

[0036] Terminal devices may include desktop computers, servers, or laptops, which typically run an operating system and allow users to interact with applications through a command line or graphical interface.

[0037] Terminal devices can also include mobile devices such as smartphones and tablets, which also have terminal emulator applications that allow users to operate through the command line or interact with applications through touch screens and keyboards.

[0038] End devices can also include embedded devices such as routers, switches, etc., which usually have a simple command line interface for configuration and management.

[0039] In step 120, a context prediction model is pre-constructed, and the context prediction model is used to perform context prediction on the previous data, and predict the context data corresponding to the previous data.

[0040] Specifically, the prediction model below is obtained by improving the large language model based on the large language model. Among them, large language models (LLMs) refer to natural language processing models with a large number of parameters trained using deep learning technology. This model can understand and generate natural language text, and is used for language understanding and text prediction.

[0041] Before making any improvements, you need to determine the corresponding large language model. The specific process can be: First, we collect a large amount of text data covering various scenarios, different language styles, and rich semantic expressions. These data are used as the basic corpus for training large language models.

[0042] The collected text data is preprocessed, such as word segmentation, part of speech marking, sentence structure division, etc., to make it conform to the format requirements of model training.

[0043] Choose a suitable deep learning architecture (such as the Transformer architecture and its variants) to build a large language model, and set the model's hyperparameters, including the number of layers, number of hidden units, number of attention heads, etc.

[0044] The preprocessed corpus data is used to conduct supervised training on the built model. The previous sentence in the text is taken as input and the corresponding next sentence is taken as the expected output. The parameters of the model are continuously adjusted through optimization algorithms (such as stochastic gradient descent and its improved algorithms, etc.) so that the model can learn the semantic association, language logic and other rules between the previous and next sentences until the model reaches the expected accuracy and performance indicators on the verification set and test set.

[0045] After obtaining the trained large language model, the original dictionary of the embedding layer is pruned based on the retained data on the basis of the large language model. The specific process of pruned original dictionary of the embedding layer is as follows: First, get the original dictionary in the embedding layer of the large language model. It should be noted that the embedding layer is the first layer in the large language model. Its main function is to convert the input text (such as words, subwords, characters, etc.) into encoded vectors. These vectors can capture the semantic and syntactic information of the terms in the text.

[0046] The vocabulary of the embedding layer refers to the collection of all unique terms used in the embedding layer of the large language model, and each term corresponds to a unique embedding vector. The embedding vector is implemented based on encoding the terms. The vocabulary contains all terms that the model can understand and generate. Terms can be words, characters, or BytePair Encoding (BPE), etc. The construction of the dictionary is usually based on a large amount of text data, and statistical methods or rule-based methods are used to determine which terms should be included in the dictionary.

[0047] After obtaining the original dictionary in the embedding layer of the large language model, the target language entries and their corresponding byte order split data in the original dictionary are extracted. The target language can be any specific language type, such as English, Chinese, Spanish, French, Arabic, Russian, etc.

[0048] To extract the entries of the target language from the original dictionary, the original dictionary may be decoded first, and then the entries of various languages ​​obtained after decoding the original dictionary may be screened to determine the entries of the target language.

[0049] It should be noted that the original dictionary in the embedding layer of the large language model is realized by encoding the entries of various languages. Specifically, it can be implemented based on the BPE algorithm. The BPE algorithm is used to split text data into smaller units (sub-words) and generate embedding vectors by encoding these units. The encoded embedding vectors can capture the semantic and syntactic information of word units and provide input for subsequent natural language processing tasks.

[0050] After determining the entries of the target language, based on the encoded data corresponding to the entries of the target language, split it to determine the byte-order split data corresponding to the entries of the target language.

[0051] For example, in the case where the encoding method is the BPE algorithm. Suppose an original dictionary contains the Chinese entry "你好" (Hello), and the encoded data corresponding to this entry is assumed to be "abcdef", which is composed of the Chinese character "你" (you) and the Chinese character "好" (good). Among them, the encoded data corresponding to the Chinese character "你" is "abc", and the encoded data corresponding to the Chinese character "好" is "def".

[0052] In the original dictionary, in addition to the encoded data "abcdef" corresponding to the entry "你好", there are also byte-order split data "abc", "def", "a", "bc", "b", "c", "d", "ef", "e", "f"; among them, "abc" and "def" are the split data of "abcdef"; "a" and "bc" are the split data of "abc"; "b" and "c" are the split data of "bc"; "e" and "f" are the split data of "ef".

[0053] Since the large language model will encode the input text and match the corresponding encoding byte by byte in the dictionary of the embedding layer before predicting the input text, it is necessary to retain the byte-order split data corresponding to the entries, which can enable the model to process continuous symbol sequences and facilitate the subsequent recognition process of the large language model for the input text.

[0054] Based on the extracted entries of the target language and the byte-order split data corresponding to the entries of the target language, delete the data in the original dictionary except for the entries of the target language and the byte-order split data corresponding to the entries of the target language, which can realize the trimming of the original dictionary in the embedding layer and obtain the trimmed dictionary.

[0055] After trimming the original dictionary in the embedding layer of the large language model, based on the trimmed dictionary, a new improved embedding layer can be obtained. Replacing the original embedding layer in the large language model with the improved new embedding layer can obtain the following prediction model.

[0056] It is understandable that the amount of data in the following prediction model is greatly reduced based on the pruning of the original dictionary. For example, if the original dictionary capacity is 10K and the dimension is 1024, then the original embedding layer size is [10000, 1024]. Based on the original dictionary, the new pruning dictionary capacity is reduced to 5K, and the dimension is still 1024, then the new embedding layer size is [5000, 1024], which greatly reduces the amount of data.

[0057] After obtaining the context prediction model, the context prediction model is trained based on the previous sample data and its corresponding context labels to obtain the trained context prediction model. The trained context prediction model is integrated into the application of the terminal device (such as the input method application, etc.), and deployed locally on the terminal device so that it can be directly called.

[0058] Furthermore, after obtaining the context data input into the terminal device in step 110, the obtained context data is input into the trained context prediction model deployed in the terminal device to implement the prediction process of the context data and obtain the context prediction result output by the context prediction model.

[0059] Specifically, when a user uses an input method to input the preceding data on a terminal device, the terminal device transmits the preceding data to a locally deployed following prediction model. The following prediction model analyzes and calculates the preceding data based on the language rules and semantic relationships it has learned, and outputs the following prediction results locally. These following prediction results can be returned to the input method interface and displayed to the user for selection.

[0060] It should be noted that the following prediction results obtained by improving the large language model are deployed in the terminal device to predict the previous data entered in the terminal device and determine the following prediction results corresponding to the previous data. It can provide users with instant following suggestions or automatic completion to improve the fluency of interaction. Users can get prediction prompts when inputting, reducing the amount of typing, especially on mobile devices.

[0061] In chat apps or communication software, quick context predictions can help users write messages faster. In search, writing and other scenarios, prediction results can save users time in finding the right words or phrases.

[0062] Furthermore, by setting the context prediction model in the terminal device, users can still use the prediction function even when the network connection is unstable or unavailable. Reducing dependence on cloud services can reduce operating costs, especially for data processing of a large number of users. The local context prediction model can be easily updated or expanded as needed to adapt to different application scenarios and needs. Reducing dependence on external services can improve the stability and reliability of the system and reduce the risk of interruption caused by external service failures.

[0063] In addition, it is understandable that, when a user performs input based on a terminal device, the input process is generally based on the same language.

[0064] The original dictionary of the embedding layer in the large language model in the related method supports entries corresponding to multiple languages. The amount of data is huge. If the large language model is directly set in the terminal device and used for the prediction process of the above data, it will affect the performance, and the performance is obviously insufficient in scenarios with high real-time requirements (such as instant messaging quick replies). In addition, by deploying the following prediction model in the terminal device, the input text data is only processed on the local terminal device and does not need to be uploaded to the cloud server, which greatly reduces the risk of privacy leakage and ensures the security of users' personal information and input content.

[0065] For terminal devices, memory resources are limited, and a large dictionary will take up a lot of memory, which may cause the terminal device to run slowly or even crash. Large language models need to query and update the vectors of the embedding layer when processing input. The larger the dictionary, the longer the query and calculation time. Frequent dictionary queries and vector calculations will increase the energy consumption of terminal devices and affect the processing speed of terminal devices.

[0066] Because the terminal device mainly serves users of a specific language, the entries in other languages ​​can be removed. Based on the original dictionary of the embedding layer in the large language model, the model is pruned. The amount of calculation required to process each input is reduced, thereby speeding up the prediction. The pruned dictionary reduces memory usage, allowing the model to run on terminal devices with limited resources. The smaller model size makes the model easier to deploy on a variety of terminal devices.

[0067] The trimmed dictionary reduces the memory requirements of the following prediction model, allowing the following prediction model to run on terminal devices with limited memory. The following prediction model is more lightweight and can be easily migrated and deployed on different hardware platforms. At the same time, the amount of calculation required for each prediction is reduced, the processing speed is improved, and the real-time requirements are met. In scenarios with high real-time requirements such as instant messaging, the model's rapid response capability is improved.

[0068] The method for predicting the following data provided by the present invention obtains a following prediction model by trimming the original dictionary in the embedding layer based on a large language model. A lightweight design of the following prediction model is achieved. The lightweight following prediction model is deployed on a terminal device, and while realizing the following prediction process for the preceding data of the terminal device, it can run efficiently under the limited hardware resources of the terminal device. The preceding data input into the terminal device is only processed in the local terminal device, without uploading to the cloud server, which greatly reduces the risk of privacy leakage and ensures the personal information security and input content security of users in the terminal device.

[0069] In one embodiment, the process of determining the retained data includes: based on the target language, screening multiple entries in the original dictionary to determine the entries in the target language in the original dictionary; splitting the encoded data corresponding to the entries in the target language to determine the byte order split data corresponding to the entries in the target language.

[0070] It should be noted that the original dictionary contains entries in multiple languages. After the target language is determined, the entries related to the target language in the original dictionary can be identified and obtained.

[0071] Based on identifying and acquiring the entries related to the target language in the original dictionary, it is possible to screen multiple entries in the original dictionary and determine the entries in the target language in the original dictionary.

[0072] After determining the entry of the target language, it is necessary to further split the encoding data corresponding to the entry of the target language to determine the byte order split data corresponding to the entry. For example, the encoded data corresponding to the entry "hello" is "abcdef". Based on the splitting of the encoded data "abcdef", the byte order split data "abc", "def", "a", "bc", "b", "c", "d", "ef", "e", "f" can be obtained; among which "abc" and "def" are the split data of "abcdef"; "a" and "bc" are the split data of "abc"; "b" and "c" are the split data of "bc"; "e" and "f" are the split data of "ef".

[0073] In one embodiment, based on the target language, multiple entries in the original dictionary are screened to determine the entries in the target language in the original dictionary, including: decoding the original dictionary based on a byte pair encoding (BPE) algorithm to obtain a decoded dictionary; and recognizing the data in the target language in the decoded dictionary to obtain the entries in the target language.

[0074] It is understandable that the original dictionary contains coded data, which is not conducive to reading. It is impossible to identify and filter the target language data contained therein directly based on the coded data.

[0075] Specifically, the original dictionary can be decoded based on the BPE algorithm to obtain a decoded dictionary, and the encoded entries in the original dictionary can be transcoded into readable entries. The obtained decoded dictionary contains readable entries in multiple languages, and then the entries in the target language that are desired to be retained are selected from all the entries.

[0076] The process of decoding the original dictionary and filtering the target language entries based on the BPE algorithm can improve readability, support multi-language recognition, and accurately filter the target language entries, thereby improving processing efficiency.

[0077] In one embodiment, when the target language is Chinese, the data of the target language in the decoding dictionary is recognized to obtain the entries of the target language, including: matching Chinese characters in the decoding dictionary based on regular expressions to recognize the Chinese data in the decoding dictionary; extracting the Chinese data recognized in the decoding dictionary to obtain the entries of the Chinese language.

[0078] A regular expression is a pattern used to match character combinations in a string. For Chinese language recognition, you can define a regular expression to match Chinese characters, because Chinese characters are usually within a certain range of the Unicode character set.

[0079] Use the defined regular expression to traverse each entry in the decoding dictionary. Apply the regular expression to each entry to check whether it contains Chinese characters.

[0080] After identifying the entries containing Chinese characters, these entries need to be further extracted for the subsequent original dictionary pruning process.

[0081] Optionally, the pruning process for the original dictionary may be as follows: first, the original IDs of the Chinese terms in the original dictionary and the original IDs of the byte-sequence split data corresponding to the Chinese terms need to be recorded. Figure 2 The flowchart of the term identification and retention provided by the present invention is shown as follows. Based on the BPE algorithm, the original dictionary is decoded to obtain a decoded dictionary. It is necessary to retain the Chinese terms in all vocab terms in the original dictionary and determine the original ID of the Chinese terms. The encoded data corresponding to the Chinese terms is split, and the ID of the byte order split data corresponding to the Chinese terms is determined in the decoded dictionary.

[0082] Further, such as Figure 3As shown in the dictionary clipping diagram provided by the present invention, the original ids of the Chinese entries and the original ids of the byte-sequence split data corresponding to the Chinese entries are recorded, the data except the recorded ids in the original dictionary are deleted, and the ids in the deleted original dictionary are rearranged in order to obtain a clipped dictionary. For example, the original dictionary capacity is 10K, the dimension is 1024, and the original embedding layer size is [10000, 1024]. Based on the original dictionary, the improvement is made to obtain a new clipped dictionary with a capacity reduced to 5K and a dimension still 1024, and the new embedding layer size is [5000, 1024], which greatly reduces the amount of data.

[0083] In one embodiment, after obtaining the context prediction result output by the context prediction model, it also includes: determining multiple context prediction data included in the context prediction result; and returning the multiple context prediction data to the input interface of the terminal device.

[0084] It is understandable that for the following prediction model, a series of prediction results are usually output, and the prediction results include multiple following prediction data and their corresponding probabilities. After determining multiple following prediction data, the multiple following prediction data are returned to the input interface of the terminal device.

[0085] Specifically, a drop-down list can be displayed next to the user input box on the front-end input interface, listing the predicted options. When the user enters input on the input interface of the terminal device, multiple predicted data for the following content are automatically displayed as suggested completions. Specifically, a pop-up window or prompt box can be used to display the predicted data.

[0086] Users can quickly complete input by selecting prediction results and provide multiple following prediction data as candidate supplementary data, which can not only improve user productivity and satisfaction, but also help better understand and serve users.

[0087] In one embodiment, the multiple context prediction data are returned to the input interface of the terminal device, including: determining the usage frequency of each context prediction data based on the historical context data input into the terminal device, and determining the sorting priority of the multiple context prediction data based on the usage frequency; sorting the multiple context prediction data based on the sorting priority, and displaying the sorted context prediction data on the input interface of the terminal device.

[0088] Collect historical context data of user input in the terminal device. The historical context data may specifically include the user's previous conversations, text input, search history, etc.

[0089] Perform statistical analysis on the collected historical context data to determine the usage frequency of each context prediction data. Based on the usage frequency, assign a sorting priority to each context prediction data. The higher the usage frequency, the higher the priority; the lower the usage frequency, the lower the priority.

[0090] The input interface is dynamically updated, and the sorted predicted data is displayed in a list, drop-down menu or other form on the input interface of the terminal device. Users can select one or more of the sorted predicted data to supplement their input according to their needs, and the system can also provide an automatic completion function.

[0091] In one embodiment, before the above context data is input into the following context prediction model deployed on the terminal device, it also includes: quantizing the model parameters of the following context prediction model from a first precision to a second precision, and the numerical range of the first precision is larger than the numerical range of the second precision.

[0092] Precision refers to the number of significant digits in a numerical representation. The numerical range of precision refers to the range of significant digits that the precision type can represent. For example, single-precision floating point numbers (float32) usually have 7 to 8 decimal digits of precision, while double-precision floating point numbers (float64) have 15 to 17 decimal digits of precision. The numerical range of the first precision is larger than the numerical range of the second precision, which reflects that the first precision contains more significant digits than the second precision.

[0093] It should be noted that large language models usually contain hundreds of millions of model parameters, which are usually stored in a format with higher data precision (such as floating point precision float32). High-precision format storage not only requires a large amount of storage space, but also consumes a lot of computing resources when performing subsequent model predictions.

[0094] When it is determined that the model parameters of the following prediction model are of the first precision, the precision of the model parameters of the following prediction model is reduced, and the model parameters of the following prediction model are quantized from the first precision with a larger numerical range to the second precision with a smaller numerical range, thereby realizing a lightweight process of the following prediction model.

[0095] Specifically, the first precision (such as float32 precision) of the model parameters in the following prediction model can be converted to the second precision (such as int8 precision) of a low numerical range. Among them, float32 represents a 32-bit floating point number, which can represent very precise but relatively small numerical values; and int8 represents an 8-bit signed integer, whose numerical range and precision are much lower than float32, but the required storage space is smaller. By quantizing the model parameters of the following prediction model to the integer int8, the storage requirements and computational complexity of the following prediction model can be significantly reduced, thereby speeding up the inference speed and reducing the running cost.

[0096] It is understandable that integer data takes up much less space than floating-point data, so quantization can significantly reduce the size of the following prediction model, making it easier to deploy on terminal devices with limited storage space. In addition, integer operations are usually faster than floating-point operations, which can speed up the reasoning process of the following prediction model and reduce energy consumption. The quantized model can achieve significant improvements in storage and computing efficiency while maintaining similar performance, allowing the following prediction model to run more smoothly on resource-constrained terminal devices.

[0097] The following is a description of a device for predicting subsequent data provided by the present invention. The device for predicting subsequent data described below and the method for predicting subsequent data described above can be referenced to each other.

[0098] like Figure 4 As shown, the device comprises: A data acquisition module 410 is used to acquire the above data; Prediction module 420, used for inputting the above context data into a context prediction model deployed on a terminal device to obtain a context prediction result output by the context prediction model, wherein the context prediction model is trained based on the above context sample data and context labels corresponding to the above context sample data; The following context prediction model is obtained by trimming the original dictionary of the embedding layer based on the retained data on the basis of the large language model, and the retained data includes the entries of the target language and the byte order splitting data corresponding to the entries of the target language.

[0099] The device for predicting the following data provided by the present invention obtains a following prediction model by cutting the original dictionary in the embedding layer based on a large language model. A lightweight design of the following prediction model is achieved. The lightweight following prediction model is deployed in a terminal device, and while realizing the following prediction process for the preceding data of the terminal device, it can run efficiently under the limited hardware resources of the terminal device. The preceding data input into the terminal device is only processed in the local terminal device, without uploading to the cloud server, which greatly reduces the risk of privacy leakage and ensures the personal information security and input content security of users in the terminal device.

[0100] In one embodiment, the prediction module 420 is specifically used to: The process of determining the retained data includes: Based on the target language, multiple entries in the original dictionary are screened to determine the entries in the target language in the original dictionary; The coded data corresponding to the entry of the target language is split to determine the byte order split data corresponding to the entry of the target language.

[0101] In one embodiment, the prediction module 420 is further configured to: Based on the target language, screening the multiple entries in the original dictionary to determine the entries in the target language in the original dictionary includes: Based on the byte pair encoding BPE algorithm, the original dictionary is decoded to obtain a decoded dictionary; The data of the target language in the decoding dictionary is recognized to obtain the entries of the target language.

[0102] In one embodiment, the prediction module 420 is further configured to: In the case where the target language is Chinese, identifying the data of the target language in the decoding dictionary to obtain the entry of the target language includes: Based on regular expressions, matching Chinese characters in the decoding dictionary to identify Chinese data in the decoding dictionary; The Chinese data recognized in the decoding dictionary is extracted to obtain entries in the Chinese language.

[0103] In one embodiment, the prediction module 420 is further configured to: After obtaining the following prediction result output by the following prediction model, the method further includes: Determining a plurality of following prediction data included in the following prediction result; The multiple subsequent prediction data are returned to the input interface of the terminal device.

[0104] In one embodiment, the prediction module 420 is further configured to: The returning the plurality of following prediction data to the input interface of the terminal device includes: Determining the usage frequency of each context prediction data based on the historical context data input into the terminal device, and determining the sorting priority of the multiple context prediction data based on the usage frequency; Based on the sorting priority, the plurality of subsequent prediction data are sorted, and the sorted subsequent prediction data are displayed on an input interface of the terminal device.

[0105] In one embodiment, the data acquisition module 410 is specifically used for: Before inputting the above context data into the context prediction model deployed in the terminal device, the method further includes: The model parameters of the following prediction model are quantized from a first precision to a second precision, wherein a numerical range of the first precision is greater than a numerical range of the second precision.

[0106] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communications interface 520 and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the prediction method of the following data, the method comprising: obtaining the above data; Inputting the above context data into a context prediction model deployed on a terminal device to obtain a context prediction result output by the context prediction model, wherein the context prediction model is trained based on the above context sample data and context labels corresponding to the above sample data; The following context prediction model is obtained by trimming the original dictionary of the embedding layer based on the retained data on the basis of the large language model, and the retained data includes the entries of the target language and the byte order splitting data corresponding to the entries of the target language.

[0107] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0108] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the method for predicting the following data provided by the above methods, the method comprising: obtaining the above data; Inputting the above context data into a context prediction model deployed on a terminal device to obtain a context prediction result output by the context prediction model, wherein the context prediction model is trained based on the above context sample data and context labels corresponding to the above sample data; The following context prediction model is obtained by trimming the original dictionary of the embedding layer based on the retained data on the basis of the large language model, and the retained data includes the entries of the target language and the byte order splitting data corresponding to the entries of the target language.

[0109] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for predicting the following data provided by the above methods is implemented, and the method comprises: obtaining the above data; Inputting the above context data into a context prediction model deployed in a terminal device to obtain a context prediction result output by the context prediction model, wherein the context prediction model is trained based on the above context sample data and context labels corresponding to the above sample data; The following context prediction model is obtained by trimming the original dictionary of the embedding layer based on the retained data on the basis of the large language model, and the retained data includes the entries of the target language and the byte order splitting data corresponding to the entries of the target language.

[0110] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0111] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting the following data, characterized in that: include: Get the above data; Inputting the above context data into a context prediction model deployed on a terminal device to obtain a context prediction result output by the context prediction model, wherein the context prediction model is trained based on the above context sample data and context labels corresponding to the above sample data; The following context prediction model is obtained by trimming the original dictionary of the embedding layer based on the retained data on the basis of the large language model, and the retained data includes the entries of the target language and the byte order splitting data corresponding to the entries of the target language.

2. The method for predicting the following data according to claim 1, characterized in that: The process of determining the retained data includes: Based on the target language, multiple entries in the original dictionary are screened to determine the entries in the target language in the original dictionary; The coded data corresponding to the entry of the target language is split to determine the byte order split data corresponding to the entry of the target language.

3. The method for predicting the following data according to claim 2, characterized in that: The step of screening multiple entries in the original dictionary based on the target language to determine the entries in the target language in the original dictionary includes: Based on the byte pair encoding BPE algorithm, the original dictionary is decoded to obtain a decoded dictionary; The data of the target language in the decoding dictionary is recognized to obtain the entries of the target language.

4. The method for predicting the following data according to claim 3, characterized in that: In the case where the target language is Chinese, identifying the data of the target language in the decoding dictionary to obtain the entry of the target language includes: Based on regular expressions, matching Chinese characters in the decoding dictionary to identify Chinese data in the decoding dictionary; The Chinese data recognized in the decoding dictionary is extracted to obtain entries in the Chinese language.

5. The method for predicting the following data according to claim 1, characterized in that: After obtaining the following prediction result output by the following prediction model, the method further includes: Determining a plurality of following prediction data included in the following prediction result; The multiple subsequent prediction data are returned to the input interface of the terminal device.

6. The method for predicting the following data according to claim 5, characterized in that: The returning the plurality of following prediction data to the input interface of the terminal device includes: Determining the usage frequency of each context prediction data based on the historical context data input into the terminal device, and determining the sorting priority of the multiple context prediction data based on the usage frequency; Based on the sorting priority, the plurality of subsequent prediction data are sorted, and the sorted subsequent prediction data are displayed on an input interface of the terminal device.

7. The method for predicting the following data according to claim 1, characterized in that: Before inputting the above context data into the context prediction model deployed in the terminal device, the method further includes: The model parameters of the following prediction model are quantized from a first precision to a second precision, wherein a numerical range of the first precision is greater than a numerical range of the second precision.

8. A device for predicting following data, characterized in that: include: A data acquisition module, used to acquire the above data; A prediction module, used for inputting the above context data into a context prediction model deployed on a terminal device to obtain a context prediction result output by the context prediction model, wherein the context prediction model is trained based on the above context sample data and context labels corresponding to the above context sample data; The following context prediction model is obtained by trimming the original dictionary of the embedding layer based on the retained data on the basis of the large language model, and the retained data includes the entries of the target language and the byte order splitting data corresponding to the entries of the target language.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the method for predicting the following data according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for predicting the following data according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for predicting the following data according to any one of claims 1 to 7 is implemented.

Images