A text input information processing method, apparatus and storage medium

Abstract

This application discloses a text input information processing method, apparatus, and storage medium. The method involves acquiring user text input information, including complete words already entered by the user and / or characters representing incomplete words currently being entered by the user; generating a first semantic feature based on the text input information to characterize the semantics of the text input information; wherein the first semantic feature further characterizes the positional information of complete words and / or input characters in the text input information, the positional information including word-level positional information corresponding to words and / or character-level positional information corresponding to characters; and generating output information corresponding to the text input information based on the first semantic feature, wherein the output information includes predicted words corresponding to the text input information. This method improves the accuracy of prediction results corresponding to the text input information and enhances compatibility with different types of prediction tasks, thereby improving applicability.

Description

Technical Field

[0001] This application relates to the field of input method information processing technology, and in particular to a text input information processing method, apparatus and storage medium. Background Technology

[0002] As a user inputs characters or words, the Latin input method program needs to predict the user's input intention. For example, predicting the complete word the user might input after entering a character, or predicting the next word the user might input after entering a complete word.

[0003] In existing technologies, statistical language modeling methods are typically used to predict user input intent, including n-gram models and tree matching methods.

[0004] Tree matching methods are primarily used in scenarios where a user might input a complete word based on the incomplete words they are currently typing (a scenario known as word completion). This method constructs a dictionary based on a trie, mapping the user's input character sequence to all possible matching paths in the dictionary. For example, when the input is "r", the system will search for all words starting with "r", such as "row", "river", and "right", and then use a ranking algorithm (such as word frequency or context probability) to select the most relevant words.

[0005] The n-gram model is primarily used in scenarios where a user might input the next word based on a set of complete words already entered (referred to as word prediction). This method achieves prediction by statistically analyzing the probability of occurrence of fixed-length phrases in the training corpus. For example, if the triplet form "ducks in" frequently appears in the training corpus when a user inputs "ducks in," then the model tends to predict that the next word the user needs to input is "a."

[0006] Existing methods for predicting user input intent have the following problems:

[0007] 1) The prediction methods are limited in functionality. For example, tree matching methods can only generate predicted words if the user's text input includes characters corresponding to incomplete words; while n-gram models can only generate predicted words if the user's text input contains only complete words. Therefore, if you want to generate predicted words under both different text input conditions, you need to deploy programs for both methods simultaneously, which brings inconvenience to the application.

[0008] 2) Tree matching methods are highly dependent on the coverage of the dictionary, making it difficult to reasonably predict words not found in the dictionary or newly formed words. Furthermore, their prediction performance is poor when users input rapidly and continuously or when the language context changes. Therefore, this method has poor applicability and low prediction accuracy.

[0009] 3) When predicting, n-gram models predict the probability of word sequences with a fixed number of words in the training corpus. Because they rely only on the context of a limited window, they struggle to capture long-distance semantic relationships. In scenarios with sparse training data or high combinatorial diversity, n-gram models are easily limited by the data coverage and have low generalization ability.

[0010] To address the technical problems of existing input method methods for predicting user input intent, such as limited functionality, poor applicability and low accuracy of word completion methods, and poor generalization ability of word prediction methods, no effective solution has yet been proposed. Summary of the Invention

[0011] The embodiments of this disclosure provide a text input information processing method, apparatus, and storage medium to at least solve the technical problems of poor applicability and low accuracy of existing methods for predicting user input intent in input methods.

[0012] According to one aspect of the present disclosure, a text input information processing method is provided, comprising: acquiring text input information of a user, the text input information including complete words already entered by the user and / or input characters of incomplete words currently being entered by the user; generating a first semantic feature for characterizing the semantics of the text input information based on the text input information, wherein the first semantic feature is further used to characterize position information of complete words and / or input characters in the text input information, the position information including word-level position information corresponding to the words and character-level position information corresponding to the characters; and generating output information corresponding to the text input information using a word prediction layer based on the first semantic feature, wherein the output information includes predicted words corresponding to the text input information.

[0013] According to another aspect of the present disclosure, a storage medium is also provided, the storage medium including a stored program, wherein, when the program is executed, a processor performs any of the methods described above.

[0014] According to another aspect of the present disclosure, a text input information processing apparatus is also provided, comprising: an acquisition module, configured to acquire text input information of a user, the text input information including complete words already entered by the user and / or input characters of incomplete words currently being entered by the user; a feature generation module, configured to generate a first semantic feature for characterizing the semantics of the text input information based on the text input information, wherein the first semantic feature is further configured to characterize the position information of complete words and / or input characters in the text input information, the position information including word-level position information corresponding to the words and character-level position information corresponding to the characters; and an information generation module, configured to generate output information corresponding to the text input information using a word prediction layer based on the first semantic feature, wherein the output information includes predicted words corresponding to the text input information.

[0015] According to another aspect of the present disclosure, a text input information processing apparatus is also provided, comprising: a processor; and a memory connected to the processor, configured to provide the processor with instructions to process the following processing steps: acquiring text input information from a user, the text input information including complete words already entered by the user and / or input characters of incomplete words currently being entered by the user; generating a first semantic feature based on the text input information to characterize the semantics of the text input information, wherein the first semantic feature is further configured to characterize positional information of complete words and / or input characters in the text input information, the positional information including word-level positional information corresponding to words and / or character-level positional information corresponding to characters; and generating output information corresponding to the text input information using a word prediction layer based on the first semantic feature, wherein the output information includes predicted words corresponding to the text input information.

[0016] In this embodiment, the method can distinguish text input information into complete words and individual input characters in incomplete words for feature extraction. During feature extraction, not only the semantics of complete words and individual input characters in the text input information are considered, but also the dual positional information (lexical-level positional information and character-level positional information) of complete words and input characters. This approach improves prediction accuracy by predicting the information the user will need to input next. Furthermore, compared to tree matching and n-gram methods, it is compatible with both word completion and word prediction scenarios, making it more versatile. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this application, illustrate exemplary embodiments of this disclosure and are used to explain this disclosure, but do not constitute an undue limitation of this disclosure. In the drawings:

[0018] Figure 1This is a hardware structure block diagram of a computing device for implementing the method described in Embodiment 1 of this disclosure;

[0019] Figure 2 This is a schematic flowchart of a text input information processing method according to the first aspect of Embodiment 1 of this disclosure;

[0020] Figure 3A This is a schematic diagram of the structure of a vocabulary prediction model for processing text input information according to Embodiment 1 of this disclosure;

[0021] Figure 3B yes Figure 3A The diagram shows the structure of the joint coding layer in the vocabulary prediction model.

[0022] Figure 4 This is a schematic diagram of a text input information processing apparatus according to the first aspect of Embodiment 2 of this disclosure; and

[0023] Figure 5 This is a schematic diagram of a text input information processing apparatus according to the first aspect of Embodiment 3 of this disclosure. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] Example 1

[0027] According to this embodiment, a method embodiment for processing text input information is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0028] The method embodiments provided in this example can be executed on mobile terminals, computer terminals, servers, or similar computing devices. Figure 1 A hardware block diagram of a computing device for implementing a text input information processing method is shown. Figure 1 As shown, a computing device may include one or more processors (processors may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, transmission device, and input / output interface are connected to the processor via a bus. In addition, it may also include a display, keyboard, and cursor control device connected to the input / output interface. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, a computing device may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0029] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element in a computing device. As involved in the embodiments of this disclosure, the data processing circuits serve as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0030] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the text input information processing method in the embodiments of this disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the text input information processing method of the aforementioned application. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the computing device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0031] The transmission device is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the computing device's communication provider. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0032] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows users to interact with the user interface of the computing device.

[0033] It should be noted here that, in some optional embodiments, the above... Figure 1 The computing device shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1 This is only one instance of a specific particular instance, and is intended to illustrate the types of components that may exist in the aforementioned computing devices.

[0034] Under the aforementioned operating environment, according to the first aspect of this embodiment, a text input information processing method is provided, which can be... Figure 1 The computing device implementation is shown. Figure 2 A flowchart illustrating the method is shown below. (Refer to...) Figure 2 As shown, the method includes:

[0035] S202: Obtain the user's text input information, which includes complete words that the user has already entered and / or input characters of incomplete words that the user is currently entering;

[0036] S204: Based on the text input information, generate a first semantic feature to characterize the semantics of the text input information, wherein the first semantic feature is further used to characterize the positional information of complete words and / or input characters in the text input information, the positional information including word-level positional information corresponding to words and character-level positional information corresponding to characters; and

[0037] S206: Based on the first semantic feature, use the vocabulary prediction layer to generate output information corresponding to the text input information, wherein the output information includes the predicted vocabulary corresponding to the text input information.

[0038] Specifically, the computing device can acquire the user's text input information, which includes the complete words that the user has already entered and / or the input characters of the incomplete words that the user is currently entering (S202).

[0039] The text input information mentioned here may contain only complete words that the user has already entered; or only characters of incomplete words that the user is currently entering; or both complete words that the user has already entered and characters of incomplete words that the user is currently entering. Here, an input character refers to a single character that can form a complete word.

[0040] The input method mentioned here can be a Latin input method. For ease of description, examples will be given using Latin input methods below. Of course, this method can also be applied to input methods of other language systems, such as English input methods.

[0041] For example, if a user ultimately needs to input "Ducks in a row", when the user inputs "Du", "Du" becomes the text input information, which only contains the input characters "D" and "u" of the incomplete word "Du". When the user inputs "Ducks in a", the text input information "Ducks in a" contains only the complete words, namely "Ducks", "in", and "a". When the user inputs "Ducks in ar", the text input information "Ducks in ar" contains both the complete words "Ducks", "in", and "a", as well as the input character "r" of the incomplete word.

[0042] The computing device can use the input method to determine which characters in the text input information are complete words and which are input characters corresponding to incomplete words. For several characters in the text input information, if the input method determines that the characters are a completed word, the computing device determines that the characters are a complete word; otherwise, if the input method has not yet determined that the characters are a completed word, the computing device determines that the characters are input characters corresponding to incomplete words.

[0043] For example, in a Latin input method, if a user types several characters and then presses the spacebar, the input method can determine that the user has completed the input of a word, and the computing device can then recognize those characters as a complete word that the user has entered. Conversely, if the user types several characters but does not press the spacebar, the input method cannot determine that the user has completed the input of a word, and the computing device can then recognize those characters as input characters for an incomplete word that the user is currently typing.

[0044] The computing device can generate a first semantic feature (S204) to characterize the semantics of the text input information based on the text input information. The first semantic feature is also used to characterize the position information of complete words and / or input characters in the text input information. The position information includes word-level position information corresponding to words and / or character-level position information corresponding to characters.

[0045] The lexical-level positional information is used to indicate which word in the text input information a complete word and / or input character belongs to. For example, in the case of the text input information "Ducks in a ro", the lexical-level positional information is used to indicate that "Ducks" is the first word in the text input information, "in" is the second word, "a" is the third word, and "r" and "o" belong to the fourth word.

[0046] Furthermore, character-level positional information is used to indicate the position of a complete word and / or input character relative to the incomplete word the user is typing. For example, in the case of the text input "Ducks in aro", since the incomplete word is "ro", the character-level positional information is used to indicate that "Ducks", "in", and "a" are not part of the incomplete word, to indicate that "r" is the first character of the incomplete word, and "o" is the second character of the incomplete word.

[0047] In other words, the first semantic feature can not only characterize the semantic features of the text input information, but also the positional features of the complete words and input characters contained in the text input information. These positional features reflect the word-level and character-level positional information of the complete words and input characters. The specific details of the first semantic feature will be explained in detail later.

[0048] Finally, the computing device can generate output information corresponding to the text input information based on the first semantic features, wherein the output information includes predicted words corresponding to the text input information (S206).

[0049] In cases where the user's text input consists entirely of complete words, the predicted word is generated based on these complete words. For example, if the text input is "Ducks in a" (containing only the complete word), the computing device can generate the predicted word "row" based on this text input, and the output can be "row". This corresponds to the word prediction scenario described in the background section.

[0050] Furthermore, when the user's text input includes both complete words and incomplete words, the predicted word is generated based on both the complete words and the incomplete words. For example, if the text input is "Ducks in a ro", the computing device can generate the predicted word "row" based on this text input, and the output information can be "row". This corresponds to the word completion scenario described in the background art.

[0051] It should be noted that although the output information matches the predicted vocabulary in the example above, there are cases where the output information may differ from the predicted vocabulary. Of course, the output information may not only contain the predicted vocabulary; it can be a phrase or short sentence that includes the predicted vocabulary. For example, if the text input is "Ducks i" and the predicted vocabulary is "in", the output information could be "Ducks in".

[0052] Furthermore, although the above explanation uses word prediction and word completion scenarios as examples, this embodiment can also satisfy both scenarios simultaneously. For example, if the text input is "Ducks i", the predicted word could also be "in a row", and the output could also be "Ducks in a row". Thus, this embodiment can further complete word prediction on the basis of word completion.

[0053] As described in the background section, existing input methods have limited functionality in user prediction. For example, tree matching methods can only generate predicted words if the user's text input includes characters corresponding to incomplete words; while n-gram models can only generate predicted words if the user's text input consists entirely of complete words. Therefore, to generate predicted words under both different text input conditions, programs using both methods need to be deployed simultaneously, which introduces inconvenience in application.

[0054] In view of this, the technical solution of this application, after extracting features from the user's text input information, generates semantic features that not only represent the semantics of complete words and each input character in the text input information, but also represent the dual positional information (vocabulary-level positional information and character-level positional information) of complete words and input characters. Thus, the technical solution of this application can perform word prediction for both cases where the text input information includes input characters corresponding to incomplete words and where the text input information consists entirely of complete words, using only a single word prediction model. This solves the technical problem of the limited functionality of existing input methods for predicting user input intent.

[0055] Optionally, when the text input information only includes complete words, the predicted word includes the next complete word relative to the last complete word in the text input information; and when the text input information includes input characters that are not complete words, the predicted word includes the complete word corresponding to the input character.

[0056] Therefore, the vocabulary prediction model described in this application can be applied to both word completion scenarios corresponding to tree matching methods and word prediction scenarios corresponding to n-gram models. Compared to tree matching methods and n-gram models, it is compatible with both word completion and word prediction scenarios, and has stronger applicability, thus solving the technical problem of the limited application scenarios of tree matching methods and n-gram models in the prior art.

[0057] Optionally, the operation of generating a first semantic feature to characterize the semantics of the text input information based on the text input information includes: generating a corresponding second semantic feature for each semantic unit in the text input information, wherein one semantic unit is a single complete word and / or a single input character of the text input information; determining positional features corresponding to each semantic unit, wherein the positional features are used to characterize the word-level positional information and character-level positional information of the corresponding semantic unit in the text input information; fusing the second semantic feature with the corresponding positional feature to obtain a corresponding third semantic feature; and determining the first semantic feature based on the third semantic feature.

[0058] Specifically, Figure 3A This diagram illustrates a vocabulary prediction model deployed on a computing device to process text input information. The vocabulary prediction model is, for example, a program module that can be executed by the computing device.

[0059] refer to Figure 3A As shown, the vocabulary prediction model can specifically include three layers: a joint encoding layer for generating third semantic features, a context modeling model for context modeling, and a vocabulary prediction layer for generating predicted words. Figure 3AThe joint coding layer in the model is mainly used to generate the third semantic features of the text input information. Specifically, this joint coding layer is primarily used to determine the second semantic features and positional features (including lexical positional encoding and character positional encoding mentioned below) of the semantic units in the text input information, and then fuse them to obtain the third semantic features. Subsequently, the third semantic features of each semantic unit can be input into the context modeling model to obtain the first semantic features. Furthermore, refer to... Figure 3B As shown, the joint coding layer further includes: a first embedding layer, a second embedding layer, a third embedding layer, and a fusion layer.

[0060] Thus, in the process of generating a first semantic feature to represent the semantics of the text input information based on the text input information, the computing device inputs the text device information. Figure 3A The vocabulary prediction model is shown. The joint encoding layer of the vocabulary prediction model then receives the text input information.

[0061] Therefore, refer to Figure 3B As shown, when extracting semantic features from text input information, the first embedding layer of the joint coding layer can take a complete word in the text input information as the smallest unit of feature extraction (i.e., the semantic unit mentioned above), or take an input character in a non-complete word in the text input information as the smallest unit of feature extraction (i.e., the semantic unit), and extract semantic features (second semantic features) for each smallest unit.

[0062] The second semantic feature is a feature that uniquely corresponds to different complete words or different input characters. For example, text input information can be represented as a set of complete words. W =[ w 1, w 2, ..., w t-1 ], a set of input characters for incomplete vocabulary C t =[ c 1, c 2, ..., c k Therefore, the first embedding layer can generate second semantic features corresponding to the complete vocabulary. E w ( w i )( i =1~ t -1), and generate a second semantic feature corresponding to the input character. E c ( c j )( j =1~ kThus, the second semantic features corresponding to the complete vocabulary. E w ( w i and the second semantic features corresponding to the input characters. E c ( c j Together, they constitute the second semantic feature corresponding to the text input information. E ( x m )( m =1~ t + k -1). The specific generation method of the second semantic feature will be explained in detail later.

[0063] Then, further reference Figure 3B As shown, the second and third embedding layers of the joint coding layer can be used to determine the positional features corresponding to each semantic unit. P m (This location feature) P m It can contain Figure 3B Lexical position features P w ( x m ) and character position features P c ( x m ), where the location feature P m It can be used to characterize the lexical-level and character-level positional information of corresponding semantic units in text input information (corresponding to lexical positional encoding and character positional encoding). Regarding positional features... P m The specific method for determining it will be explained in detail later.

[0064] Then, the fusion layer will incorporate the second semantic features. E ( x m ) and corresponding location features P m The two semantic features are then fused to obtain the corresponding third semantic feature. In this embodiment, for example, the second semantic feature can be fused together. E ( x m ) and corresponding location features P m The fusion is achieved by summing, thereby generating the corresponding third semantic features. X ( xm However, the methods for fusing second semantic features and positional features of the same semantic unit are not limited to this.

[0065] Then, refer to Figure 3A As shown, the context modeling model then uses the third semantic feature... X ( x m The corresponding first semantic feature is determined. The specific operation for determining the first semantic feature will be explained in detail later.

[0066] Therefore, in the technical solution of this application, for each semantic unit, a corresponding semantic feature (i.e., a second semantic feature) and a positional feature are generated. The second semantic feature characterizes the semantics of the semantic unit, while the positional feature characterizes the lexical-level and character-level positional information of the semantic unit. Thus, the third semantic feature generated by fusing the second semantic feature and the positional feature characterizes both the semantic information and the lexical-level and character-level positional information of the corresponding semantic unit. Furthermore, the first semantic feature determined based on the third semantic feature characterizes both the semantic information and the lexical-level and character-level positional information of the corresponding semantic unit. In this way, the vocabulary prediction model can perform more accurate vocabulary prediction based on the lexical-level and character-level positional information of each semantic unit for text input information.

[0067] Optionally, the operation of generating a corresponding second semantic feature for each semantic unit in the text input information includes: when the semantic unit is a single complete word, using the word embedding vector of the single complete word as the second semantic feature corresponding to the semantic unit; and when the semantic unit is a single input character, using the character embedding vector of the single input character as the second semantic feature corresponding to the semantic unit.

[0068] Specifically, refer to Figure 3B As shown, in generating the second semantic feature E ( x m In the process, the first embedding layer performs embedding operations on semantic units. When a semantic unit is a complete word, the second semantic feature of the semantic unit is the semantic feature generated by embedding the complete word itself. When a semantic unit is an input character in a non-complete word, the second semantic feature of the semantic unit is the semantic feature generated by embedding the input character itself.

[0069] Specifically, the first embedding layer can obtain the word embedding vector or character embedding vector of a semantic unit by looking up a table. x m It is a complete vocabulary. w i At that time, you can use the preset vocabulary list. v w Search for the complete vocabulary in [the search bar]. w i The index is used to retrieve the complete word by querying the index. w i Word embedding vectors E w ( w i )∈R d As the second semantic feature corresponding to this semantic unit E ( x m ),in d Represents the dimension of a vector.

[0070] Similarly, when semantic units x m It is the input character c j At that time, the first embedding layer can be in a preset character table. v c Query the input character c j The index is used to retrieve the input character. c j Character embedding vector E c ( c j )∈R d As the second semantic feature corresponding to this semantic unit E ( x m ).

[0071] Therefore, when the text input information includes a complete set of words... W =[ w 1, w 2, ..., w t-1 ] and incomplete vocabulary input character set C t =[ c 1, c 2, ..., c k When [the text input information is displayed], the second semantic feature sequence of each semantic unit corresponding to the text input information is:

[0072] E =[ E w ( w 1), ..., E w ( w t-1 ), E c ( c 1), ..., E c ( c 1)]∈ R (t-1+k)×d .

[0073] Preferably, a vocabulary list v w and character table v c These can be mapped to the same vector space. In this case, the computing device can calculate the distance between the word vectors of complete words and the character vectors of individual characters. Therefore, the same model can be used to perform both word prediction and word completion functions. For example, if the text input only contains complete words, the complete words can be mapped to this vector space for further operations corresponding to word prediction; if the text input includes both complete words and incomplete characters, the complete words and characters can be mapped to this vector space separately for further operations corresponding to word completion.

[0074] Furthermore, the computing device can predetermine the word embedding vectors of each word and the character embedding vectors of each character, thereby constructing a vocabulary and a character table. Therefore, when the first embedding layer determines the word embedding vectors of each word in the vocabulary and the character embedding vectors of each character in the character table, it can query and determine them in the corresponding vocabulary and character tables, which will not be elaborated further here.

[0075] Optionally, the operation of determining the positional features corresponding to each semantic unit includes: determining the word position code corresponding to the semantic unit based on the position of the word corresponding to the semantic unit in the text input information; determining the character position code corresponding to the semantic unit, wherein, when the semantic unit is a character in a non-complete word, the corresponding character position code indicates the character position of the semantic unit in the corresponding word, and when the semantic unit is a complete word, the corresponding character position code can be used to determine that the semantic unit is a complete word; and determining the positional features corresponding to the semantic unit based on the word position code and the character position code corresponding to the semantic unit.

[0076] In other words, the positional features corresponding to a semantic unit contain two layers of positional information. One layer represents the positional information at the word level, which is reflected through the corresponding word position encoding. The other layer represents the positional information at the character level, which is reflected through the corresponding character position encoding. This allows the positional features of each semantic unit to fully and hierarchically represent the position of each semantic unit in the entire text input information.

[0077] For example, when the text input is "Ducks in a row" (assuming the user has entered the characters "r", "o", and "w" but has not yet pressed the space bar to indicate that "r", "o", and "w" form a complete word), the text input includes the three complete words "Ducks", "in", and "a", as well as the three input characters "r", "o", and "w". That is, it contains six semantic units: "Ducks", "in", "a", "r", "o", and "w". Table 1 below shows, as an example, the word position codes and character position codes corresponding to each semantic unit of this text input.

[0078] Table 1

[0079]

[0080] The word position encoding of these six semantic units can be [1,2,3,4,4,4], which means that “ducks” is the first word in the text input information, “in” is the second word in the text input information, “a” is the third word in the text input information, and “r”, “o”, and “w” all belong to the fourth word in the text input information (that is, “r”, “o”, and “w” are the characters of “row”).

[0081] The character position encoding for these six semantic units can be [0,0,0,1,2,3]. That is, the character position encoding for the complete words "Ducks", "in", and "a" can be set to 0, indicating that the semantic unit is a complete word and not an input character in a non-complete word (of course, the character position encoding can also be "a" or "b", etc., to indicate that the semantic unit is a complete word and does not belong to the input character of a non-complete word). Input characters in non-complete words are encoded according to their position in the word. In [0,0,0,1,2,3], "1,2,3" means that "r", "o", and "w" correspond to the 1st, 2nd, and 3rd characters of the word "row", respectively.

[0082] Then, to facilitate feature extraction of lexical and character-level positional information of semantic units by the neural network model, the second embedding layer can embed the lexical positional encoding of the semantic unit to obtain lexical positional features. P w ( x m )∈ R d The third embedding layer can embed the character position encoding of semantic units to obtain character position features. P c ( x m )∈ R d Then, based on the character position features and word position features of the semantic unit, the positional features corresponding to that semantic unit can be determined. For example, the sum of the character position features and word position features of the semantic unit can be directly used as the positional feature corresponding to that semantic unit. P m ∈ R d The second and third embedding layers can, for example, perform embedding operations on word position encoding and character position encoding by referring to word embedding.

[0083] Then, after determining the second semantic feature, lexical position feature, and character position feature of each semantic unit, the fusion layer can fuse these features to obtain the corresponding third semantic feature. Specifically, the second semantic feature, lexical position feature, and character position feature of the same semantic unit can be summed, and this summation result is used as the third semantic feature corresponding to that semantic unit. The formula can be expressed as follows:

[0084] X ( x m )= E ( x m )+ P m ,in, P m = P w ( x m )+ P c ( x m )

[0085] in, X (x m ) represents a semantic unit x m The third semantic feature, E ( x m ) represents a semantic unit x m The second semantic feature, P w ( x m ) represents a semantic unit x m The word position features, among which, P w ( x m )∈ R d , P c ( x m ) represents a semantic unit x m The character position features, where, P c ( x m )∈ R d , P m Representation and semantic units x m The corresponding positional features, as shown in the formula above, represent the third semantic feature of a semantic unit, which is the sum of the corresponding second semantic feature, lexical positional feature, and character positional feature. Furthermore, X ( x m )∈ R d .

[0086] Thus, the sequence composed of the third semantic features corresponding to each semantic unit X The form can be shown in the following formula:

[0087] X =[ X ( x 1), X ( x 2), ..., X ( x t-1+k )]∈R (t-1+k)*d .

[0088] In subsequent steps, this sequence XThe first semantic feature is obtained by inputting it into the context modeling model of the vocabulary prediction model.

[0089] In this way, the second semantic feature can not only represent the semantic features of each semantic unit, but also indicate which word each semantic unit belongs to in the text input information, and which character a semantic unit, as a character, belongs to within which word. Therefore, the second semantic feature contains more comprehensive and detailed positional information corresponding to the semantic unit. Consequently, the word prediction model can perform more accurate word prediction based on the second semantic feature.

[0090] Optionally, the operation of determining the first semantic feature based on the third semantic feature includes: inputting the third semantic feature into a context modeling model based on a self-attention mechanism to determine the first semantic feature.

[0091] In other words, after determining a third semantic feature that contains both semantic information and multi-layered location information, the computing device can input the third semantic feature into a context modeling model based on a self-attention mechanism to determine the first semantic feature for context modeling of the third semantic feature.

[0092] Specifically, the sequence composed of the third semantic features corresponding to each of the aforementioned semantic units can be considered as follows: X The input is fed into the context modeling model to obtain the aforementioned first semantic feature. The context modeling model can employ... N Layered Transformer network implementation, N The first semantic feature output by the context modeling model can be represented as a pre-defined numerical value:

[0093] H =Transformer( X )∈ R (t+k)×d .

[0094] One layer of the Transformer network used can be composed of a masked multi-head self-attention module and a feedforward neural network. The calculation process of one layer is shown in the following formula:

[0095]

[0096] in, Q = XW Q ; K = XW K ; V = XW V ; MThe mask is used to ensure that the prediction task completed by the context modeling model and the vocabulary prediction layer is an autoregressive task.

[0097] As described in the background section, n-gram models predict the probability of word sequences with a fixed number of words in the training corpus. Because they rely only on a limited window of context, they struggle to capture long-distance semantic relationships. According to the technical solution of this application, the first semantic feature generated by a multi-head self-attention-based text model can extract contextual information across the entire text input, thus more effectively capturing long-distance semantic features and exhibiting stronger generalization ability. Therefore, prediction based on the first semantic feature, which implicitly contains contextual relationship information, can further improve prediction accuracy.

[0098] Optionally, the operation of generating output information corresponding to the text input information using a vocabulary prediction layer based on the first semantic features includes: using the vocabulary prediction layer to determine the probability distribution of candidate words based on the first semantic features; determining predicted words based on the determined probability distribution; and generating output information based on the predicted words.

[0099] In other words, a vocabulary prediction model can input the first semantic features into the vocabulary prediction layer to obtain the predicted words, and then generate output information based on the predicted words. For example, the output information can be directly the predicted words.

[0100] Specifically, the vocabulary prediction layer can first determine the fourth semantic feature to represent the semantic features of the predicted vocabulary based on the first semantic feature. H t+k ∈ R d ,in, H t+k First semantic feature H The Middle t+k The features corresponding to each token. Then, the vocabulary prediction layer can analyze the fourth semantic features. H t+k Projecting yields the probability distribution of candidate words. P ( y t (wherein, the probability distribution can be a probability distribution projected onto the vocabulary), as shown in the following formula:

[0101] P ( y t )=softmax( H t+k W o + b o )

[0102] in, W ∈ R d*|v| The output projection matrix is ​​used to generate the probability distribution of candidate words. Thus, the vocabulary prediction layer can... P ( y t The prediction word is determined. This prediction word can be the candidate word with the highest probability value, or it can be the word ranked first by probability value. n All candidate words are used as predicted words for users to choose from. n The specific values ​​can be preset manually.

[0103] It should be noted that the text input processing method in this manual supports three prediction scenarios for input methods, including the word completion scenario mentioned above, the scenario of predicting the next word the user might input based on several complete words already entered by the user (referred to here as the word prediction scenario), and the scenario of correcting incomplete words that the user is currently inputting and that contain spelling errors (referred to here as the spelling correction scenario). Specifically, through three types of training samples, the same model can be used to complete the tasks of all three prediction scenarios.

[0104] Optionally, the method further includes: acquiring training samples, which include text input information samples and annotation information, wherein the annotation information includes predicted words corresponding to the corresponding text input information samples, wherein the training samples include a first type of training samples, a second type of training samples, and a third type of training samples, wherein the text input information samples of the first type of training samples only include complete words, the text input information samples of the second type of training samples contain incomplete words, and the text input information samples of the third type of training samples contain input characters with spelling errors; and training the context modeling model and the word prediction layer based on the training samples.

[0105] The predicted words mentioned here, corresponding to the corresponding text input information samples, are the words that the user actually wants to input, as labeled in the text input information samples. The training samples include three types: the first type, the second type, and the third type. The first type corresponds to the word prediction scenario, the second type corresponds to the word completion scenario, and the third type corresponds to the spelling correction scenario.

[0106] The following examples illustrate three prediction scenarios.

[0107] The word prediction scenario is as follows: after a user inputs an incomplete word, the system automatically recommends the next most likely word based on the context of the user's text input. For example, if the input is "Li Bai was a poet of the", the system predicts the target word "Tang". This word prediction scenario corresponds to the first type of training samples. If the text input sample in the first type of training samples is "Li Bai was a poet of the", then the corresponding annotation information is "Tang".

[0108] Therefore, whether in the training or prediction phase, the text input information only contains sequences of complete words. W =[ w 1, ..., w t The goal is to predict. w t+1 The training objective is equivalent to maximizing the likelihood of obtaining the labeled information, as shown in the following equation:

[0109]

[0110] in, This is for labeling information.

[0111] Word completion is a scenario where, when the text input contains an incomplete word that the user is typing, the system predicts the most likely complete word corresponding to that incomplete word based on the context of the user's input and partial characters from the incomplete word. Let the text input be "You are so b". W = [“You”,“are”,“so”], the user is typing the character “b”, at this time C = [b], the computing device needs to predict the target word “beautiful”.

[0112] The word completion scenario corresponds to the second type of training samples. If the text input information sample in the second type of training samples is "You are so b", then the corresponding annotation information is "beautiful". The training objective is equivalent to maximizing the likelihood of obtaining the annotation information, as shown in the following formula.

[0113]

[0114] in, This is for labeling information.

[0115] The spelling correction scenario is as follows: when a user inputs a misspelled word (e.g., the user inputs "raow" but should actually input "row"), this method can correct the spelling using context and character distribution. In this case, the user's incomplete vocabulary contains errors, and this method needs to guide the input towards the most likely target word during the prediction process. The spelling correction scenario corresponds to the third type of training sample. If the text input information sample in the first type of training sample is "Ducks in a raow", then the corresponding annotation information is "row".

[0116] This method utilizes contextual semantics and deep modeling to automatically correct text input information containing spelling errors. The training objective can be represented as:

[0117]

[0118] in, For annotation information, The input character set is a set of characters from an incomplete vocabulary containing spelling errors.

[0119] In addition, the first semantic feature generated by the above context modeling method can also be used to push accessibility features to users. Optionally, the method further includes: based on the first semantic feature, using an accessibility feature judgment model to determine whether to push a preset accessibility feature to the user.

[0120] The aforementioned accessibility function judgment model can be a classification model, specifically a binary classification model. The output of the accessibility function judgment model can be whether or not to push a preset accessibility function to the user. Of course, since there can be multiple accessibility functions, this accessibility function judgment model can also be a multi-class classification model, and the output can include the type of accessibility function to be pushed to the user and whether to push an accessibility function to the user. Training samples for training the accessibility function judgment model can be obtained through manual annotation or weak supervision, thereby pre-training the accessibility function judgment model.

[0121] The accessibility features pushed can refer to the functions that the user needs based on the user's text input information. These accessibility features can include search intent recognition, intelligent assistant invocation, and knowledge Q&A navigation, for example:

[0122] Search intent recognition: When inputting something like "population of Rome", the computing device can recognize the user's potential intent to search for Rome-related information and push the function to search for "Rome" to the user.

[0123] Intelligent Assistant Invocation: When the user enters "remind me to call John at 8", the computing device can push the "Create Reminder" function to the user. If the user agrees, the computing device can call the system calendar interface to create the corresponding reminder for the user.

[0124] Knowledge Q&A Jump: When the user enters a question such as "who is the president of France", the computing device can push the function "use the smart assistant to find the answer" to the user.

[0125] Among them, when the text input information is [ w 1, w 2, ..., w t-1 , c 1, c 2, ..., c k The computing device can determine the first semantic feature corresponding to the text input information using the text input information processing method provided in this specification. H ∈ R (t+k)*d And determine the feature representation of the last semantic unit from it. h = H t+k Then, the feature representation is input into the auxiliary function judgment model to obtain the judgment result (or the first semantic feature corresponding to the text input information can be input into the auxiliary function judgment model to obtain the judgment result). During the training phase, the auxiliary function judgment model can be trained by minimizing the cross-entropy loss.

[0126] In addition, refer to Figure 1 As shown, according to a second aspect of this embodiment, a storage medium is provided. The storage medium includes a stored program, wherein, when the program is executed, a processor performs any of the methods described above.

[0127] Therefore, according to this embodiment, the accuracy of generating prediction results based on text input information can be improved, as well as the compatibility with different types of prediction tasks can be improved, thereby increasing applicability.

[0128] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0129] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0130] Example 2

[0131] Figure 4 A text input information processing apparatus 400 according to a first aspect of this embodiment is shown, which corresponds to the method described according to the first aspect of Embodiment 1. Reference Figure 4 As shown, the device 400 includes: an acquisition module 410, used to acquire text input information of a user, the text input information including complete words that the user has already entered and / or input characters of incomplete words that the user is currently entering;

[0132] The feature generation module 420 is used to generate a first semantic feature to characterize the semantics of the text input information based on the text input information. The first semantic feature is also used to characterize the position information of complete words and / or input characters in the text input information. The position information includes word-level position information corresponding to words and / or character-level position information corresponding to characters.

[0133] The information generation module 430 is used to generate output information corresponding to the text input information based on the first semantic features and using the vocabulary prediction layer, wherein the output information includes the predicted vocabulary corresponding to the text input information.

[0134] Optionally, when the text input information only includes complete words, the predicted word includes the next complete word relative to the last complete word in the text input information; and when the text input information includes input characters that are not complete words, the predicted word includes the complete word corresponding to the input character. The device further includes: an auxiliary function judgment module 440, used to determine whether to push a preset auxiliary function to the user based on the first semantic feature and using an auxiliary function judgment model.

[0135] Optionally, the feature generation module 420 is specifically configured to: generate a corresponding second semantic feature for each semantic unit in the text input information, wherein the semantic unit is a single complete word and / or a single input character of the text input information; determine the positional features corresponding to each semantic unit, wherein the positional features are used to characterize the word-level positional information and character-level positional information of the corresponding semantic unit in the text input information; fuse the second semantic features with the corresponding positional features to obtain a corresponding third semantic feature; and determine the first semantic feature based on the third semantic feature.

[0136] Optionally, the feature generation module 420 is specifically used to: when the semantic unit is a single complete word, use the word embedding vector of the single complete word as the second semantic feature corresponding to the semantic unit; and when the semantic unit is a single input character, use the character embedding vector of the single input character as the second semantic feature corresponding to the semantic unit.

[0137] Optionally, the feature generation module 420 is specifically used to: determine the word position code corresponding to the semantic unit based on the position of the word corresponding to the semantic unit in the text input information; determine the character position code corresponding to the semantic unit, wherein, when the semantic unit is a character in a non-complete word, the corresponding character position code indicates the character position of the semantic unit in the corresponding word, and when the semantic unit is a complete word, the corresponding character position code can be used to determine that the semantic unit is a complete word; and determine the position feature corresponding to the semantic unit based on the word position code and the character position code corresponding to the semantic unit.

[0138] Optionally, the feature generation module 420 is specifically used to input the third semantic feature into a context modeling model based on a self-attention mechanism to determine the first semantic feature. The device further includes a training module 450, used to acquire training samples, which include text input information samples and annotation information. The annotation information includes predicted words corresponding to the corresponding text input information samples. The training samples include a first type of training sample, a second type of training sample, and a third type of training sample. The text input information samples of the first type of training sample only include complete words, the text input information samples of the second type of training sample contain incomplete words, and the text input information samples of the third type of training sample contain input characters with spelling errors. The module also trains the context modeling model and the word prediction layer based on the training samples.

[0139] Optionally, the information generation module 430 is specifically used to: utilize the vocabulary prediction layer to determine the probability distribution of candidate words based on the first semantic features; determine the predicted words based on the determined probability distribution; and generate output information based on the predicted words.

[0140] Therefore, according to this embodiment, the accuracy of the prediction results corresponding to the generated text input information can be improved, as well as the compatibility with different types of prediction tasks can be improved, thereby increasing applicability.

[0141] Example 3

[0142] Figure 5 A text input information processing apparatus 500 according to a first aspect of this embodiment is shown, which corresponds to the method described according to the first aspect of Embodiment 1. Reference Figure 5 As shown, the device 500 includes: a processor 510; and a memory 520 connected to the processor 510, for providing the processor with instructions to process the following steps: acquiring text input information from a user, the text input information including complete words already entered by the user and / or input characters of incomplete words currently being entered by the user; generating a first semantic feature based on the text input information to characterize the semantics of the text input information, wherein the first semantic feature is further used to characterize the positional information of complete words and / or input characters in the text input information, the positional information including word-level positional information corresponding to words and / or character-level positional information corresponding to characters; and generating output information corresponding to the text input information using a word prediction layer based on the first semantic feature, wherein the output information includes predicted words corresponding to the text input information.

[0143] Optionally, when the text input information only includes complete words, the predicted word includes the next complete word relative to the last complete word in the text input information; and when the text input information includes input characters that are not complete words, the predicted word includes the complete word corresponding to the input character.

[0144] Optionally, the operation of generating a first semantic feature to characterize the semantics of the text input information based on the text input information includes: generating a corresponding second semantic feature for each semantic unit in the text input information, wherein the semantic unit is a single complete word and / or a single input character of the text input information; determining a positional feature corresponding to each semantic unit, wherein the positional feature is used to characterize the word-level positional information and character-level positional information of the corresponding semantic unit in the text input information; fusing the second semantic feature with the corresponding positional feature to obtain a corresponding third semantic feature; and determining the first semantic feature based on the third semantic feature.

[0145] Optionally, the operation of generating a corresponding second semantic feature for each semantic unit in the text input information includes: when the semantic unit is a single complete word, using the word embedding vector of the single complete word as the second semantic feature corresponding to the semantic unit; and when the semantic unit is a single input character, using the character embedding vector of the single input character as the second semantic feature corresponding to the semantic unit.

[0146] Optionally, the operation of determining the positional features corresponding to each semantic unit specifically includes: determining the word position code corresponding to the semantic unit based on the position of the word corresponding to the semantic unit in the text input information; determining the character position code corresponding to the semantic unit, wherein, when the semantic unit is a character in a non-complete word, the corresponding character position code indicates the character position of the semantic unit in the corresponding word, and when the semantic unit is a complete word, the corresponding character position code can be used to determine that the semantic unit is a complete word; and determining the positional features corresponding to the semantic unit based on the word position code and the character position code corresponding to the semantic unit.

[0147] Optionally, the operation of determining the first semantic feature based on the third semantic feature includes: inputting the third semantic feature into a context modeling model based on a self-attention mechanism to determine the first semantic feature.

[0148] Optionally, the operation of determining the output information corresponding to the text input information using a vocabulary prediction layer based on the first semantic feature specifically includes: using the vocabulary prediction layer to determine the probability distribution of candidate words based on the first semantic feature; determining the predicted words based on the determined probability distribution; and generating output information based on the predicted words.

[0149] Optionally, the method further includes: acquiring training samples, which include text input information samples and annotation information, wherein the annotation information includes predicted words corresponding to the corresponding text input information samples, wherein the training samples include a first type of training samples, a second type of training samples, and a third type of training samples, wherein the text input information samples of the first type of training samples only include complete words, the text input information samples of the second type of training samples contain incomplete words, and the text input information samples of the third type of training samples contain input characters with spelling errors; and training the context modeling model and the word prediction layer based on the training samples.

[0150] Optionally, the method further includes: using an accessibility feature judgment model based on the first semantic feature to determine whether to push a preset accessibility feature to the user.

[0151] Therefore, according to this embodiment, the accuracy of the prediction results corresponding to the generated text input information can be improved, as well as the compatibility with different types of prediction tasks can be improved, thereby increasing applicability.

[0152] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0153] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0154] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0155] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0156] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0157] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0158] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A text input information processing method characterized by comprising: The method is used for Latin input method or English input method, comprising: obtaining text input information of a user, the text input information comprising complete words that have been input by the user and / or input characters of incomplete words that are being input by the user; generating, according to the text input information, a first semantic feature for representing semantics of the text input information, wherein the first semantic feature is also used for representing position information of the complete words and / or the input characters in the text input information, the position information comprising word-level position information corresponding to the words and character-level position information corresponding to the characters; and generating, according to the first semantic feature, an output information corresponding to the text input information by using a word prediction layer, wherein the output information comprises predicted words corresponding to the text input information, and wherein, the operation of generating, according to the text input information, a first semantic feature for representing semantics of the text input information, comprises: for each semantic unit in the text input information, generating a corresponding second semantic feature, wherein the semantic unit is a single complete word and / or a single input character of the text input information; determining a position feature corresponding to the respective semantic unit, wherein the position feature is used for representing word-level position information and character-level position information of the respective semantic unit in the text input information; fusing the second semantic feature and the corresponding position feature to obtain a corresponding third semantic feature; and determining the first semantic feature according to the third semantic feature, wherein the first semantic feature is obtained by a Transfromer network; and wherein, the operation of determining the position feature corresponding to the respective semantic unit specifically comprises: determining a word position encoding corresponding to the semantic unit according to a position of a word corresponding to the semantic unit in the text input information, wherein the word position encoding represents the position information of the respective semantic unit at the word level; determining a character position encoding corresponding to the semantic unit, wherein in the case that the semantic unit is a character in an incomplete word, the corresponding character position encoding indicates a character position of the semantic unit in the corresponding word, and in the case that the semantic unit is a complete word, the corresponding character position encoding can be used to determine that the semantic unit is a complete word; and determining the position feature corresponding to the semantic unit according to the word position encoding and the character position encoding corresponding to the semantic unit.

2. The method of claim 1, wherein, in the case that the text input information only comprises complete words, the predicted words comprise a next complete word relative to a last complete word in the text input information; and in the case that the text input information comprises input characters of the incomplete words, the predicted words comprise complete words corresponding to the input characters, and wherein, the method further comprises: determining, according to the first semantic feature, whether to push a preset auxiliary function to the user by using an auxiliary function judgment model.

3. The method of claim 1, wherein, the operation of generating, for each semantic unit in the text input information, a corresponding second semantic feature, comprises: In a case where the semantic unit is a single complete word, a word embedding vector of the single complete word is taken as a second semantic feature corresponding to the semantic unit; and In a case where the semantic unit is a single input character, a character embedding vector of the single input character is taken as a second semantic feature corresponding to the semantic unit.

4. The method of claim 1, wherein, According to the third semantic feature, the operation of determining the first semantic feature includes: inputting the third semantic feature into a context modeling model based on a self-attention mechanism to determine the first semantic feature, and wherein the method further includes: obtaining training samples, the training samples including text input information samples and label information, the label information including predicted words corresponding to the corresponding text input information samples, wherein the training samples include first type training samples, second type training samples and third type training samples, the text input information samples of the first type training samples only include complete words, the text input information samples of the second type training samples contain incomplete words, and the text input information samples of the third type training samples contain input characters with spelling errors; and According to the training samples, the context modeling model and the word prediction layer are trained.

5. The method of claim 1, wherein, According to the first semantic feature, the operation of generating output information corresponding to the text input information by using a word prediction layer includes: determining a probability distribution of candidate words according to the first semantic feature by using the word prediction layer; determining the predicted word according to the determined probability distribution; and generating the output information based on the predicted word.

6. A storage medium, characterized by The storage medium includes a stored program, wherein when the program is running, the processor executes the method of any one of claims 1 to 5.

7. A text input information processing apparatus characterized by comprising: The device is used for Latin input method or English input method, including: an obtaining module configured to obtain text input information of a user, the text input information including complete words that have been input by the user and / or input characters of incomplete words that are being input by the user; a feature generation module configured to generate, according to the text input information, a first semantic feature for representing semantics of the text input information, wherein the first semantic feature is also configured to represent position information of complete words and / or input characters in the text input information, the position information including word-level position information corresponding to words and / or character-level position information corresponding to characters, and wherein the feature generation module is specifically configured to, for each semantic unit in the text input information, generate a corresponding second semantic feature, wherein a semantic unit is a single complete word and / or a single input character of the text input information; determine a position feature corresponding to each semantic unit, wherein the position feature is configured to represent word-level position information and character-level position information of the corresponding semantic unit in the text input information; fuse the second semantic feature and the corresponding position feature to obtain a corresponding third semantic feature; and determine the first semantic feature according to the third semantic feature, the first semantic feature being obtained by a Transfromer network, and wherein the feature generation module is specifically configured to determine a word position encoding corresponding to the semantic unit according to a position of a word corresponding to the semantic unit in the text input information, wherein the word position encoding represents the position information of the corresponding semantic unit at the word level; determine a character position encoding corresponding to the semantic unit, wherein in a case where the semantic unit is a character in an incomplete word, the corresponding character position encoding indicates a character position of the semantic unit in the corresponding word, and in a case where the semantic unit is a complete word, the corresponding character position encoding can be used to determine that the semantic unit is a complete word; and determine the position feature corresponding to the semantic unit according to the word position encoding and the character position encoding corresponding to the semantic unit; and an information generation module configured to generate, according to the first semantic feature, output information corresponding to the text input information by using a word prediction layer, wherein the output information includes predicted words corresponding to the text input information.

8. A text input information processing apparatus characterized by comprising: The device is used for Latin input method or English input method, comprising: a processor; and a memory connected with the processor, configured to provide the processor with instructions for processing the following processing steps: obtaining text input information of a user, the text input information including complete words that have been input by the user and / or input characters of incomplete words that are being input by the user; generating, according to the text input information, a first semantic feature for representing semantics of the text input information, wherein the first semantic feature is also configured to represent position information of complete words and / or input characters in the text input information, the position information including word-level position information corresponding to words and / or character-level position information corresponding to characters; generating, according to the first semantic feature, output information corresponding to the text input information by using a word prediction layer, wherein the output information includes predicted words corresponding to the text input information, and wherein, According to the text input information, an operation of generating a first semantic feature for representing semantics of the text input information comprises: For each semantic unit in the text input information, a corresponding second semantic feature is generated, wherein the semantic unit is a single complete word and / or a single input character of the text input information; a position feature corresponding to each semantic unit is determined, wherein the position feature is used to represent word-level position information and character-level position information of the corresponding semantic unit in the text input information; the second semantic feature is fused with the corresponding position feature to obtain a corresponding third semantic feature; and the first semantic feature is determined according to the third semantic feature, wherein the first semantic feature is obtained by a Transfromer network, and wherein The operation of determining the position feature corresponding to each semantic unit specifically comprises: According to a position of a word corresponding to the semantic unit in the text input information, a word position code corresponding to the semantic unit is determined; A character position code corresponding to the semantic unit is determined, wherein in a case where the semantic unit is a character in a non-complete word, the corresponding character position code indicates a character position of the semantic unit in the corresponding word, and in a case where the semantic unit is a complete word, the corresponding character position code can be used to determine that the semantic unit is a complete word, wherein the word position code represents position information of the corresponding semantic unit at a word level; and according to the word position code and the character position code corresponding to the semantic unit, the position feature corresponding to the semantic unit is determined.

Images