Information prediction method and device, equipment and medium

Abstract

The information prediction method provided by the embodiment of the invention is applied to the technical field of computers, and comprises the following steps: acquiring a current lexical element input by a user, and predicting the current lexical element by using an information prediction tool to obtain current prediction information; wherein the information prediction tool is provided with a state space model, an output equation of the state space model is obtained through adjustment according to a target parameter matrix set, the target parameter matrix set is determined from a historical parameter matrix set, and the historical parameter matrix set is obtained according to historical lexical meta information. The state space model is adjusted through the historical parameter matrix set corresponding to the historical lexical element information so as to predict the input information, the context relevance of the lexical elements is improved, the memory space is saved, and the calculation amount is reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment, and medium for information prediction. Background Art

[0002] In recent years, with the development of deep learning, deep learning models based on the Transformer structure have achieved great success in the field of natural language processing, and can be specifically applied to input method prediction. However, the computational complexity and space complexity of the Transformer increase quadratically with the increase in sequence length, which creates a bottleneck for processing ultra-long sequence data. The computational overhead of long sequences is not limited to the training phase, but is also significant during inference.

[0003] The use of state-space models for input method prediction can process word metadata. However, the output of the current state in the state-space model must be forcibly associated with all historical states corresponding to historical word metadata, which increases the introduction of invalid information. In addition, the output at the current moment in the state-space model is determined by the historical state and the current input, but the hyperparameters before the historical state and the current input can only be determined by the current input, which makes it impossible to fully utilize the information of the historical state. Therefore, the amount of calculation for input method prediction based on the state-space model is relatively large, and the predicted content cannot fully refer to the historical content. Summary of the invention

[0004] In view of the above problems, a method, device, apparatus, and medium for information prediction are proposed to overcome the above problems or at least partially solve the above problems, including:

[0005] In a first aspect of the implementation of the present application, a method for information prediction is first provided, characterized in that the method comprises:

[0006] Get the current word input by the user;

[0007] Using an information prediction tool to predict the current word-gram to obtain current prediction information; wherein the information prediction tool is provided with a state space model, the output equation of the state space model is obtained by adjusting according to a target parameter matrix set, the target parameter matrix set is determined from a historical parameter matrix set, and the historical parameter matrix set is obtained according to historical word-gram information;

[0008] The current forecast information is presented to the user.

[0009] Optionally, before obtaining the current word element input by the user, the method further includes:

[0010] Inputting information corresponding to the plurality of sample word units into a preset state space model to obtain a current parameter matrix of a first sample word unit and a set of historical parameter matrices of a plurality of second sample word units, wherein the plurality of sample word units include the first sample word unit and the plurality of second sample word units;

[0011] Comparing the current parameter matrix with the plurality of historical parameter matrix sets for similarity to obtain a similarity comparison result;

[0012] According to the similarity comparison result, a target parameter matrix set is selected from the plurality of historical parameter matrix sets, and an output equation in the state space model is adjusted according to the target parameter matrix set;

[0013] The prediction information corresponding to the first word-gram is generated using the adjusted output equation in the state-space model.

[0014] Optionally, the comparing the current parameter matrix with the plurality of historical parameter matrix sets to obtain a similarity comparison result includes:

[0015] Obtain each historical parameter matrix in the historical parameter matrix set;

[0016] A similarity calculation is performed on the current parameter matrix and each of the historical parameter matrices to obtain a similarity result.

[0017] Optionally, the performing similarity calculation on the current parameter matrix and each of the historical parameter matrices to obtain a similarity result includes:

[0018] The Euclidean distance between the current parameter matrix and each of the historical parameter matrices is calculated, and the Euclidean distance is determined as the similarity result.

[0019] Optionally, before the similarity comparison between the current parameter matrix and the plurality of historical parameter matrix sets is performed to obtain a similarity comparison result, the method further includes:

[0020] Inputting information corresponding to a plurality of training word units into the state space model to obtain a current prediction output matrix of a first word unit and a set of historical prediction output matrices of a plurality of second word units;

[0021] The step of selecting a target parameter matrix set from the historical parameter matrix set according to the similarity result includes:

[0022] Sorting the similarity results to obtain a sorted historical parameter matrix set;

[0023] According to a preset target number, a target parameter matrix set is screened out from the sorted historical parameter matrix set, wherein the number of target parameter matrices in the target parameter matrix set is the preset target number;

[0024] The step of adjusting the output equation in the state space model according to the target parameter matrix set includes:

[0025] Determine a target index corresponding to each target parameter matrix in the target parameter matrix set;

[0026] Determining a target prediction output matrix set from the historical prediction output matrix set according to the target index;

[0027] Fusing the current prediction output matrix corresponding to the first word-gram with the target prediction output matrix set to obtain a fused output matrix;

[0028] Using an attention mechanism to perform weighted calculation on the fusion output matrix to obtain a weighted calculation result;

[0029] The output equation in the state space model is adjusted according to the target parameter matrix set and the weighted calculation result.

[0030] Optionally, the step of generating prediction information corresponding to the first sample word-gram by using the adjusted output equation in the state-space model includes:

[0031] The output equation in the adjusted state-space model is used to generate prediction information corresponding to the first sample word-gram; wherein the prediction information includes a plurality of prediction word-grams corresponding to the first sample word-gram and the probability corresponding to each of the prediction word-grams.

[0032] Optionally, the using an information prediction tool to predict the current word to obtain current prediction information includes:

[0033] Using the information prediction tool to predict the current word-gram, obtaining a plurality of current predicted word-grams and word-gram probabilities corresponding to the plurality of current predicted word-grams;

[0034] Sorting the multiple current predicted word-grams according to the word-gram probabilities to obtain sorted current prediction information;

[0035] The presenting the current prediction information to the user comprises:

[0036] The sorted current prediction information is presented to the user.

[0037] In a second aspect of the implementation of the present application, there is also provided an information prediction device, characterized in that it includes:

[0038] A first acquisition module, used to acquire the current word unit input by the user;

[0039] A first prediction module is used to predict the current word using an information prediction tool to obtain current prediction information; wherein the information prediction tool is provided with a state space model, and the output equation of the state space model is obtained by adjusting according to a target parameter matrix set, wherein the target parameter matrix set is determined from a historical parameter matrix set, and the historical parameter matrix set is obtained according to historical word units;

[0040] The first presentation module is used to present the current prediction information to the user.

[0041] In the third aspect of the implementation of the present application, the embodiments of the present application also disclose an electronic device, including: a processor; and a memory, on which executable code is stored, and when the executable code is executed, the processor executes the information prediction method as described in one or more of the embodiments of the present application.

[0042] In the fourth aspect of the implementation of the present application, the embodiments of the present application further disclose a non-volatile readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method of information prediction as described in one or more of the embodiments of the present application is implemented.

[0043] The embodiments of the present application also disclose a computer program product, including a computer program / instruction, which, when executed by a processor, implements the information prediction method as described in one or more of the embodiments of the present application.

[0044] The embodiments of the present application have the following advantages:

[0045] The current word-gram input by the user is obtained, and the current word-gram is predicted by using an information prediction tool to obtain current prediction information; wherein, the information prediction tool is provided with a state-space model, and the output equation of the state-space model is obtained by adjusting according to a target parameter matrix set, and the target parameter matrix set is determined from a historical parameter matrix set, and the historical parameter matrix set is obtained according to historical word-gram information, and the current prediction information is presented to the user, and the state-space model is adjusted by a historical parameter matrix set corresponding to the historical word-gram information to predict the input information, thereby improving the contextual relevance of the word-gram, saving memory space, and reducing the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0047] Figure 1 One of the flow charts of the steps of the information prediction method provided in some embodiments of the present application;

[0048] Figure 2 A schematic diagram of the structure of a state space model provided in some embodiments of the present application;

[0049] Figure 3 A schematic diagram of a parallel computing process provided for some embodiments of the present application;

[0050] Figure 4a A schematic diagram of a continuous addition operation provided in some embodiments of the present application;

[0051] Figure 4b The continuous addition operation y provided in some embodiments of the present application n Calculation diagram;

[0052] Figure 5 It is a structural diagram of an information prediction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in each embodiment of the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not constitute any limitation, and the various embodiments can be combined with each other and referenced to each other without contradiction.

[0055] In recent years, with the development of deep learning, especially deep learning models based on Transformer structure, great success has been achieved in natural language processing, image processing and other fields. Transformer model has powerful representation ability, can capture long-distance dependencies, and performs well in processing sequence data. However, the computational and spatial complexity of Transformer increases quadratically with the increase of sequence length (O(n 2 )), which creates a bottleneck for processing very long sequence data. The computational overhead of long sequences not only exists in the training phase, but also in the inference phase. Therefore, finding algorithms and models that can process long sequence data at a lower computational and storage cost has become an important research direction. This application predicts the current word unit input by the user by setting an information prediction tool with an adjusted state space model, thereby improving the contextual relevance of the word unit in the information prediction, saving memory space and reducing the amount of calculation.

[0056] Reference Figure 1 , shows one of the step flow charts of the information prediction method provided by some embodiments of the present application, the method may include:

[0057] Step 101, obtaining the current word unit input by the user.

[0058] The above-mentioned user input operations can be specifically various applications, and the applications run on various devices, including but not limited to: smart phones, tablet computers, e-book readers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, car computers, desktop computers, set-top boxes, smart TVs, wearable devices, etc.

[0059] The information prediction method mentioned in the embodiment of the present application can support multiple input methods. Among them, the input method can be a coding method used to input various symbols into electronic devices such as computers and mobile phones, and the user can use the input method application to conveniently input the required characters or strings into the electronic device. It should be noted that in the embodiment of the present application, in addition to supporting common Chinese input methods (such as pinyin input method, Wubi input method, Zhuyin input method, voice input method, handwriting input method, etc.), the input method can also support input methods of other languages ​​(such as Japanese Hiragana input method, Korean input method, etc.), and no limitation is made on the input method and the language type of the input method.

[0060] In this embodiment, the current token input by the user is obtained. Suppose the user inputs the current token "Hello" in the client of a certain application, and the input method needs to predict the next possible character or word. The information prediction tool will predict the next possible character or word based on the context information of "Hello", such as ",", "!", "world", etc.

[0061] The above-mentioned current token input by the user can support multiple input methods. For example, it supports input methods such as input through the input method and paste input. Among them, the input method through the input method can include, but is not limited to, text input, voice input, etc.

[0062] In some embodiments of the present application, before step 101, it further includes:

[0063] S1: Input the information corresponding to multiple sample tokens into a pre-set state space model to obtain the current parameter matrix of the first sample token and a set of historical parameter matrices of multiple second sample tokens. The multiple sample tokens include the first sample token and multiple second sample tokens.

[0064] Among them, the sample token refers to a large amount of sample data that needs to be input during model training. For the sake of distinction, the present application divides multiple sample tokens into the first sample token and multiple second sample tokens. The first sample token refers to the latest input sample token, and the multiple second sample tokens refer to multiple historical tokens before the first sample token. The embodiments of the present application take the language model modeling in the natural language processing of the input method as an example. The modeling goal of the language model is that the constructed model can judge whether a piece of text is a sentence that can be normally understood by humans. According to this goal, in the application of the input method, we can put a large number of corpora into the model to make the predicted output be the output at the corresponding position. The overall structure of the model is an autoregressive language model, that is, it can only see the information before the current position. The input of the language model is a series of vectorized tokens, and the output is the probability of the next token.

[0065] Specifically, the state space model (State Space Model, SSM) can be used as the speech model applied to the input method in the present application. The state space model is a statistical model used to describe and predict time series data. The state space model captures the dynamic behavior of the system by defining a state equation and an output equation. The state equation describes the evolution of the system state, and the output equation relates the state and the observed data.

[0066] As Figure 2 shown, it is a schematic structural diagram of the state space model provided by some embodiments of the present application. The figure shows the calculation process, and the code corresponding to the calculation process in the figure is as follows:

[0067] Input; x: (B, L, D)

[0068] Output; y: (B, L, D)

[0069] 1. A: (D, N) ← some_structured_matrix

[0070] 2. B:(B, L, N)←SB(x)

[0071] 3. C:(B, L, N)←SC(x)

[0072] 4. Δ: (B, L, D) ←τΔ (parameter+sΔ(x))

[0073] 5. A, B: (B, L, D, N) ← discretize (Δ, A, B)

[0074] 6. y←SSM(A, B, C)(x)

[0075] 7. return y

[0076] against Figure 2 The calculation process and the above code are explained in detail. The input is x, and the shape of x is (B, L, D), where B is the batch size, L is the sequence length, and D is the dimension. Project the input x to get xz, that is, map the input from (B, L, D) to (B, L, d_inner*2) through the linear layer, and then rearrange it to (B, d_inner*2, L). Separate xz to get x and z, and the shapes of x and z are both (B, d_inner, L).

[0077] Next, x and z are calculated in two paths. In the first path, x is outputted by a 1D convolution operation, and the shape remains unchanged. The convolution output x_conv is mapped to x_dbl through linear mapping, and the shape of x_dbl is (B*L, dt_rank+2*d_state). Split x_dbl into three parts in the third dimension, namely dt, B, and C. Then dt is linearly mapped and its shape becomes (B, d_inner, L). Calculate the state and output of dt, B, C, and A (where A is a D*N matrix) using the SSM discrete formula, and the output shape is (B, L, D). In the second path, the z path passes through the sigmoid activation function with nonlinear characteristics, and is multiplied point by point with the output of the SSM discrete formula to obtain the output of the final mamba block. In the shape, d_state is equivalent to N in the formula, and d_ineer is equivalent to D in the formula.

[0078] The above steps are the input and output process of a block. Block usually refers to a basic building block in the model. Block usually contains multiple sub-layers. These blocks are repeatedly stacked in the model to form a deeper network structure, thereby enhancing the model's expressiveness and performance. When training or using a language model, having a series of such blocks can obtain the probability distribution of the next word.

[0079] From the above content, it can be obtained that inputting an x ​​into the preset state space model can obtain the corresponding A, B, and C matrices. In the embodiment of the present application, the B matrix is ​​called the parameter matrix. Therefore, in the process of model training, multiple sample word units including a first sample word unit and multiple second sample word units are input into the preset state space model, and the current parameter matrix corresponding to the first sample word unit can be obtained, and multiple historical parameter matrix sets corresponding to the multiple second sample word units can also be obtained.

[0080] The calculation process of the SSM discrete formula in the above content is explained in detail below, where the input is word unit x and the output is word unit y.

[0081] h'(t)=Ah(t)+Bx(t)

[0082] y(t)=Ch(t)

[0083]

[0084] y k =Ch k

[0085]

[0086] Among them, h,(t) is the state equation, x(t) maps the input sequence, h(t) is the hidden state, y(t) is the output equation, and the specific sources of A, B, and C have been explained in the previous steps and will not be repeated here. The calculation method of is to perform matrix calculation on dt and A in the above steps; The calculation method is to calculate the matrix of dt and B in the previous step, and C is the original C matrix in the previous step. If the A, B, C, and dt matrices are token (word or word) independent, that is, their dimensions are D or D*N, then the calculation process of the SSM formula can be completed in parallel (CNN form) or serially (RNN form), such as Figure 3Shown is a schematic diagram of the parallel computing process provided by some embodiments of the present application. It can be seen from the figure that the A, B, and C matrices are independent of the word units, so the product can be calculated in parallel first, and then the summation can be performed. The preset state space model can maintain linear complexity (O(n)) when processing long sequences, which gives it significant potential in long sequence tasks. The preset state space model can convert the processing of long sequence data into a processing process of state and input, thereby reducing the amount of calculation. In addition, the structure of the state space model can naturally adapt to dynamic changes in time series, which is of great significance in practical applications.

[0087] The Mamba model is an advanced sequence modeling architecture. It is an improvement based on the concept of state-space model. However, the Mamba model cannot use the above-mentioned CNN acceleration method for parallel training. The reason is that for each token, its A, B, and C are inconsistent. That is to say, the convolution kernel coefficients before x0, x1, and x2 cannot be kept consistent when calculating each output. n The coefficients of calculation will change.

[0088] Therefore, in some embodiments of the present application, a new "add" operation is defined. In parallel computing, consecutive addition operations can be performed in parallel. The new "add" operation specifically refers to (A t , B t x t )plus(A t+1 , B t+1 x t+1 )=(A t A t+1 , A t+1 B t x t +B t+ 1x t+1 ). The specific parallel operation of the continuous addition operation is: first calculate each x in parallel n Corresponding to A, B, and C respectively; then use the continuous addition operation to get (A t A t+1 , A t+1 B t x t +B t+1 x t+1 ); at this time y1=C1(A1B0x0+B1x1), y2=C2(A2(A1B0x0+B1x1)+B2x2), and so on, we can get any y n The value of Figure 4a , which is a schematic diagram of the continuous addition operation provided in some embodiments of the present application, from Figure 4aAs can be seen from the figure, each pair can be computed in parallel, and the results of the parallel computation can be further computed in parallel in pairs. Figure 4b , which is a continuous addition operation y provided in some embodiments of the present application n Calculation diagram, from Figure 4b It can be seen that the “addition” operation defined in the embodiment of the present application can also be calculated in parallel in groups of two, and any y can be obtained. n The value of .

[0089] S2: performing a similarity comparison between the current parameter matrix and a plurality of sets of historical parameter matrices to obtain a similarity comparison result;

[0090] It can be inferred from the formula in the above content that during the model training process, y n The output of is combined with all historical states to complete the word unit conversion. Therefore, in order to predict the current output, it is necessary to forcibly associate it with all historical states, which also increases the introduction of invalid information.

[0091] Based on this, the present application proposes an improved method to introduce sparsity to multiple parameter matrices B during model training. Introducing sparsity means making the values ​​of most elements or parameters zero or close to zero by some means, thereby reducing the dimension of the data, reducing the computational complexity or improving the generalization ability of the model; it can be understood that sparsity can reduce the dimension of the data, remove redundant information, and thus reduce computational complexity and storage requirements.

[0092] In the specific implementation, in order to introduce sparsity, firstly, the parameter matrix B corresponding to the current input and each historical parameter matrix B at the historical position are i Do similarity calculation, similarity is high B i Only those who participate in the current moment y n , and the maximum length does not exceed the set length, so as to remove unimportant historical information, reduce the amount of calculation and reduce the occupied storage space.

[0093] In some embodiments of the present application, the following sub-steps may be included before step S2:

[0094] S11: Inputting information corresponding to a plurality of training word-grams into the state space model to obtain a current prediction output matrix of a first word-gram and a set of historical prediction output matrices of a plurality of second word-grams.

[0095] From the content in step S1, we can know that the input is x, and the shape is (B, L, D), where B is the batch size, L is the sequence length, and D is the dimension. Project the input x to get xz, separate xz to get x and z, and the shapes of x and z are (B, d_inner, L). Convolve x through 1D to get the output x_conv, and the shape remains unchanged; linearly map the convolution output x_conv to x_dbl, and the shape is (B*L, dt_rank+2*d_state). Split x_dbl into three parts in the third dimension, namely dt, B, and C.

[0096] Among them, the matrix C corresponding to each input x is the prediction output matrix mentioned in the embodiment of the present application. In the state space model, the C matrix is ​​used to map the state vector of the system to the output vector. It can be understood that there are multiple prediction output matrices corresponding to multiple training word units. Specifically, the matrix C corresponding to the first word unit is called the current prediction output matrix, and the multiple matrices C corresponding to the multiple historical word units (second word units) input before the first word unit are the historical prediction output matrix sets.

[0097] In some embodiments of the present application, step S2 includes the following sub-steps:

[0098] S21: Obtain each historical parameter matrix in the historical parameter matrix set;

[0099] In the specific implementation, each x n The dimension of the corresponding B matrix is ​​D*N, so all B matrices can be calculated in parallel first. In other words, each historical parameter matrix in the historical parameter matrix set can be obtained first to prepare for the next step of calculation.

[0100] S22: Calculate the similarity between the current parameter matrix and each historical parameter matrix to obtain a similarity result.

[0101] Specifically, the current position B can be calculated n and the historical parameter matrices B0, B1, B2, ..., B corresponding to all previous positions n-1 Perform similarity calculation to obtain a similarity result.

[0102] In some embodiments of the present application, sub-step S22 includes the following sub-steps:

[0103] S221: Calculate the Euclidean distance between the current parameter matrix and each of the historical parameter matrices, and determine the Euclidean distance as the similarity result.

[0104] Among them, Euclidean distance is a commonly used method to measure the similarity between two vectors or matrices.

[0105] In the specific implementation, the Euclidean distance can be used to calculate the B of the current position n and all previous positions B0-B n-1 Similarity sim i The specific steps for calculating the Euclidean distance are as follows: First, determine the current parameter matrix B n and the historical parameter matrix B i , where B i B0-B n-1 Any of them, it can be understood that B n and B i The dimensions of the two matrices are the same; secondly, calculate the difference D between the corresponding elements of the two matrices ij Where i and j represent the row and column indices of the matrix respectively; next, the square of the difference matrix is ​​calculated; then, the sum of the squares of all elements is calculated; finally, the arithmetic square root of the sum of the squares of all elements is taken to obtain the Euclidean distance; the obtained Euclidean distance is determined as the similarity result.

[0106] In some cases, the Euclidean distance can be used directly as the similarity result. However, usually, the similarity result will be a value between 0 and 1, indicating the similarity between the two matrices. Therefore, the Euclidean distance can also be normalized, so that when the Euclidean distance is smaller (that is, the two matrices are more similar), the similarity result is closer to 1; when the Euclidean distance is larger (that is, the two matrices are less similar), the similarity result is closer to 0.

[0107] S3: selecting a target parameter matrix set from the plurality of historical parameter matrix sets according to the similarity comparison result, and adjusting the output equation in the state space model according to the target parameter matrix set;

[0108] In a specific implementation, the following process can be used to compare similarities, screen the target parameter matrix set, and adjust the output equation.

[0109] sim i =Bn·(B i ),i <n

[0110] sel(y n (m))=Top m (sim i ),i <n

[0111]

[0112] First, calculate B at the current position n and all previous positions B0-B n-1 The similarity comparison result sim i, where i ranges from 0 to n-1, and the nearest fixed number m is selected. The fixed number m can be a preset number and can be adjusted according to the actual situation. The result is expressed as sel(y n (m)), then sel(y n (m)) is the target parameter matrix set. For example, Top m (sim i ) means to select the m with the highest similarity from the similarity comparison results. Based on the introduction of sparsity, a parallel scanning algorithm is performed to complete the current moment y n The calculation process is to adjust the output equation of the state space model according to the target parameter matrix set.

[0113] In the specific implementation, the parallel scanning algorithm can be expressed as SparseParam scan , SparsePara scan The core implementation is to select the x corresponding to the index according to the determined target parameter matrix. n , A n , and then the selected index is calculated in parallel layer by layer according to the parallel scanning algorithm. For example, the current parameter matrix corresponding to the current input is B 100 According to the similarity calculation results, the top 5 historical parameter matrices with similarity are selected as the target parameter matrix set (B 21 , B 35 , B 49 , B 88 , B 97 ), the corresponding index is (21, 35, 49, 88, 97), and then locate the corresponding (x 21 、x 35 、x 49 、x 88 、x 97 )、(A 21 , A 35 , A 49 , A 88 , A 97 ), and then according to the parallel scanning algorithm SparsePara scan For the selected index, the y corresponding to the current moment is calculated layer by layer in parallel. n .

[0114] Among them, introducing sparsity means that the values ​​of most elements or parameters are made zero or close to zero through the embodiments of the present application, thereby reducing the dimension of the data, reducing the computational complexity or improving the generalization ability of the model; it can be understood that in the embodiments of the present application, sparsity is introduced by calculating the similarity and screening based on the similarity, so that important historical data can be captured while ignoring noise and irrelevant information. In addition, due to the introduction of sparsity, most operations can be skipped on zero elements, reducing unnecessary calculations, thereby reducing memory usage and improving calculation speed.

[0115] In some embodiments of the present application, in step S3, "selecting a target parameter matrix set from the plurality of historical parameter matrix sets according to the similarity comparison result" may also include the following sub-steps:

[0116] S31: sorting the similarity results to obtain a sorted historical parameter matrix set;

[0117] S32: According to a preset target number, a target parameter matrix set is screened out from the sorted historical parameter matrix set, wherein the number of target parameter matrices in the target parameter matrix set is the preset target number.

[0118] In the specific implementation, calculate the B of the current position n and all previous positions B0-B n-1 The similarity comparison results of sim0-sim n-1 , and select the 2048 closest ones, the selection result is expressed as sel(y n (2048)), at this time sel(y n (2048)) is the target parameter matrix set, and then a sparse parallel scanning algorithm is used to complete the current moment y n The calculation process.

[0119] sim i =Bn·(B i ),i <n

[0120] sel(y n (2048))=Top 2048 (sim i ),i <n

[0121]

[0122] In some embodiments of the present application, in step S3, "adjusting the output equation in the state space model according to the target parameter matrix set" may also include the following sub-steps:

[0123] S33: Determine the target index corresponding to each target parameter matrix in the target parameter matrix set;

[0124] From the above content, we can know that after calculating the similarity results of the current B matrix and the historical matrix set, we select multiple historical B matrices with the highest similarity as the target parameter matrix set according to the similarity sorting, and then we can determine the index corresponding to the B matrix as the target index. The target index is B i The set of corresponding i in .

[0125] S34: determining a target prediction output matrix set from the historical prediction output matrix set according to the target index;

[0126] From the foregoing, it can be seen that the calculation of the predicted output matrix C in the related art is actually calculated using the current input x, that is, the matrix C calculated by the parameter output equation only refers to the current input and cannot be dynamically adjusted based on historical information.

[0127] In some embodiments of the present application, in order to dynamically select the historical state and the current input, C is further improved, according to the target parameter matrix set B i The corresponding target index determines multiple corresponding C matrices from the multiple C matrices corresponding to the historical positions i , multiple C i That is, the target prediction output matrix set. It can be understood that, due to the sparsity introduced in the above steps, the target prediction output matrix set determined at this time is also completed on the basis of sparsity.

[0128] S35: Fusing the current prediction output matrix corresponding to the first word-unit with the target prediction output matrix set to obtain a fused output matrix;

[0129] In the specific implementation, the target prediction output matrix set and the current prediction output matrix C matrix are used to perform the Attention calculation. Of course, such Attention calculation will also be completed on the basis of sparsity. The Attention mechanism is a mechanism widely used in deep learning, especially in the fields of natural language processing (NLP) and computer vision. Its main purpose is to allow the model to dynamically focus on different parts of the input sequence when processing sequence data, thereby improving the performance of the model. Based on this, the embodiment of the present application not only adds sparsity but also ensures that the remaining state data is added to the current state according to the weight.

[0130] Every x n The dimension of the corresponding prediction output matrix C is N, so the target prediction output matrix set and the current prediction output matrix C can be fused to obtain a fused matrix C 融合This application does not make any specific restrictions on the specific fusion method.

[0131] S36: Use the attention mechanism to perform weighted calculation on the fusion output matrix to obtain a weighted calculation result.

[0132] In the present application, C 融合 (The fused target prediction output matrix set and the current prediction output matrix C matrix) perform Attention calculation. The Attention calculation at this time is to obtain the attention allocation score of the historical state.

[0133] Q=C 融合 *W Q

[0134] K=C 融合 *W k

[0135] V=C 融合 *W v

[0136]

[0137] Among them, Q, K, and V correspond to query, key, and value respectively, d k is the feature dimension, W Q , W K , W V It can be a preset weight matrix or a trained weight matrix. The fusion output matrix is ​​weighted by the attention mechanism to obtain the weighted calculation result.

[0138] S37: Adjusting the output equation in the state space model according to the target parameter matrix set and the weighted calculation result.

[0139]

[0140] According to the above steps, the target parameter matrix set and the weighted calculation results obtained by attention calculation are used to adjust the output equation in the state space model. Specifically, the final weighted calculation result can be replaced by C in the above step. n Complete the output equation y n The output calculation is performed so that the output equation is adjusted according to the target parameter matrix set and the weighted calculation result.

[0141] S4: Generate prediction information corresponding to the first sample word using the adjusted output equation in the state-space model.

[0142] In the specific implementation, we calculate y according to the scanning parallel operation process of the previous steps n, because the output equation of the state space model introduces sparsity and dynamic selection mechanism, the parallel scanning algorithm "addition" operation can still be used to complete the parallel addition operation layer by layer on the sparse index. After x at each moment is operated with its own A and B matrices, it can be added. That is, the output equation in the adjusted state space model is used to generate the prediction information corresponding to the first sample word.

[0143] By introducing sparsity in the state-space model, the current token can be selectively allowed to pay attention to the tokens in the historical sequence, which is helpful for better context association and memory and computational savings. By adding an attention mechanism operation to the output gate of the state-space model, the weight of more relevant outputs can be made larger; in addition, the attention mechanism operation does not bring quadratic complexity due to the introduction of sparsity and length restrictions, but increases the correlation between contextual semantics. Moreover, the state-space model introduces sparsity and dynamic selection mechanisms, and a parallel scanning algorithm can still be used. The new "addition" operation provided in the embodiment of the present application can be used to complete parallel addition operations layer by layer on the sparse index. The improved scheme does not increase the amount of calculation during reasoning. It can perform state accumulation for reasoning in the manner of RNN, and the historical states that need to be saved are B and C matrices of finite length.

[0144] In some embodiments of the present invention, step S4 may further include the following sub-steps:

[0145] S41: Generate prediction information corresponding to the first sample word-gram using the output equation in the adjusted state-space model; wherein the prediction information includes a plurality of prediction word-grams corresponding to the first sample word-gram and the probability corresponding to each prediction word-gram.

[0146] The similarity calculation result introduces sparsity into the parameter matrix. On the basis of sparsity, the prediction output matrix is ​​fused and weighted through the attention mechanism. Therefore, the output equation of the adjusted state space model can be used to generate prediction information corresponding to the first sample word. In some embodiments of the present application, the output vector y nis a probability distribution vector, which indicates the probability of each word being the next word. We can generate multiple predicted words based on this probability distribution. For example, assuming that the first sample word is "hello", there are three words to generate predicted words: "world", "goodbye" and "morning". Among them, the probability of "world" is 0.67, the probability of "goodbye" is 0.86, and the probability of "morning" is 1.05. The probability of each predicted word is calculated. That is, the prediction information includes the multiple predicted words "world", "goodbye" and "morning" corresponding to the first sample word "hello" and the probabilities of each predicted word 0.67, 0.86, and 1.05 respectively.

[0147] Step 102, using an information prediction tool to predict the current word to obtain current prediction information; wherein the information prediction tool is provided with a state space model, the output equation of the state space model is adjusted according to a target parameter matrix set, the target parameter matrix set is determined from a historical parameter matrix set, and the historical parameter matrix set is obtained according to historical word information.

[0148] In some embodiments of the present application, a user can input the current word element through an input method in an application client, and correspondingly, an information prediction tool can be deployed on the server corresponding to the input method. The information prediction tool is a tool or system for predicting the current word element. The specific implementation of the information prediction tool can be based on the state space model provided in the embodiments of the present application. The predicted content corresponding to the current input content will be predicted through the adjusted state space model, and the current input, historical data and parameter matrix set are referenced in the prediction process.

[0149] In some embodiments of the present application, step 102 may include the following sub-steps:

[0150] Sub-step 21: predicting the current word-gram using an information prediction tool to obtain a plurality of current predicted word-grams and word-gram probabilities corresponding to the plurality of current predicted word-grams;

[0151] From the above content, it can be seen that, assuming that the first sample word is "hello", the information prediction tool in the server is called to predict the current word, and the predicted word has three words: "world", "goodbye" and "morning". Among them, the probability of "world" is 0.67, the probability of "goodbye" is 0.86, and the probability of "morning" is 1.05. Multiple current predicted words and the word probabilities corresponding to multiple current predicted words are prediction information, including multiple predicted words "world", "goodbye" and "morning" corresponding to the first sample word "hello" and the probabilities of 0.67, 0.86, and 1.05 corresponding to each predicted word.

[0152] Sub-step 22: sorting the multiple current predicted word-grams according to the word-gram probabilities to obtain sorted current prediction information;

[0153] After obtaining the prediction information, multiple current prediction word units can be sorted according to the word unit probability corresponding to each prediction word unit to obtain the sorted current prediction information. For example, the probabilities corresponding to "world", "goodbye" and "morning" are 0.67, 0.86, and 1.05, respectively, and 1.05, 0.86, and 0.67 can be obtained. Therefore, the sorted current prediction information can be "morning, 10.05", "goodbye, 0.86", and "world, 0.67".

[0154] Step 103: presenting current prediction information to the user.

[0155] When a user inputs part of a word through a mobile phone input method or a computer input method, the input method client captures the user's input and sends it as a request to the input method server. This request contains the word information of the current input. After receiving the client's request, the input method server calls the information prediction tool, which makes a prediction based on the current input word combined with the state space model. During use, the state space model calculates the target parameter matrix set based on the current input word and historical word information. Then, the output equation is used to adjust the output of the model to obtain the current prediction information. The server returns the prediction information to the client. The prediction information may include the next possible character, word, or phrase.

[0156] After the client receives the prediction information returned by the server, it will present it to the user. The presentation methods may include: candidate word list, automatic completion, prompt information and other methods. Among them, the candidate word list specifically refers to a candidate word list displayed on the input method interface, and the user can complete the input by selecting one of the candidate words. Automatic completion specifically means that if the prediction information is accurate enough, the client can directly automatically complete the currently input word, and the user only needs to confirm or modify it. The prompt information can be specifically displayed below the input box to help users complete the input faster.

[0157] For example, suppose a user enters "Hello," in a mobile phone input method in an application. The client will send "Hello," as a request to the input method server. After receiving the request, the server calls the information prediction tool, which uses the state space model to predict and obtain the next possible character or word, such as "world", "!", "world!", etc. The server returns these prediction information to the client, and the client displays a list of candidate words on the input method interface. The user can choose "world" to complete the input. Or if the user enters "graduate student" on some consulting platforms, the client sends "graduate student" as a request to the server. After receiving the request, the server calls the information prediction tool, which uses the state space model to predict and obtain the next possible phrase as "exam". The server returns the prediction information to the client, and the client automatically completes "graduate student" with "graduate student exam" on the interface, thereby improving the user's input efficiency.

[0158] In some embodiments of the present application, step 103 may also include the following sub-steps:

[0159] Sub-step 31: presenting the sorted current prediction information to the user.

[0160] Since the prediction information may include multiple prediction words corresponding to the first sample word and the probability corresponding to each prediction word, when presenting to the user, the sorted current prediction information may be presented, for example, after inputting "hello", the corresponding sorted prediction information may be "morning", "goodbye", "world". And since the output equation of the state space model in the present application is adjusted and only meaningful historical states are referenced, the amount of calculation can be reduced, thereby improving the input efficiency of the user and the accuracy of the prediction.

[0161] The information prediction method provided in the embodiment of the present application obtains the current word-gram input by the user, and uses the information prediction tool to predict the current word-gram to obtain the current prediction information; wherein, the information prediction tool is provided with a state-space model, and the output equation of the state-space model is obtained by adjusting according to the target parameter matrix set, and the target parameter matrix set is determined from the historical parameter matrix set, and the historical parameter matrix set is obtained according to the historical word-gram information, and the current prediction information is presented to the user, and the state-space model is adjusted by the historical parameter matrix set corresponding to the historical word-gram information to predict the input information, thereby improving the context relevance of the word-gram, saving memory space, and reducing the amount of calculation.

[0162] Reference Figure 5 , showing a schematic diagram of the structure of an information prediction device provided by an embodiment of the present application, such as Figure 5 As shown, the device may include:

[0163] The first acquisition module 501 is used to acquire the current word unit input by the user.

[0164] The first prediction module 502 is used to predict the current word by using an information prediction tool to obtain current prediction information; wherein, the information prediction tool is provided with a state space model, and the output equation of the state space model is adjusted according to a target parameter matrix set, and the target parameter matrix set is determined from a historical parameter matrix set, and the historical parameter matrix set is obtained according to historical word information.

[0165] The first presentation module 503 is configured to present the current prediction information to the user.

[0166] In an optional embodiment of the present application, the information prediction device further includes:

[0167] A first training module is used to input information corresponding to a plurality of sample word units into a preset state space model to obtain a current parameter matrix of a first sample word unit and a set of historical parameter matrices of a plurality of second sample word units, wherein the plurality of sample word units include the first sample word unit and the plurality of second sample word units;

[0168] A first comparison module is used to compare the current parameter matrix with a plurality of historical parameter matrix sets to obtain a similarity comparison result;

[0169] A first screening module, configured to screen out a target parameter matrix set from the plurality of historical parameter matrix sets according to the similarity comparison result, and adjust an output equation in the state space model according to the target parameter matrix set;

[0170] The first adjustment module is used to generate prediction information corresponding to the first sample word by using the adjusted output equation in the state space model.

[0171] In an optional embodiment of the present application, the first comparison module further includes:

[0172] A first acquisition submodule, used to acquire each historical parameter matrix in the historical parameter matrix set;

[0173] The first comparison submodule is used to calculate the similarity between the current parameter matrix and each historical parameter matrix to obtain a similarity result.

[0174] In an optional embodiment of the present application, the first comparison submodule further includes:

[0175] The first comparison unit is used to calculate the Euclidean distance between the current parameter matrix and each of the historical parameter matrices, and determine the Euclidean distance as the similarity result.

[0176] In an optional embodiment of the present application, the information prediction device further includes:

[0177] The second training module is used to input information corresponding to multiple training word units into the state space model to obtain a current prediction output matrix of the first word unit and a set of historical prediction output matrices of multiple second word units.

[0178] The first screening module further includes:

[0179] A first sorting submodule is used to sort the similarity results to obtain a sorted historical parameter matrix set;

[0180] The first screening submodule is used to screen out a target parameter matrix set from the sorted historical parameter matrix set according to a preset target number, wherein the number of target parameter matrices in the target parameter matrix set is the preset target number.

[0181] The first screening module further includes:

[0182] An index determination submodule, used to determine a target index corresponding to each target parameter matrix in the target parameter matrix set;

[0183] A first determination submodule, configured to determine a target prediction output matrix set from the historical prediction output matrix set according to the target index;

[0184] A first fusion submodule, configured to fuse the current prediction output matrix corresponding to the first word-gram with the target prediction output matrix set to obtain a fused output matrix;

[0185] A weighted calculation submodule, used for performing weighted calculation on the fusion output matrix by using an attention mechanism to obtain a weighted calculation result;

[0186] The first adjustment submodule is used to adjust the output equation in the state space model according to the target parameter matrix set and the weighted calculation result.

[0187] In an optional embodiment of the present application, the first adjustment module further includes:

[0188] The second adjustment submodule is used to generate prediction information corresponding to the first sample word-gram using the adjusted output equation in the state-space model; wherein the prediction information includes multiple predicted word-grams corresponding to the first sample word-gram and the probability corresponding to each of the predicted word-grams.

[0189] In an optional embodiment of the present application, the first prediction module 502 further includes:

[0190] A first prediction submodule, configured to predict the current word-gram using the information prediction tool, and obtain a plurality of current predicted word-grams and word-gram probabilities corresponding to the plurality of current predicted word-grams;

[0191] A second sorting submodule, configured to sort the multiple current predicted word units according to the word unit probabilities to obtain sorted current prediction information;

[0192] The first presentation module 503 includes:

[0193] The first presentation submodule is used to present the sorted current prediction information to the user.

[0194] The information prediction method provided in the embodiment of the present application obtains the current word-gram input by the user, and uses the information prediction tool to predict the current word-gram to obtain the current prediction information; wherein, the information prediction tool is provided with a state-space model, and the output equation of the state-space model is obtained by adjusting according to the target parameter matrix set, and the target parameter matrix set is determined from the historical parameter matrix set, and the historical parameter matrix set is obtained according to the historical word-gram information, and the current prediction information is presented to the user, and the state-space model is adjusted by the historical parameter matrix set corresponding to the historical word-gram information to predict the input information, thereby improving the context relevance of the word-gram, saving memory space, and reducing the amount of calculation.

[0195] An embodiment of the present application further provides a computer program product, including a computer program / instruction, which implements the above-mentioned information prediction method when the computer program / instruction is executed by a processor.

[0196] An embodiment of the present application further provides an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, the information prediction method as described above is implemented.

[0197] An embodiment of the present application further provides a non-volatile readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the information prediction method described above is implemented.

[0198] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0199] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0200] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0201] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0202] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0203] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0204] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0205] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0206] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.

[0207] The above is a detailed introduction to the provided information prediction method and device. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for information prediction, characterized in that: The method comprises: Get the current word input by the user; Using an information prediction tool to predict the current word-gram to obtain current prediction information; wherein the information prediction tool is provided with a state space model, the output equation of the state space model is obtained by adjusting according to a target parameter matrix set, the target parameter matrix set is determined from a historical parameter matrix set, and the historical parameter matrix set is obtained according to historical word-gram information; The current forecast information is presented to the user.

2. The method according to claim 1, characterized in that Before obtaining the current word element input by the user, the method further includes: Inputting information corresponding to the plurality of sample word units into a preset state space model to obtain a current parameter matrix of a first sample word unit and a set of historical parameter matrices of a plurality of second sample word units, wherein the plurality of sample word units include the first sample word unit and the plurality of second sample word units; Comparing the current parameter matrix with the plurality of historical parameter matrix sets for similarity to obtain a similarity comparison result; According to the similarity comparison result, a target parameter matrix set is selected from the plurality of historical parameter matrix sets, and an output equation in the state space model is adjusted according to the target parameter matrix set; The prediction information corresponding to the first sample word is generated using the adjusted output equation in the state-space model.

3. The method according to claim 2, characterized in that The comparing the current parameter matrix with the plurality of historical parameter matrix sets to obtain a similarity comparison result includes: Obtain each historical parameter matrix in the historical parameter matrix set; A similarity calculation is performed on the current parameter matrix and each of the historical parameter matrices to obtain a similarity result.

4. The method according to claim 3, characterized in that The performing similarity calculation on the current parameter matrix and each of the historical parameter matrices to obtain a similarity result includes: The Euclidean distance between the current parameter matrix and each of the historical parameter matrices is calculated, and the Euclidean distance is determined as the similarity result.

5. The method according to claim 3, characterized in that: Before comparing the current parameter matrix with the plurality of historical parameter matrix sets for similarity to obtain a similarity comparison result, the method further includes: Inputting information corresponding to a plurality of training word units into the state space model to obtain a current prediction output matrix of a first word unit and a set of historical prediction output matrices of a plurality of second word units; The step of selecting a target parameter matrix set from the plurality of historical parameter matrix sets based on the similarity comparison result includes: Sorting the similarity results to obtain a sorted historical parameter matrix set; According to a preset target number, a target parameter matrix set is screened out from the sorted historical parameter matrix set, wherein the number of target parameter matrices in the target parameter matrix set is the preset target number; The step of adjusting the output equation in the state space model according to the target parameter matrix set includes: Determine a target index corresponding to each target parameter matrix in the target parameter matrix set; Determining a target prediction output matrix set from the historical prediction output matrix set according to the target index; Fusing the current prediction output matrix corresponding to the first word-gram with the target prediction output matrix set to obtain a fused output matrix; Using an attention mechanism to perform weighted calculation on the fusion output matrix to obtain a weighted calculation result; The output equation in the state space model is adjusted according to the target parameter matrix set and the weighted calculation result.

6. The method according to claim 2, characterized in that The step of generating prediction information corresponding to the first sample word by using the adjusted output equation in the state space model includes: The output equation in the adjusted state-space model is used to generate prediction information corresponding to the first sample word-gram; wherein the prediction information includes a plurality of prediction word-grams corresponding to the first sample word-gram and the probability corresponding to each of the prediction word-grams.

7. The method according to any one of claims 1 to 6, characterized in that: The using of the information prediction tool to predict the current word to obtain current prediction information includes: Using the information prediction tool to predict the current word-gram, obtaining a plurality of current predicted word-grams and word-gram probabilities corresponding to the plurality of current predicted word-grams; Sorting the multiple current predicted word-grams according to the word-gram probabilities to obtain sorted current prediction information; The presenting the current prediction information to the user comprises: The sorted current prediction information is presented to the user.

8. An information prediction device, characterized in that: include: A first acquisition module, used to acquire the current word unit input by the user; A first prediction module is used to predict the current word using an information prediction tool to obtain current prediction information; wherein the information prediction tool is provided with a state space model, and the output equation of the state space model is obtained by adjusting according to a target parameter matrix set, wherein the target parameter matrix set is determined from a historical parameter matrix set, and the historical parameter matrix set is obtained according to historical word units; The first presentation module is used to present the current prediction information to the user.

9. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the method for information prediction according to any one of claims 1 to 7 is implemented.

10. A non-volatile readable storage medium, characterized in that: The non-volatile readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for information prediction according to any one of claims 1 to 7 is implemented.

Images