Information prediction method, device, equipment, and medium

Abstract

The information prediction method provided in the embodiment of the present application is applied to the field of computer technology, obtains the current word element input by the user, uses the information prediction tool to predict the current word element, and obtains current prediction information; wherein, the information prediction tool is provided with a state space model, and the output equation of the state space model is obtained by adjusting according to the target parameter matrix set, the target parameter matrix set is determined from the historical parameter matrix set, and the historical parameter matrix set is obtained according to the historical word element information. The current prediction information is presented to the user, and the state space model is adjusted by the historical parameter matrix set corresponding to the historical word element information to predict the input information, thereby improving the context relevance of the word element, saving memory space, and reducing the amount of calculation.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, device, and medium for information prediction. Background Art

[0002] In recent years, with the development of deep learning, deep learning models based on the Transformer architecture have achieved tremendous success in natural language processing, specifically in input method prediction. However, the computational and spatial complexity of the Transformer increases quadratically with sequence length, creating a bottleneck for processing extremely long sequences. The computational overhead of long sequences is not limited to the training phase; it is also significant during inference.

[0003] Using a state-space model for input method prediction can process word metadata. However, the output of the current state in a state-space model must be forcibly associated with all historical states corresponding to the previous word, which increases the introduction of invalid information. Furthermore, the output at the current moment in a state-space model is determined by the historical state and the current input, but the hyperparameters before the historical state and the current input are determined only by the current input, which prevents the full utilization of historical state information. Therefore, input method prediction based on the state-space model is relatively computationally intensive, and the prediction content cannot fully reference historical content. Summary of the Invention

[0004] In view of the above problems, a method, apparatus, device, and medium for information prediction are proposed to overcome the above problems or at least partially solve the above problems, including:

[0005] In a first aspect of the present application, a method for information prediction is provided, characterized in that the method comprises:

[0006] Get the current word input by the user;

[0007] Predicting the current word-gram using an information prediction tool to obtain current prediction information; wherein the information prediction tool is provided with a state-space model, and an output equation of the state-space model is adjusted based on a target parameter matrix set, wherein the target parameter matrix set is determined from a historical parameter matrix set, and wherein the historical parameter matrix set is obtained based on historical word-gram information;

[0008] The current forecast information is presented to the user.

[0009] Optionally, before obtaining the current word element input by the user, the method further includes:

[0010] Inputting information corresponding to the plurality of sample word-grams into a preset state space model to obtain a current parameter matrix of a first sample word-gram and a set of historical parameter matrices of a plurality of second sample word-grams, the plurality of sample word-grams including the first sample word-gram and the plurality of second sample word-grams;

[0011] Comparing the current parameter matrix with the plurality of historical parameter matrix sets for similarity to obtain a similarity comparison result;

[0012] According to the similarity comparison result, a target parameter matrix set is selected from the plurality of historical parameter matrix sets, and an output equation in the state space model is adjusted according to the target parameter matrix set;

[0013] Prediction information corresponding to the first word-gram is generated using the adjusted output equation in the state-space model.

[0014] Optionally, performing similarity comparison between the current parameter matrix and the plurality of historical parameter matrix sets to obtain a similarity comparison result includes:

[0015] Obtain each historical parameter matrix in the historical parameter matrix set;

[0016] A similarity calculation is performed on the current parameter matrix and each of the historical parameter matrices to obtain a similarity result.

[0017] Optionally, performing similarity calculation on the current parameter matrix and each of the historical parameter matrices to obtain a similarity result includes:

[0018] The Euclidean distance between the current parameter matrix and each of the historical parameter matrices is calculated, and the Euclidean distance is determined as the similarity result.

[0019] Optionally, before performing similarity comparison between the current parameter matrix and the plurality of historical parameter matrix sets to obtain a similarity comparison result, the method further includes:

[0020] Inputting information corresponding to a plurality of training word units into the state space model to obtain a current prediction output matrix of a first word unit and a set of historical prediction output matrices of a plurality of second word units;

[0021] The step of selecting a target parameter matrix set from the historical parameter matrix set based on the similarity result includes:

[0022] Sorting the similarity results to obtain a sorted historical parameter matrix set;

[0023] According to a preset target number, a target parameter matrix set is screened from the sorted historical parameter matrix set, wherein the number of target parameter matrices in the target parameter matrix set is the preset target number;

[0024] The adjusting the output equation in the state-space model according to the target parameter matrix set includes:

[0025] Determine a target index corresponding to each target parameter matrix in the target parameter matrix set;

[0026] Determining a target prediction output matrix set from the historical prediction output matrix set according to the target index;

[0027] Fusing the current prediction output matrix corresponding to the first word with the target prediction output matrix set to obtain a fused output matrix;

[0028] Using an attention mechanism to perform weighted calculation on the fusion output matrix to obtain a weighted calculation result;

[0029] An output equation in the state space model is adjusted according to the target parameter matrix set and the weighted calculation result.

[0030] Optionally, the generating prediction information corresponding to the first sample word using the adjusted output equation in the state-space model includes:

[0031] The output equation in the adjusted state-space model is used to generate prediction information corresponding to the first sample word-gram; wherein the prediction information includes a plurality of predicted word-grams corresponding to the first sample word-gram and a probability corresponding to each of the predicted word-grams.

[0032] Optionally, the using an information prediction tool to predict the current word to obtain current prediction information includes:

[0033] Using the information prediction tool to predict the current word-gram, obtaining a plurality of current predicted word-grams and word-gram probabilities corresponding to the plurality of current predicted word-grams;

[0034] sorting the multiple current predicted word-grams according to the word-gram probabilities to obtain sorted current prediction information;

[0035] The presenting the current prediction information to the user includes:

[0036] The sorted current prediction information is presented to the user.

[0037] In a second aspect of the present application, there is also provided an information prediction device, characterized in that it includes:

[0038] A first acquisition module is used to acquire the current word input by the user;

[0039] a first prediction module, configured to predict the current word-gram using an information prediction tool to obtain current prediction information; wherein the information prediction tool is provided with a state-space model, wherein an output equation of the state-space model is adjusted based on a target parameter matrix set, wherein the target parameter matrix set is determined from a historical parameter matrix set, wherein the historical parameter matrix set is obtained based on historical word-grams;

[0040] A first presentation module is configured to present the current prediction information to the user.

[0041] In the third aspect of the implementation of the present application, the embodiments of the present application also disclose an electronic device, including: a processor; and a memory, on which executable code is stored. When the executable code is executed, the processor executes the information prediction method as described in one or more of the embodiments of the present application.

[0042] In the fourth aspect of the implementation of the present application, the embodiments of the present application further disclose a non-volatile readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method of information prediction as described in one or more of the embodiments of the present application is implemented.

[0043] The embodiments of the present application also disclose a computer program product, including a computer program / instruction, which, when executed by a processor, implements the information prediction method as described in one or more embodiments of the present application.

[0044] The embodiments of the present application have the following advantages:

[0045] The current word-gram input by the user is obtained, and the current word-gram is predicted using an information prediction tool to obtain current prediction information; wherein, the information prediction tool is provided with a state-space model, and the output equation of the state-space model is obtained by adjusting the target parameter matrix set, and the target parameter matrix set is determined from the historical parameter matrix set, and the historical parameter matrix set is obtained based on historical word-gram information. The current prediction information is presented to the user, and the state-space model is adjusted by the historical parameter matrix set corresponding to the historical word-gram information to predict the input information, thereby improving the contextual relevance of the word-gram, saving memory space, and reducing the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the description of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 One of the flow charts of the steps of the information prediction method provided in some embodiments of the present application;

[0048] Figure 2 A schematic diagram of the structure of a state space model provided in some embodiments of the present application;

[0049] Figure 3 A schematic diagram of a parallel computing process provided for some embodiments of the present application;

[0050] Figure 4a A schematic diagram of a continuous addition operation provided in some embodiments of the present application;

[0051] Figure 4b The continuous addition operation y provided in some embodiments of the present application n Calculation diagram;

[0052] Figure 5 It is a structural diagram of an information prediction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments described are only a portion of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0054] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those skilled in the art that in each embodiment of the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be implemented. The division of the following embodiments is for convenience of description and should not constitute any limitation on the specific implementation of the present application. The various embodiments can be combined with each other and referenced to each other under the premise of no contradiction.

[0055] In recent years, with the development of deep learning, especially deep learning models based on the Transformer structure, great success has been achieved in natural language processing, image processing and other fields. The Transformer model has a strong representation ability and can capture long-distance dependencies, and performs well in processing sequence data. However, the computational and space complexity of the Transformer increases quadratically with the increase of sequence length (O(n 2 )), which creates a bottleneck for processing very long sequence data. The computational overhead of long sequences exists not only in the training phase, but also in the inference phase. Therefore, finding algorithms and models that can process long sequence data at lower computational and storage costs has become an important research direction. This application predicts the current word unit input by the user by setting an information prediction tool with an adjusted state space model, thereby improving the contextual relevance of the word unit in the information prediction, saving memory space and reducing the amount of computation.

[0056] Reference Figure 1 , shows one of the step flow charts of the information prediction method provided by some embodiments of the present application, the method may include:

[0057] Step 101: Obtain the current word input by the user.

[0058] The above-mentioned user input operations can be specifically various applications, which run on various devices, including but not limited to: smartphones, tablets, e-book readers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, laptops, car computers, desktop computers, set-top boxes, smart TVs, wearable devices, etc.

[0059] The information prediction method mentioned in the embodiment of the present application can support multiple input methods. Among them, the input method can be a coding method used to input various symbols into electronic devices such as computers and mobile phones. The user can use the input method application to conveniently input the required characters or strings into the electronic device. It should be noted that in the embodiment of the present application, in addition to supporting common Chinese input methods (such as pinyin input method, Wubi input method, Zhuyin input method, voice input method, handwriting input method, etc.), the input method can also support input methods of other languages ​​(such as Japanese Hiragana input method, Korean input method, etc.). No limitation is made to the input method and the language type of the input method.

[0060] In this embodiment, the current token input by the user is obtained. Suppose the user inputs the current token "Hello" in the client of a certain application, and the input method needs to predict the next possible character or word. The information prediction tool will predict the next possible character or word based on the context information of "Hello", such as ",", "!", "world", etc.

[0061] The above-mentioned current token input by the user can support multiple input methods. For example, it supports input methods such as input through the input method and paste input. Among them, the input method through the input method can include, but is not limited to, text input, voice input, etc.

[0062] In some embodiments of the present application, before step 101, it further includes:

[0063] S1: Input the information corresponding to multiple sample tokens into a pre-set state space model to obtain the current parameter matrix of the first sample token and a set of historical parameter matrices of multiple second sample tokens. The multiple sample tokens include the first sample token and multiple second sample tokens.

[0064] Among them, the sample token refers to a large amount of sample data that needs to be input during model training. For the sake of distinction, the present application divides multiple sample tokens into the first sample token and multiple second sample tokens. The first sample token refers to the latest input sample token, and the multiple second sample tokens refer to multiple historical tokens before the first sample token. The embodiments of the present application take the language model modeling in the natural language processing of the input method as an example. The modeling goal of the language model is that the constructed model can judge whether a piece of text is a sentence that can be normally understood by humans. According to this goal, in the application of the input method, we can put numerous corpora into the model to make its predicted output the output at the corresponding position. The overall structure of the model is an autoregressive language model, that is, it can only see the information before the current position. The input of the language model is a series of vectorized tokens, and the output is the probability of the next token.

[0065] Specifically, a state space model (State Space Model, SSM) can be used as the speech model applied to the input method in the present application. The state space model is a statistical model used to describe and predict time series data. The state space model captures the dynamic behavior of the system by defining a state equation and an output equation. The state equation describes the evolution of the system state, and the output equation associates the state and the observed data.

[0066] As Figure 2 shown, it is a schematic structural diagram of the state space model provided by some embodiments of the present application. The figure shows the calculation process, and the code corresponding to the calculation process in the figure is as follows:

[0067] Input; x: (B, L, D)

[0068] Output; y: (B, L, D)

[0069] 1. A: (D, N) ← some_structured_matrix

[0070] 2. B:(B, L, N)←SB(x)

[0071] 3. C:(B, L, N)←SC(x)

[0072] 4. Δ: (B, L, D) ←τΔ (parameter+sΔ(x))

[0073] 5. A, B: (B, L, D, N) ← discretize (Δ, A, B)

[0074] 6. y←SSM(A, B, C)(x)

[0075] 7. return y

[0076] against Figure 2 The computational flow and the code above are detailed. The input is x, whose shape is (B, L, D), where B is the batch size, L is the sequence length, and D is the dimension. The input x is projected to obtain xz, which is mapped from (B, L, D) to (B, L, d_inner*2) through a linear layer and then rearranged to (B, d_inner*2, L). xz is separated to obtain x and z, both of which have shapes (B, d_inner, L).

[0077] Next, x and z are calculated in two paths. In the first path, x undergoes a 1-dimensional convolution operation to obtain the output x_conv, with the shape remaining unchanged. The convolution output x_conv is mapped to x_dbl via linear mapping, with the shape of x_dbl being (B*L, dt_rank+2*d_state). x_dbl is split into three parts in the third dimension: dt, B, and C. dt is then linearly mapped to a shape of (B, d_inner, L). The state and output of dt, B, C, and A (where A is a D*N matrix) are calculated using the SSM discrete formula, with the output shape being (B, L, D). In the second path, the z path passes through a nonlinear sigmoid activation function and is point-wise multiplied with the output of the SSM discrete formula to obtain the output of the final mamba block. The shape of d_state is equivalent to N in the formula, and d_ineer is equivalent to D in the formula.

[0078] The above steps describe the input and output of a block. A block is a basic building block in a model and typically contains multiple sublayers. These blocks are repeatedly stacked within the model to form a deeper network structure, thereby enhancing the model's expressiveness and performance. When training or using a language model, having a series of such blocks allows us to obtain the probability distribution of the next word.

[0079] From the above, we can see that inputting a value x into a preset state-space model yields the corresponding A, B, and C matrices. In this embodiment, the B matrix is ​​referred to as the parameter matrix. Therefore, during model training, by inputting multiple sample word-grams, including a first sample word-gram and multiple second sample word-grams, into the preset state-space model, we can obtain the current parameter matrix corresponding to the first sample word-gram, and also obtain multiple sets of historical parameter matrices corresponding to the multiple second sample word-grams.

[0080] The calculation process of the SSM discrete formula in the above content is explained in detail below, where the input is word unit x and the output is word unit y.

[0081] h'(t)=Ah(t)+Bx(t)

[0082] y(t)=Ch(t)

[0083]

[0084] y k =Ch k

[0085]

[0086] Among them, h,(t) is the state equation, x(t) maps the input sequence, h(t) is the hidden state, and y(t) is the output equation. The specific sources of A, B, and C have been explained in the previous steps and will not be repeated here. The calculation method is to perform matrix calculation on dt and A in the above steps; The calculation method is to calculate the matrix of dt and B in the previous step, and C is the original C matrix in the previous step. If the A, B, C, and dt matrices are token (word or word) independent, that is, their dimensions are D or D*N, then the calculation process of the SS M formula can be completed in parallel (CNN form) or serially (RNN form), such as Figure 3Shown is a schematic diagram of the parallel computing process provided by some embodiments of the present application. It can be seen from the figure that the A, B, and C matrices are independent of the word units, so the product can be calculated in parallel first, and then the summation can be performed. The preset state space model can maintain linear complexity (O(n)) when processing long sequences, which gives it significant potential in long sequence tasks. The preset state space model can convert the processing of long sequence data into a state and input processing process, thereby reducing the amount of calculation. In addition, the structure of the state space model can naturally adapt to dynamic changes in time series, which is of great significance in practical applications.

[0087] The Mamba model is an advanced sequence modeling architecture that is based on the concept of state-space model. However, the Mamba model cannot be trained in parallel using the CNN acceleration method mentioned above. The reason is that for each token, the A, B, and C are inconsistent. In other words, the convolution kernel coefficients before x0, x1, and x2 cannot be kept consistent when calculating each output. n The coefficients will change when calculating .

[0088] Therefore, in some embodiments of the present application, a new "add" operation is defined. In parallel computing, consecutive addition operations can be performed in parallel. The new "add" operation specifically refers to (A t , B t x t )plus(A t+1 , B t+1 x t+1 )=(A t A t+1 , A t+1 B t x t +B t+ 1x t+1 ). The specific parallel operation of the continuous addition operation is: first calculate each x in parallel n Corresponding to A, B, C respectively; then use the continuous addition operation to get (A t A t+1 , A t+1 B t x t +B t+1 x t+1 ); At this time, y1=C1(A1B0x0+B1x1), y2=C2(A2(A1B0x0+B1x1)+B2x2), and so on, any y n The value of . Figure 4a , which is a schematic diagram of the continuous addition operation provided in some embodiments of the present application, from Figure 4aAs can be seen, each pair can be calculated in parallel, and the results of the parallel calculation can be further calculated in parallel in pairs. Figure 4b , which is a continuous addition operation y provided in some embodiments of the present application n Calculation diagram, from Figure 4b It can be seen that the “add” operation defined in the embodiment of the present application can also be calculated in parallel in groups of two, and any y can be obtained. n The value of .

[0089] S2: performing a similarity comparison between the current parameter matrix and a plurality of historical parameter matrix sets to obtain a similarity comparison result;

[0090] It can be inferred from the formula in the above content that during the model training process, y n The output of is combined with all historical states to complete the word unit conversion. Therefore, in order to predict the current output, it is necessary to forcibly associate it with all historical states, which increases the introduction of invalid information.

[0091] Based on this, this application proposes an improved method to introduce sparsity to multiple parameter matrices B during model training. Introducing sparsity means making the values ​​of most elements or parameters zero or close to zero through certain means, thereby reducing the dimensionality of the data, reducing computational complexity, or improving the generalization ability of the model; it can be understood that sparsity can reduce the dimensionality of the data and remove redundant information, thereby reducing computational complexity and storage requirements.

[0092] In the specific implementation, in order to introduce sparsity, first the parameter matrix B corresponding to the current input and the historical parameter matrices B at the historical positions are calculated. i Do similarity calculation, similarity is high B i Only those who participate in the current moment y n The calculation of the data is performed, and the maximum length does not exceed the set length, so as to remove unimportant historical information, reduce the amount of calculation and reduce the occupied storage space.

[0093] In some embodiments of the present application, the following sub-steps may be included before step S2:

[0094] S11: Inputting information corresponding to a plurality of training word-grams into the state space model to obtain a current prediction output matrix of a first word-gram and a set of historical prediction output matrices of a plurality of second word-grams.

[0095] As shown in step S1, the input is x, with a shape of (B, L, D), where B is the batch size, L is the sequence length, and D is the dimension. The input x is projected to obtain xz, which is then separated to obtain x and z, both of which have shapes (B, d_inner, L). x is convolved with a 1D convolution to obtain the output x_conv, which retains its shape. The convolution output x_conv is then linearly mapped to x_dbl, with a shape of (B*L, dt_rank+2*d_state). x_dbl is then split into three parts in the third dimension: dt, B, and C.

[0096] Among them, the matrix C corresponding to each input x is the predicted output matrix mentioned in the embodiment of this application. In the state space model, the C matrix is ​​used to map the state vector of the system to the output vector. It can be understood that there are multiple predicted output matrices corresponding to multiple training word units. Specifically, the matrix C corresponding to the first word unit is called the current predicted output matrix, and the multiple matrices C corresponding to the multiple historical word units (second word units) input before the first word unit are the historical predicted output matrix sets.

[0097] In some embodiments of the present application, step S2 includes the following sub-steps:

[0098] S21: Obtain each historical parameter matrix in the historical parameter matrix set;

[0099] In the specific implementation, each x n The corresponding B matrix has a dimension of D*N, so all B matrices can be calculated in parallel. In other words, each historical parameter matrix in the historical parameter matrix set can be obtained first to prepare for the next step of calculation.

[0100] S22: Perform similarity calculation on the current parameter matrix and each historical parameter matrix to obtain a similarity result.

[0101] Specifically, the current position B can be calculated n And the historical parameter matrices B0, B1, B2, ..., B corresponding to all previous positions n-1 Perform similarity calculation to obtain a similarity result.

[0102] In some embodiments of the present application, sub-step S22 includes the following sub-steps:

[0103] S221: Calculate the Euclidean distance between the current parameter matrix and each of the historical parameter matrices, and determine the Euclidean distance as the similarity result.

[0104] Among them, Euclidean distance is a commonly used method to measure the similarity between two vectors or matrices.

[0105] In the specific implementation, the Euclidean distance can be used to calculate the B of the current position n and all previous positions B0-B n-1 Similarity sim i The steps for calculating the Euclidean distance are as follows: First, determine the current parameter matrix B n and the historical parameter matrix B i , where B i B0-B n-1 Any one of them, it can be understood that B n and B i The dimensions of the two matrices are the same; secondly, calculate the difference D between the corresponding elements of the two matrices ij Where i and j represent the row and column indices of the matrix, respectively. Next, the square of the difference matrix is ​​calculated. Then, the sum of the squares of all elements is calculated. Finally, the square root of the sum of the squares of all elements is taken to obtain the Euclidean distance. The obtained Euclidean distance is determined as the similarity result.

[0106] In some cases, the Euclidean distance can be used directly as the similarity result. However, in general, the similarity result will be a value between 0 and 1, indicating the degree of similarity between the two matrices. Therefore, the Euclidean distance can also be normalized so that the smaller the Euclidean distance (i.e., the more similar the two matrices are), the closer the similarity result is to 1; the larger the Euclidean distance (i.e., the less similar the two matrices are), the closer the similarity result is to 0.

[0107] S3: Filtering a target parameter matrix set from the plurality of historical parameter matrix sets based on the similarity comparison result, and adjusting an output equation in the state space model based on the target parameter matrix set;

[0108] In a specific implementation, the following process can be used to compare similarities, screen the target parameter matrix set, and adjust the output equation.

[0109] sim i =Bn·(B i ),i <n

[0110] sel(y n (m))=Top m (sim i ),i <n

[0111]

[0112] First, calculate B at the current position n and all previous positions B0-B n-1 The similarity comparison results sim i, where i ranges from 0 to n-1, and the nearest fixed number m is selected. The fixed number m can be a preset number and can be adjusted according to the actual situation. The result is expressed as sel(y n (m)), at this time sel(y n (m)) is the target parameter matrix set. For example, Top m (sim i ) means to select the m with the highest similarity from the similarity comparison results. Based on the introduction of sparsity, a parallel scanning algorithm is performed to complete the current moment y n The calculation process is to adjust the output equation of the state space model according to the target parameter matrix set.

[0113] In the specific implementation, the parallel scanning algorithm can be expressed as SparsePara scan , SparsePara scan The core implementation is to select the x corresponding to the index according to the determined target parameter matrix n , A n , and then the selected index is calculated in parallel layer by layer according to the parallel scanning algorithm. For example, the current parameter matrix corresponding to the current input is B 100 , according to the similarity calculation results, the top 5 historical parameter matrices with similarity are selected as the target parameter matrix set (B 21 、B 35 、B 49 、B 88 、B 97 ), the corresponding index is (21, 35, 49, 88, 97), and then locate the corresponding (x 21 、x 35 、x 49 、x 88 、x 97 )、(A 21 、A 35 、A 49 、A 88 、A 97 ), and then according to the parallel scanning algorithm SparsePara scan For the selected index, the y corresponding to the current moment is calculated layer by layer in parallel. n .

[0114] Among them, introducing sparsity means that the values ​​of most elements or parameters are made zero or close to zero through the embodiment of the present application, thereby reducing the dimension of the data, reducing the computational complexity or improving the generalization ability of the model; it can be understood that in the embodiment of the present application, by calculating the similarity and filtering based on the similarity, the sparsity is introduced, which can capture important historical data while ignoring noise and irrelevant information. In addition, due to the introduction of sparsity, most operations can be skipped on zero elements, reducing unnecessary calculations, thereby reducing memory usage and improving calculation speed.

[0115] In some embodiments of the present application, step S3 of "screening a target parameter matrix set from the multiple historical parameter matrix sets based on the similarity comparison result" may further include the following sub-steps:

[0116] S31: sorting the similarity results to obtain a sorted historical parameter matrix set;

[0117] S32: According to a preset target number, a target parameter matrix set is screened out from the sorted historical parameter matrix set, wherein the number of target parameter matrices in the target parameter matrix set is the preset target number.

[0118] In the specific implementation, calculate the B of the current position n and all previous positions B0-B n-1 Similarity comparison results sim0-sim n-1 , and select the 2048 closest ones, the selection result is expressed as sel(y n (2048)), at this time sel(y n (2048)) is the target parameter matrix set, and then a sparse parallel scanning algorithm is used to complete the current moment y n calculation process.

[0119] sim i =Bn·(B i ),i <n

[0120] sel(y n (2048))=Top 2048 (sim i ),i <n

[0121]

[0122] In some embodiments of the present application, in step S3, “adjusting the output equation in the state-space model according to the target parameter matrix set” may further include the following sub-steps:

[0123] S33: Determine the target index corresponding to each target parameter matrix in the target parameter matrix set;

[0124] From the above content, we can know that after calculating the similarity between the current B matrix and the historical matrix set, we select the multiple historical B matrices with the highest similarity as the target parameter matrix set according to the similarity sorting, and then we can determine the index corresponding to the B matrix as the target index. The target index is also B i The set of corresponding i in .

[0125] S34: Determine a target prediction output matrix set from the historical prediction output matrix set according to the target index;

[0126] From the above content, it can be seen that the calculation of the predicted output matrix C in the related art is actually calculated using the current input x, that is, the matrix C calculated by the parameter output equation only refers to the current input and cannot be dynamically adjusted based on historical information.

[0127] In some embodiments of the present application, in order to dynamically select the historical state and the current input, C is further improved, and the target parameter matrix set B is used. i The corresponding target index determines multiple corresponding C matrices from the multiple C matrices corresponding to the historical positions i , multiple C i That is, the target prediction output matrix set. It can be understood that, due to the sparsity introduced in the above steps, the target prediction output matrix set determined at this time is also completed based on sparsity.

[0128] S35: Fusing the current prediction output matrix corresponding to the first word with the target prediction output matrix set to obtain a fused output matrix;

[0129] In a specific implementation, the target prediction output matrix set and the current prediction output matrix C matrix are used to perform attention calculations. Of course, such attention calculations will also be completed on the basis of sparsity. The Attention mechanism is a mechanism widely used in deep learning, especially in the fields of natural language processing (NLP) and computer vision. Its main purpose is to allow the model to dynamically focus on different parts of the input sequence when processing sequence data, thereby improving the performance of the model. Based on this, the embodiment of the present application not only adds sparsity but also ensures that the remaining state data is added to the current state according to the weight.

[0130] Every x n The dimension of the corresponding prediction output matrix C is N, so the target prediction output matrix set and the current prediction output matrix C can be fused to obtain a fused matrix C 融合This application does not impose any specific restrictions on the specific fusion method.

[0131] S36: Using the attention mechanism to perform weighted calculation on the fusion output matrix to obtain a weighted calculation result.

[0132] In the embodiment of this application, C 融合 (The fused target prediction output matrix set and the current prediction output matrix C matrix) perform Attention calculation. The Attention calculation at this time is to obtain the attention allocation score of the historical state.

[0133] Q=C 融合 *W Q

[0134] K=C 融合 *W k

[0135] V=C 融合 *W v

[0136]

[0137] Among them, Q, K, and V correspond to query, key, and value respectively, d k is the feature dimension, W Q 、W K 、W V The weight matrix can be a preset one or a trained one. The fusion output matrix is ​​weighted by the attention mechanism to obtain the weighted calculation result.

[0138] S37: Adjusting the output equation in the state space model according to the target parameter matrix set and the weighted calculation result.

[0139]

[0140] According to the above steps, the target parameter matrix set and the weighted calculation results obtained by attention calculation are used to adjust the output equation in the state space model. Specifically, the final weighted calculation result can be replaced by C in the above step. n Complete the output equation y n , so that the output equation is adjusted according to the target parameter matrix set and the weighted calculation result.

[0141] S4: Generate prediction information corresponding to the first sample word using the adjusted output equation in the state-space model.

[0142] In the specific implementation, we calculate y according to the scanning parallel operation process of the above steps nBecause the output equations of the state-space model incorporate sparsity and a dynamic selection mechanism, the parallel scanning algorithm's "add" operation can still be used. Parallel continuous addition operations are performed layer by layer on the sparse index. At each moment, x is calculated with its own A and B matrices and then summed. In other words, the output equations of the adjusted state-space model are used to generate prediction information corresponding to the first sample word.

[0143] By introducing sparsity in the state-space model, the current token can be selectively made to pay attention to the tokens in the historical sequence, which is helpful for better context association and memory and computational savings. By adding an attention mechanism operation (attention operation) to the output gate of the state-space model, the weight of more relevant outputs can be made larger; in addition, the attention mechanism operation (attention operation) does not bring quadratic complexity due to the introduction of sparsity and length restrictions, but will increase the correlation between contextual semantics. Moreover, the state-space model introduces sparsity and a dynamic selection mechanism, and a parallel scanning algorithm can still be used. By using the new "addition" operation provided in the embodiment of the present application, parallel addition operations can be completed layer by layer on the sparse index. The improved scheme does not increase the amount of computation during reasoning. It can perform reasoning by accumulating states in the manner of RNN, and the historical states that need to be saved are B and C matrices of finite length.

[0144] In some embodiments of the present invention, step S4 may further include the following sub-steps:

[0145] S41: Generate prediction information corresponding to the first sample word-gram using the output equation in the adjusted state-space model; wherein the prediction information includes multiple predicted word-grams corresponding to the first sample word-gram and the probability corresponding to each predicted word-gram.

[0146] The similarity calculation result introduces sparsity into the parameter matrix. On the basis of sparsity, the prediction output matrix is ​​fused and weighted by the attention mechanism. Therefore, the output equation of the adjusted state space model can be used to generate prediction information corresponding to the first sample word. In some embodiments of the present application, the output vector y nis a probability distribution vector that represents the probability of each word being the next word. We can generate multiple predicted words based on this probability distribution. For example, assuming the first sample word is "hello," there are three predicted words: "world," "goodbye," and "morning." The probability of "world" is 0.67, the probability of "goodbye" is 0.86, and the probability of "morning" is 1.05. The probability of each predicted word is calculated. That is, the prediction information includes the multiple predicted words "world," "goodbye," and "morning" corresponding to the first sample word "hello," as well as the probabilities of 0.67, 0.86, and 1.05 for each predicted word, respectively.

[0147] Step 102: Use an information prediction tool to predict the current word to obtain current prediction information; wherein, the information prediction tool is provided with a state space model, and the output equation of the state space model is adjusted according to a target parameter matrix set, and the target parameter matrix set is determined from a historical parameter matrix set, and the historical parameter matrix set is obtained based on historical word information.

[0148] In some embodiments of the present application, a user may input a current word using an input method in an application client. Accordingly, an information prediction tool may be deployed on a server corresponding to the input method. The information prediction tool is a tool or system for predicting the current word. The information prediction tool may be implemented based on the state-space model provided in the embodiments of the present application. The predicted content corresponding to the current input content is predicted using an adjusted state-space model. The prediction process references the current input, historical data, and a parameter matrix set.

[0149] In some embodiments of the present application, step 102 may include the following sub-steps:

[0150] Sub-step 21: using an information prediction tool to predict the current word-gram, obtaining a plurality of current predicted word-grams and word-gram probabilities corresponding to the plurality of current predicted word-grams;

[0151] As can be seen from the above content, assuming that the first sample word is "hello", the information prediction tool on the server is called to predict the current word. The resulting predicted word-grams include three word-grams: "world", "goodbye", and "morning". Among them, the probability of "world" is 0.67, the probability of "goodbye" is 0.86, and the probability of "morning" is 1.05. The multiple current predicted word-grams and the word-gram probabilities corresponding to the multiple current predicted word-grams are the prediction information, including the multiple predicted word-grams "world", "goodbye", and "morning" corresponding to the first sample word-gram "hello", and the probabilities of 0.67, 0.86, and 1.05 corresponding to each predicted word-gram, respectively.

[0152] Sub-step 22: sorting the multiple current predicted word-grams according to the word-gram probabilities to obtain sorted current prediction information;

[0153] After obtaining the prediction information, the multiple current predicted word-grams can be sorted according to the word-gram probabilities corresponding to each predicted word-gram to obtain the sorted current prediction information. For example, if the probabilities corresponding to "world", "goodbye", and "morning" are 0.67, 0.86, and 1.05, respectively, the results are 1.05, 0.86, and 0.67. Therefore, the sorted current prediction information can be "morning, 10.05", "goodbye, 0.86", and "world, 0.67".

[0154] Step 103: Present the current prediction information to the user.

[0155] When a user enters a partial word using a mobile phone or computer input method, the input method client captures the user's input and sends it as a request to the input method server. This request contains the information about the currently entered word. After receiving the client's request, the input method server calls the information prediction tool, which uses the state-space model to make a prediction based on the currently entered word. During use, the state-space model calculates a target parameter matrix set based on the currently entered word and historical word information. The output equation is then used to adjust the model's output to obtain the current prediction information. The server returns the prediction information to the client. The prediction information may include the next possible character, word, or phrase.

[0156] After the client receives the prediction information returned by the server, it will present it to the user. The presentation methods may include: a list of candidate words, automatic completion, prompt information, and other methods. Among them, the list of candidate words specifically refers to displaying a list of candidate words on the input method interface, and the user can complete the input by selecting one of the candidate words. Automatic completion specifically means that if the prediction information is accurate enough, the client can directly automatically complete the current input token, and the user only needs to confirm or modify it. The prompt information can specifically be to display prompt information below the input box to help the user complete the input faster.

[0157] For example, assume that the user enters "Hello," in a mobile input method in a certain application. The client will send "Hello," as a request to the input method server. After the server receives the request, it calls the information prediction tool. The information prediction tool uses the state space model for prediction and obtains the next possible character or word, such as "world," "!", "world!" and so on. The server returns this prediction information to the client, and the client displays a list of candidate words on the input method interface. The user can select "world" to complete the input. Or the user enters "graduate student" on some consulting platforms. The client sends "graduate student" as a request to the server. After the server receives the request, it calls the information prediction tool. The information prediction tool uses the state space model for prediction and obtains the next possible phrase as "exam." The server returns the prediction information to the client, and the client automatically completes "graduate student" as "graduate student exam" on the interface, thereby improving the user's input efficiency

[0158] In some embodiments of the present application, step 103 may further include the following sub-steps:

[0159] Sub-step 31: Present the sorted current prediction information to the user.

[0160] Since the prediction information may include multiple predicted tokens corresponding to the first sample token and the probabilities respectively corresponding to each predicted token, when presenting to the user, the sorted current prediction information can be presented. For example, after entering "hello," the corresponding sorted prediction information may be "morning," "goodbye," "world." And because the output equation of the state space model in the present application has been adjusted to only refer to meaningful historical states, the calculation amount can be smaller, so the input efficiency of the user can be improved and the accuracy of the prediction can be improved.

[0161] The information prediction method provided in the embodiment of the present application obtains the current word element input by the user, uses the information prediction tool to predict the current word element, and obtains current prediction information; wherein, the information prediction tool is provided with a state space model, and the output equation of the state space model is obtained by adjusting according to the target parameter matrix set, the target parameter matrix set is determined from the historical parameter matrix set, and the historical parameter matrix set is obtained according to the historical word element information. The current prediction information is presented to the user, and the state space model is adjusted by the historical parameter matrix set corresponding to the historical word element information to predict the input information, thereby improving the context relevance of the word element, saving memory space, and reducing the amount of calculation.

[0162] Reference Figure 5 , shows a schematic structural diagram of an information prediction device provided by an embodiment of the present application, such as Figure 5 As shown, the device may include:

[0163] The first acquisition module 501 is used to acquire the current word input by the user.

[0164] The first prediction module 502 is used to use an information prediction tool to predict the current word to obtain current prediction information; wherein, the information prediction tool is provided with a state space model, and the output equation of the state space model is adjusted according to a target parameter matrix set, and the target parameter matrix set is determined from a historical parameter matrix set, and the historical parameter matrix set is obtained based on historical word information.

[0165] The first presentation module 503 is configured to present the current prediction information to the user.

[0166] In an optional embodiment of the present application, the information prediction device further includes:

[0167] A first training module is configured to input information corresponding to a plurality of sample word-grams into a preset state-space model to obtain a current parameter matrix of the first sample word-gram and a set of historical parameter matrices of a plurality of second sample word-grams, wherein the plurality of sample word-grams include the first sample word-gram and the plurality of second sample word-grams;

[0168] A first comparison module is configured to perform similarity comparison between the current parameter matrix and a plurality of historical parameter matrix sets to obtain a similarity comparison result;

[0169] a first screening module, configured to screen a target parameter matrix set from the plurality of historical parameter matrix sets based on the similarity comparison result, and adjust an output equation in the state-space model based on the target parameter matrix set;

[0170] The first adjustment module is configured to generate prediction information corresponding to the first sample word by using the adjusted output equation in the state-space model.

[0171] In an optional embodiment of the present application, the first comparison module further includes:

[0172] A first acquisition submodule is configured to acquire each historical parameter matrix in the historical parameter matrix set;

[0173] The first comparison submodule is used to perform similarity calculation on the current parameter matrix and each historical parameter matrix to obtain a similarity result.

[0174] In an optional embodiment of the present application, the first comparison submodule further includes:

[0175] The first comparison unit is configured to calculate the Euclidean distance between the current parameter matrix and each of the historical parameter matrices, and determine the Euclidean distance as the similarity result.

[0176] In an optional embodiment of the present application, the information prediction device further includes:

[0177] The second training module is used to input information corresponding to multiple training word units into the state space model to obtain a current prediction output matrix of the first word unit and a set of historical prediction output matrices of multiple second word units.

[0178] The first screening module further includes:

[0179] A first sorting submodule is used to sort the similarity results to obtain a sorted historical parameter matrix set;

[0180] The first screening submodule is configured to screen out a target parameter matrix set from the sorted historical parameter matrix set according to a preset target number, wherein the number of target parameter matrices in the target parameter matrix set is the preset target number.

[0181] The first screening module further includes:

[0182] An index determination submodule, configured to determine a target index corresponding to each target parameter matrix in the target parameter matrix set;

[0183] A first determining submodule is configured to determine a target prediction output matrix set from the historical prediction output matrix set according to the target index;

[0184] A first fusion submodule, configured to fuse the current prediction output matrix corresponding to the first word with the target prediction output matrix set to obtain a fused output matrix;

[0185] A weighted calculation submodule, configured to perform weighted calculation on the fusion output matrix using an attention mechanism to obtain a weighted calculation result;

[0186] The first adjustment submodule is used to adjust the output equation in the state space model according to the target parameter matrix set and the weighted calculation result.

[0187] In an optional embodiment of the present application, the first adjustment module further includes:

[0188] The second adjustment submodule is used to generate prediction information corresponding to the first sample word using the output equation in the adjusted state-space model; wherein the prediction information includes multiple predicted word elements corresponding to the first sample word element and the probability corresponding to each of the predicted word elements.

[0189] In an optional embodiment of the present application, the first prediction module 502 further includes:

[0190] A first prediction submodule is configured to predict the current word-gram using the information prediction tool to obtain a plurality of current predicted word-grams and word-gram probabilities corresponding to the plurality of current predicted word-grams;

[0191] A second sorting submodule is configured to sort the multiple current predicted word units according to the word unit probabilities to obtain sorted current prediction information;

[0192] The first presentation module 503 includes:

[0193] The first presentation submodule is configured to present the sorted current prediction information to the user.

[0194] The information prediction method provided in the embodiment of the present application obtains the current word-gram input by the user, uses the information prediction tool to predict the current word-gram, and obtains the current prediction information; wherein, the information prediction tool is provided with a state-space model, and the output equation of the state-space model is obtained by adjusting according to the target parameter matrix set, the target parameter matrix set is determined from the historical parameter matrix set, and the historical parameter matrix set is obtained according to the historical word-gram information. The current prediction information is presented to the user, and the state-space model is adjusted by the historical parameter matrix set corresponding to the historical word-gram information to predict the input information, thereby improving the context relevance of the word-gram, saving memory space, and reducing the amount of calculation.

[0195] An embodiment of the present application further provides a computer program product, including a computer program / instruction, which implements the above-mentioned information prediction method when the computer program / instruction is executed by a processor.

[0196] An embodiment of the present application further provides an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the information prediction method described above is implemented.

[0197] An embodiment of the present application further provides a non-volatile readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the information prediction method described above is implemented.

[0198] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0199] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0200] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0201] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0202] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0203] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0204] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0205] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0206] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the above elements.

[0207] The above is a detailed introduction to the provided information prediction method and device. Specific examples are used in this article to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method of this application and its core idea. At the same time, for general technical personnel in this field, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on this application.

Claims

1. A method for information prediction, characterized in that: The method comprises: Get the current word input by the user; Predicting the current word-gram using an information prediction tool to obtain current prediction information; wherein the information prediction tool is provided with a state-space model, and an output equation of the state-space model is adjusted based on a target parameter matrix set, wherein the target parameter matrix set is determined from a historical parameter matrix set, and wherein the historical parameter matrix set is obtained based on historical word-gram information; Inputting information corresponding to a plurality of training word units into the state space model to obtain a current prediction output matrix of a first word unit and a set of historical prediction output matrices of a plurality of second word units; Determine a target index corresponding to each target parameter matrix in the target parameter matrix set; Determining a target prediction output matrix set from the historical prediction output matrix set according to the target index; Fusing the current prediction output matrix corresponding to the first word with the target prediction output matrix set to obtain a fused output matrix; Using an attention mechanism to perform weighted calculation on the fusion output matrix to obtain a weighted calculation result; Adjusting an output equation in the state-space model according to the target parameter matrix set and the weighted calculation result; The current forecast information is presented to the user.

2. The method according to claim 1, characterized in that Before obtaining the current word element input by the user, the method further includes: Inputting information corresponding to the plurality of sample word-grams into a preset state space model to obtain a current parameter matrix of a first sample word-gram and a set of historical parameter matrices of a plurality of second sample word-grams, the plurality of sample word-grams including the first sample word-gram and the plurality of second sample word-grams; Comparing the current parameter matrix with the plurality of historical parameter matrix sets for similarity to obtain a similarity comparison result; According to the similarity comparison result, a target parameter matrix set is selected from the plurality of historical parameter matrix sets, and an output equation in the state space model is adjusted according to the target parameter matrix set; Prediction information corresponding to the first sample word is generated using the adjusted output equation in the state-space model.

3. The method according to claim 2, characterized in that The comparing the current parameter matrix with the plurality of historical parameter matrix sets to obtain a similarity comparison result includes: Obtain each historical parameter matrix in the historical parameter matrix set; A similarity calculation is performed on the current parameter matrix and each of the historical parameter matrices to obtain a similarity result.

4. The method according to claim 3, characterized in that The performing similarity calculation on the current parameter matrix and each of the historical parameter matrices to obtain a similarity result includes: The Euclidean distance between the current parameter matrix and each of the historical parameter matrices is calculated, and the Euclidean distance is determined as the similarity result.

5. The method according to claim 3, characterized in that The step of selecting a target parameter matrix set from the plurality of historical parameter matrix sets based on the similarity comparison result includes: Sorting the similarity results to obtain a sorted historical parameter matrix set; According to a preset target number, a target parameter matrix set is screened out from the sorted historical parameter matrix set, wherein the number of target parameter matrices in the target parameter matrix set is the preset target number.

6. The method according to claim 2, characterized in that The generating prediction information corresponding to the first sample word using the adjusted output equation in the state-space model includes: The output equation in the adjusted state-space model is used to generate prediction information corresponding to the first sample word-gram; wherein the prediction information includes a plurality of predicted word-grams corresponding to the first sample word-gram and a probability corresponding to each of the predicted word-grams.

7. The method according to any one of claims 1 to 6, characterized in that: The using of the information prediction tool to predict the current word to obtain current prediction information includes: Using the information prediction tool to predict the current word-gram, obtaining a plurality of current predicted word-grams and word-gram probabilities corresponding to the plurality of current predicted word-grams; sorting the multiple current predicted word-grams according to the word-gram probabilities to obtain sorted current prediction information; The presenting the current prediction information to the user includes: The sorted current prediction information is presented to the user.

8. An information prediction device, characterized in that: include: A first acquisition module is used to acquire the current word input by the user; a first prediction module, configured to predict the current word-gram using an information prediction tool to obtain current prediction information; wherein the information prediction tool is provided with a state-space model, wherein an output equation of the state-space model is adjusted based on a target parameter matrix set, wherein the target parameter matrix set is determined from a historical parameter matrix set, wherein the historical parameter matrix set is obtained based on historical word-grams; A second training module is configured to input information corresponding to a plurality of training word-grams into the state-space model to obtain a current prediction output matrix of a first word-gram and a set of historical prediction output matrices of a plurality of second word-grams; An index determination submodule, configured to determine a target index corresponding to each target parameter matrix in the target parameter matrix set; A first determining submodule is configured to determine a target prediction output matrix set from the historical prediction output matrix set according to the target index; A first fusion submodule, configured to fuse the current prediction output matrix corresponding to the first word with the target prediction output matrix set to obtain a fused output matrix; A weighted calculation submodule, configured to perform weighted calculation on the fusion output matrix using an attention mechanism to obtain a weighted calculation result; A first adjustment submodule, configured to adjust an output equation in the state space model according to the target parameter matrix set and the weighted calculation result; A first presentation module is configured to present the current prediction information to the user.

9. An electronic device, characterized in that: The invention comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the method for information prediction according to any one of claims 1 to 7 is implemented.

10. A non-volatile readable storage medium, characterized in that: The non-volatile readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for information prediction according to any one of claims 1 to 7 is implemented.

Images