A method, device and storage medium for memory enhancement of natural language association features

Enhanced memory of natural language correlation features through multi-layer linear computing units and loss functions, solving the problems of high computational cost and low prediction efficiency of Transformer model in long word sequence processing, and achieving more efficient natural language correlation feature representation and reality Sexual prediction.

CN117313716BActive Publication Date: 2024-11-22ROCK AI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311240469.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-09-22
Publication Date
2024-11-22
Estimated Expiration
2043-09-22

AI Technical Summary

Technical Problem

When processing long word sequences, the Transformer model has high computational cost, poor location information capture, noise sensitive, resource consumption, huge storage requirements, difficult and low efficiency, resulting in the computational complexity of the representation of natural language correlation feature and Inefficient prediction.

Method used

A multi-layer linear computing unit is used to generate a natural language correlation feature matrix through dislocation superposition prediction, to build an artificial intelligence content generation model, use loss function to enhance memory, and generate natural language predictions based on objective facts.

Benefits of technology

It significantly shortens the model training cycle, improves the computing performance, makes the predicted natural language conform to objective facts, and improves the scalability and authenticity of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117313716B_ABST
    Figure CN117313716B_ABST
Patent Text Reader

Abstract

The present invention provides a method for enhancing the memory of natural language association features, including: S1, obtaining a word sequence of a natural language, generating an association feature matrix D representing the association features of the natural language; S2, constructing an artificial intelligence content generation model, wherein the artificial intelligence content generation model includes a multi-layer linear calculation unit; S3, inputting the word vector of the first word corresponding to the association feature matrix D representing the association features of the natural language into the artificial intelligence content generation model; starting from the word vector of the first word through the multi-layer linear calculation unit, sequentially performing dislocation superposition prediction, generating a predicted word vector for each word, and training the artificial intelligence content generation model; S4, evaluating each predicted word vector through a loss function, and enhancing the memory of the association features of the natural language. The present invention enhances the memory of the association features of the natural language based on the natural language prediction of objective facts, and the predicted natural language conforms to the objective facts.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device and storage medium for enhancing the memory of natural language association features. Background Art

[0002] Since there are mutual relationships between the n words in natural language, for example, in a sentence: [tomorrow, the, weather, is, rainy], "weather" has an important correlation with "rainy day", in order to express such a correlation, it is generally necessary to calculate the correlation features in the existing technology. The current commonly used method is based on the attention mechanism (Attention).

[0003] In the prior art, the AI ​​content generation model is mainly based on the Transformer model of the attention mechanism. The existing Transformer model has the following technical problems:

[0004] (1) The computational complexity of the attention mechanism increases with the length of the word sequence, because each position needs to calculate the attention weight with all other positions. This leads to a significant increase in the computational cost of the model when processing long word sequences, limiting its scalability in practical applications.

[0005] (2) The attention mechanism itself does not directly consider the position information of words, so it may perform poorly when processing disordered input (such as set data). To solve this problem, the Transformer neural network (a deep learning neural network model based on the self-attention mechanism) introduced position encoding to inject position information, but this method does not always capture the accurate relationship between all positions.

[0006] (3) The attention mechanism may become sensitive when there is a certain degree of noise in the input, resulting in unstable output. Some methods (such as dropout or layer normalization) can alleviate this problem, but they cannot completely solve it.

[0007] (4) Transformer models usually require a lot of computing resources for training and inference, which increases the cost of implementing large models.

[0008] (5) Large Transformer models have tens or even hundreds of billions of parameters, which results in huge storage requirements for the models and takes longer to train.

[0009] (6) The Transformer model requires a large amount of data to achieve good performance during training, which is a challenge for certain specific tasks or low-resource languages.

[0010] (7) The Transformer model has difficulty processing longer text sequences because its self-attention mechanism causes a significant increase in computational complexity.

[0011] (8) The self-attention mechanism of the Transformer model may be less efficient when processing sparse input.

[0012] The computational complexity of representing natural language related features based on the attention mechanism has seriously affected the training efficiency and prediction efficiency of large models of the attention mechanism. Although there have been a lot of explorations in the industry, it has been impossible to improve performance within the scope of linear calculations.

[0013] The Transformer model can learn natural language expressions well, but in addition to the above defects, in actual application, there is also the phenomenon of "hallucination", that is, the content output by the model lacks authenticity or is even "fabricated". However, for some scenarios (such as intelligent customer service, knowledge question answering, etc.), the authenticity of the content output of the large model is required. Summary of the invention

[0014] The present invention provides a method, device and storage medium for memory enhancement of natural language association features to solve the technical problems existing in the Transformer model in the prior art.

[0015] One aspect of the present invention is to provide a method for enhancing the memory of natural language association features, the method comprising the following steps:

[0016] S1, obtaining a natural language word sequence and generating a correlation feature matrix D representing the correlation features of the natural language;

[0017] S2. Constructing an artificial intelligence content generation model, wherein the artificial intelligence content generation model includes multiple layers of linear computing units;

[0018] S3, inputting the word vector of the first word corresponding to the association feature matrix D representing the association features of natural language into the artificial intelligence content generation model;

[0019] Starting from the word vector of the first word, multi-layer linear computing units are used to perform offset superposition predictions in sequence to generate predicted word vectors for each word, and train the artificial intelligence content generation model.

[0020] S4. Each predicted word vector is evaluated through the loss function to enhance the memory of natural language related features.

[0021] In a preferred embodiment, in step S1, generating a correlation feature matrix D representing natural language correlation features includes the following method steps:

[0022] S11, obtaining a word sequence of a natural language, and generating a first feature matrix A of the natural language with n rows and m columns, wherein n is the number of words in the natural language, and m is the number of encoding bits of each word;

[0023] S12, cumulatively summing each column of the first characteristic matrix A with n rows and m columns to generate a second characteristic matrix B with n rows and m columns;

[0024] S13, performing a linear transformation of a neural network on the second characteristic matrix B having n rows and m columns to obtain a third characteristic matrix C having n rows and m columns;

[0025] S14. Normalize the third feature matrix C with n rows and m columns to obtain a correlation feature matrix D with n rows and m columns; the correlation feature matrix D represents natural language correlation features.

[0026] In a preferred embodiment, in step S11, a unique and non-repetitive ID number is assigned to each word of the natural language to obtain a word sequence of the natural language;

[0027] For each word ID number, an m-bit code is randomly generated to generate the first feature matrix A of the natural language with n rows and m columns.

[0028] In a preferred embodiment, it is characterized in that, in step S12, for each element value of each column of the first characteristic matrix A, the previous element value is sequentially accumulated to generate a second characteristic matrix B with n rows and m columns.

[0029] In a preferred embodiment, in step S13, a linear transformation matrix X with m rows and m columns is constructed, and the second characteristic matrix B with n rows and m columns is multiplied by the linear transformation matrix X with m rows and m columns to obtain a third characteristic matrix C with n rows and m columns.

[0030] In a preferred embodiment, in step S14, the value of each element of the association feature matrix D with n rows and m columns ranges from -1 to 1.

[0031] In a preferred embodiment, in step S14, the third characteristic matrix C of n rows and m columns is normalized by using the Min-Max normalization method or the Z-Score normalization method, so that the value of each element of the obtained association characteristic matrix D of n rows and m columns ranges from -1 to 1.

[0032] In a preferred embodiment, in step S1, a natural language word sequence is obtained by taking a character of the natural language as a term, and a correlation feature matrix D representing the correlation features of the natural language is generated.

[0033] Another aspect of the present invention is to provide a memory enhancement device for natural language association features, the memory enhancement device comprising:

[0034] A correlation feature matrix D generating unit, used to obtain a word sequence of a natural language and generate a correlation feature matrix D representing correlation features of the natural language;

[0035] An artificial intelligence content generation model construction unit, used to construct an artificial intelligence content generation model, wherein the artificial intelligence content generation model includes multiple layers of linear calculation units;

[0036] A model training unit, for inputting the word vector of the first word corresponding to the association feature matrix D representing the association features of the natural language into the artificial intelligence content generation model;

[0037] And through multi-layer linear computing units, starting from the word vector of the first word, dislocation superposition prediction is performed in sequence to generate the predicted word vector of each word, and the artificial intelligence content generation model is trained;

[0038] The evaluation unit is used to evaluate each predicted word vector through a loss function to enhance the memory of natural language related features.

[0039] Another aspect of the present invention is to provide a storage medium, wherein the storage medium is used to store computer execution instructions, and the computer execution instructions are used to execute a memory enhancement method for natural language association features provided by the present invention.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention provides a method, device and storage medium for memory enhancement of natural language association features, which use a natural language association feature representation method of linear calculation, thereby shortening the training cycle of an artificial intelligence content generation model and greatly improving computing performance.

[0042] The present invention provides a method, device and storage medium for enhancing the memory of natural language associated features. Natural language prediction is no longer based on the transition probability of natural language, but is based on objective facts. The memory of natural language associated features is enhanced, and the predicted natural language conforms to objective facts. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 It is a flow chart of a method for memory enhancement of natural language association features of the present invention.

[0045] Figure 2 It is a flow chart of the present invention for generating a correlation feature matrix representing natural language correlation features.

[0046] Figure 3 It is a structural block diagram of the artificial intelligence content generation model of the present invention.

[0047] Figure 4 It is a structural block diagram of a memory enhancement device for natural language association features of the present invention. DETAILED DESCRIPTION

[0048] In order to make the above and other features and advantages of the present invention more clear, the present invention is further described below in conjunction with the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explaining to those skilled in the art and are only exemplary and not restrictive.

[0049] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0050] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0051] like Figure 1As shown, according to an embodiment of the present invention, a memory enhancement method for natural language association features is provided, including the following method steps:

[0052] Step S1: Obtain the word sequence of the natural language and generate an association feature matrix D representing the natural language association features.

[0053] Specifically, as Figure 2 shown, through a representation method of natural language association features based on linear calculation, an association feature matrix D representing the natural language association features is generated, including the following method steps:

[0054] Step S11: Obtain the word sequence of the natural language and generate a first feature matrix A of n rows and m columns of the natural language, where n is the number of words in the natural language and m is the number of encoding bits for each word.

[0055] Specifically, assign a unique and non-repeating ID number to each word of the natural language and obtain the word sequence of the natural language.

[0056] For the ID number of each word, randomly generate m-bit encoding to generate a first feature matrix A of n rows and m columns of the natural language.

[0057] For example, for a natural language ["tomorrow", "the", "weather", "is", "rainy day"], there are n = 5 words in total. Assign a unique and non-repeating ID number to each word. For example, "tomorrow" is assigned ID number 1, "the" is assigned ID number 2, "weather" is assigned ID number 3, "is" is assigned ID number 4, and "rainy day" is assigned ID number 5 to obtain the natural language sequence [1, 2, 3, 4, 5].

[0058] For the ID number of each word, randomly generate m-bit encoding. For example, for the ID number of each word, randomly generate m = 4-bit encoding. Then, the word "tomorrow" with ID number 1 generates a 4-bit encoded word vector (Embedding): [0.1, 0.2, 0.3, 0.4];

[0059] The word "the" with ID number 2 generates a 4-bit encoded word vector (Embedding): [0.2, 0.5, 0.1, 0.3];

[0060] The word "weather" with ID number 3 generates a 4-bit encoded word vector (Embedding): [0.5, 0.9, 0.2, 0.3];

[0061] The word "is" with ID number 4 generates a 4-bit encoded word vector (Embedding): [0.3, 0.2, 0.1, 0.7];

[0062] The word vector (Embedding) with an ID number of 5 for "rainy day" generates a 4-bit encoded vector: [0.1, 0.4, 0.2, 0.9].

[0063] According to the natural language sequence [1, 2, 3, 4, 5] of n words and the word vector (Embedding) with m = 4-bit encoding, generate the first feature matrix A of n rows and m columns for natural language. For example, for a natural language [tomorrow, the, weather, is, rainy day], generate a first feature matrix A of 5 rows and 4 columns:

[0064] [[0.1, 0.2, 0.3, 0.4]

[0065] [0.2, 0.5, 0.1, 0.3]

[0066] [0.5, 0.9, 0.2, 0.3]

[0067] [0.3, 0.2, 0.1, 0.7]

[0068] [0.1, 0.4, 0.2, 0.9]].

[0069] It should be noted that the encoding bit number m of the word vector (Embedding) generated for each word is randomly initialized, and the element values of the word vector (Embedding) generated for each word are randomly initialized. The element values of the word vector (Embedding) generated for each word by random initialization will be continuously adjusted during the backpropagation learning process of the neural network to make the first feature matrix A reach the optimal state.

[0070] In some preferred embodiments, take one character of the natural language as one word to obtain the word sequence of the natural language, and generate the association feature matrix D representing the association features of the natural language. For example, for a natural language [ming, tian, de, tian, qi, shi, xia, yu, tian], taking one character as one word to obtain, there are a total of n = 9 words, and obtain the natural language sequence [1, 2, 3, 4, 5, 6, 7, 8, 9]

[0071] Step S12: Perform cumulative summation on each column of the first feature matrix A with n rows and m columns to generate the second feature matrix B with n rows and m columns.

[0072] Specifically, for each element value in each column of the first feature matrix A, successively accumulate the previous element value to generate the second feature matrix B with n rows and m columns.

[0073] Taking the first column [0.1, 0.2, 0.5, 0.3, 0.1] of the 5-row and 4-column first feature matrix A generated by a natural language [tomorrow, the, weather, is, rainy day] as an example,

[0074] The value of the first element 0.1 remains unchanged, and the cumulative sum is itself, that is, 0.1;

[0075] The second element value is 0.2, which is added to the previous element value, that is, 0.2+0.1=0.3;

[0076] The third element value is 0.5, which is added to the previous element value, that is, 0.5+0.3=0.8;

[0077] The fourth element value is 0.3, which is added to the previous element value, that is, 0.3+0.8=1.1;

[0078] The fifth element value is 0.1, which is the sum of the previous element value, that is, 0.1+1.1=1.2.

[0079] By analogy, for each element value of each column of the first characteristic matrix A, the previous element value is accumulated in sequence to obtain the second characteristic matrix B with 5 rows and 4 columns:

[0080] [[0.1000,0.2000,0.3000,0.4000]

[0081] [0.3000,0.7000,0.4000,0.7000]

[0082] [0.8000,1.6000,0.6000,1.0000]

[0083] [1.1000,1.8000,0.7000,1.7000]

[0084] [1.2000,2.2000,0.9000,2.6000]].

[0085] Step S13: Perform a linear transformation of a neural network on the second characteristic matrix B having n rows and m columns to obtain a third characteristic matrix C having n rows and m columns.

[0086] Specifically, a linear transformation matrix X with m rows and m columns is constructed, and the second characteristic matrix B with n rows and m columns is multiplied by the linear transformation matrix X with m rows and m columns to obtain a third characteristic matrix C with n rows and m columns.

[0087] For example, for a natural language [tomorrow, the, weather, is, rainy], according to the second feature matrix B with 5 rows and 4 columns generated in step S12, a linear transformation matrix X with 4 rows and 4 columns is constructed:

[0088] [[0.7260,0.3795,0.6275,0.8122]

[0089] [0.9521,0.5301,0.5481,0.2933]

[0090] [0.1535,0.7780,0.6424,0.1868]

[0091] [0.3251,0.4054,0.7136,0.0900]].

[0092] Each element value of the linear transformation matrix X is randomly initialized.

[0093] The second characteristic matrix B of 5 rows and 4 columns is subjected to a linear transformation of a neural network. Specifically, the second characteristic matrix B is multiplied by a linear transformation matrix X of 4 rows and 4 columns to obtain a third characteristic matrix C of 5 rows and 4 columns, that is, B×X=C.

[0094] In this embodiment, the obtained 5-row 4-column third characteristic matrix C is:

[0095] [[0.4391,0.5395,0.6505,0.2319]

[0096] [1.1732,1.0799,1.3284,0.5867]

[0097] [2.5214,2.0240,2.4780,1.3211]

[0098] [3.1725,2.6054,3.3396,1.7051]

[0099] [3.9492,3.3759,4.3923,2.0220]].

[0100] Step S14: normalize the third characteristic matrix C with n rows and m columns to obtain a correlation characteristic matrix D with n rows and m columns.

[0101] The association feature matrix D with n rows and m columns represents the natural language association features, and the value of each element of the association feature matrix D with n rows and m columns ranges from -1 to 1.

[0102] Furthermore, the Min-Max (deviation standardization) normalization method or the Z-Score (standard score) normalization method is used to normalize the third characteristic matrix C of n rows and m columns, so that the value of each element of the obtained association characteristic matrix D of n rows and m columns ranges from -1 to 1.

[0103] For example, for a natural language [tomorrow, the, weather, is, rainy], the third feature matrix C with 5 rows and 4 columns is normalized using the Min-Max normalization method to obtain a correlation feature matrix D with 5 rows and 4 columns.

[0104] The 5-row and 4-column correlation feature matrix D represents the correlation features of a piece of natural language ["Tomorrow", "the", "weather", "is", "rainy day"].

[0105] Step S2: Construct an artificial intelligence content generation model, where the artificial intelligence content generation model includes multiple layers of linear calculation units.

[0106] As Figure 3 shown, the artificial intelligence content generation model (AIGC model) constructed by the present invention includes multiple layers of linear calculation units. Specifically, the artificial intelligence content generation model includes a first linear calculation unit 201 in the first layer, a second linear unit 202 in the second layer, a third linear unit 203 in the third layer, a fourth linear unit 204 in the fourth layer, a fifth linear unit 205 in the fifth layer,..., and an Nth linear unit 20N in the Nth layer.

[0107] Step S3: Input the word vector corresponding to the first word of the correlation feature matrix D representing the natural language correlation features into the artificial intelligence content generation model.

[0108] Starting from the word vector of the first word, perform staggered superposition prediction in sequence through multiple layers of linear calculation units to generate the predicted word vectors of each word, and train the artificial intelligence content generation model.

[0109] Combined Figure 3 , taking a piece of natural language ["Tomorrow", "the", "weather", "is", "rainy day"] as an example, in step S1, a 5-row and 4-column correlation feature matrix D representing the natural language correlation features is obtained.

[0110] The word vectors corresponding to each word of the 5-row and 4-column correlation feature matrix D representing the natural language correlation features are:

[0111] "Tomorrow": The first row of the correlation feature matrix D;

[0112] "the": The second row of the correlation feature matrix D;

[0113] "weather": The third row of the correlation feature matrix D;

[0114] "is": The fourth row of the correlation feature matrix D;

[0115] "rainy day": The fifth row of the correlation feature matrix D.

[0116] Denote the word vector corresponding to the first word "Tomorrow" of the 5-row and 4-column correlation feature matrix D representing the natural language correlation features as x 1 , and input it into the artificial intelligence content generation model.

[0117] Starting from the word vector of the first word in the correlation feature matrix D, through multiple-layer linear calculation units, perform staggered superposition predictions in sequence to generate the predicted word vectors of each word after the first word in the correlation feature matrix D, and train the artificial intelligence content generation model.

[0118] Table 1: Table for generating predicted word vectors

[0119] Natural Language (Word Vectors) Expected y value (predicted word vector) <![CDATA[Tomorrow (x 1 )]]> <![CDATA[(y 2 )]]> <![CDATA[(x of tomorrow 2 = x 1 × y 2 + x 1 )]]> <![CDATA[Weather (y 3 )]]> <![CDATA[Tomorrow's weather (x 3 = x 2 × y 3 + x 2 )]]> <![CDATA[Yes (y 4 )]]> <![CDATA[The weather tomorrow is (x 4 = x 3 × y 4 + x 3 )]]> <![CDATA[Rainy day (y 5 )]]> <![CDATA[The weather tomorrow is rainy day (x 5 = x 4 × y 5 + x 4 )]]> <![CDATA[ <eos> (y 6 )]]> < / eos>

[0120] As shown in Table 1 for generating the table of predicted word vectors, the word vector of the first word "tomorrow" is denoted as x 1 , and the predicted word vector of the word "of" is denoted as y 2 .

[0121] Similarly, the word vector of the word "tomorrow of" is denoted as x 2 , and the predicted word vector of the word "weather" is denoted as y 3 .

[0122] Similarly, the word vector of the word "tomorrow's weather" is denoted as x 3 , and the predicted word vector of the word "is" is denoted as y 4 .

[0123] Similarly, the word vector of the word "tomorrow's weather is" is denoted as x 4 , and the predicted word vector of the word "rainy day" is denoted as y 5 .

[0124] The word vector of the natural language "tomorrow's weather is rainy day" is denoted as x 5 , and the added sentence end symbol <eos>, the predicted word vector of the natural language "The weather tomorrow is rainy" is denoted as y 6 .

[0125] Combined with Figure 3 , in a specific embodiment, the 5-row and 4-column association feature matrix D representing the natural language association features corresponds to the word vector x of the first word "tomorrow" 1 , and is input into the first linear calculation unit 201 of the artificial intelligence content generation model. Through the first linear calculation unit 201, the predicted word vector y of the next word "of" is output 2 .

[0126] For the word vector x of the word "tomorrow" 1 The predicted word vector of the word "of" is superimposed as y 2 , and the word vector x of the word "tomorrow of" is generated 2 = x 1 × y 2 + x 1 .

[0127] The word vector x of the word "tomorrow of" 2 , is input into the second linear calculation unit 202 of the artificial intelligence content generation model. Through the second linear calculation unit 202, the predicted word vector y of the next word "weather" is output 3 .

[0128] For the word vector x of the word "tomorrow of" 2 The predicted word vector of the word "weather" is superimposed as y 3 , and the word vector x of the word "tomorrow's weather" is generated 3 = x 2 × y 3 + x 2 .

[0129] The word vector x of the word "tomorrow's weather" 3 , is input into the third linear calculation unit 203 of the artificial intelligence content generation model. Through the third linear calculation unit 203, the predicted word vector y of the next word "is" is output 4 .

[0130] For the word vector x of the word "tomorrow's weather" 3 The predicted word vector of the word "is" is superimposed as y 4 , and the word vector x of the word "tomorrow's weather is" is generated 4 = x 3 × y 4 + x 3 .

[0131] The word vector x of the word "tomorrow's weather is" 4 , input the fourth linear calculation unit 204 of the artificial intelligence content generation model. After passing through the fourth linear calculation unit 204, output the predicted word vector y of the next word "rainy day" 5 .

[0132] For the word vector x of the word "The weather tomorrow is" 4 Overlay the predicted word vector of the word "rainy day" as y 5 , generate the word vector x of the natural language "The weather tomorrow is rainy day" 5 = x 4 × y 5 + x 4 .

[0133] Input the word vector x of the natural language "The weather tomorrow is rainy day" 5 , into the fifth linear calculation unit 205 of the artificial intelligence content generation model. After passing through the fifth linear calculation unit 205, output the predicted word vector y of the natural language "The weather tomorrow is rainy day" 6 .

[0134] In this embodiment, a piece of natural language [tomorrow, of, weather, is, rainy day] represents an association feature matrix D of 5 rows and 4 columns for natural language association features, with a total of 5 words: "tomorrow", "of", "weather", "is", "rainy day".

[0135] In some preferred embodiments, a word sequence of the natural language is obtained with one character of the natural language as one word, and an association feature matrix D representing the natural language association features is generated. For example, for a piece of natural language [ming, tian, de, tian, qi, shi, xia, yu, tian], a word sequence of the natural language is obtained with one character as one word, and an association feature matrix D representing the natural language association features is generated, with a total of n = 9 words: "ming", "tian", "de", "tian", "qi", "shi", "xia", "yu", "tian".

[0136] As Figure 3 shown, when n = 9, starting from the word vector x of the first word of the association feature matrix D 1 , perform staggered overlay prediction in sequence according to x n = x n-1 × y n + x n-1 to generate the predicted word vector of each word after one word of the association feature matrix D: y n+1 , where n = 1, 2,..., 9, x 1 = x 0 , y 1 = 0.

[0137] The word vector of the first word "tomorrow" corresponding to the associated feature matrix D is input into the artificial intelligence content generation model. Starting from the word vector of the first word of the associated feature matrix D, sequential staggered superposition prediction is performed through a multi-layer linear calculation unit to generate the predicted word vectors of each word after one word of the associated feature matrix D, and the artificial intelligence content generation model is trained.

[0138] Step S4: Evaluate each predicted word vector through a loss function to enhance the memory of natural language association features.

[0139] In step S3, the predicted word vectors of each generated word are evaluated through a loss function to enhance the memory of natural language association features, enabling the artificial intelligence content generation model to enhance the output of "of", "weather", "is", "rainy day", " <eos>” memory.

[0140] In one embodiment, the loss function adopts a cross entropy loss function. Further, a CrossEntropyLoss cross entropy loss function or an nllloss cross entropy loss function is adopted.

[0141] The present invention no longer predicts natural language based on the transition probability of natural language, but predicts natural language based on objective facts. By constructing an artificial intelligence content generation model, the word vectors of the association feature matrix representing the association features of natural language based on linear calculation are input into the artificial intelligence content generation model for sequential staggered superposition prediction. The artificial intelligence content generation model is trained, the memory of natural language association features is enhanced, and the predicted natural language conforms to objective facts.

[0142] like Figure 4 As shown, according to an embodiment of the present invention, a memory enhancement device for natural language association features is provided, comprising:

[0143] The correlation feature matrix D generating unit 100 is used to obtain a word sequence of a natural language and generate a correlation feature matrix D representing correlation features of the natural language.

[0144] The artificial intelligence content generation model construction unit 200 is used to construct an artificial intelligence content generation model, wherein the artificial intelligence content generation model includes multiple layers of linear calculation units.

[0145] The model training unit 300 is used to input the word vector of the first word corresponding to the association feature matrix D representing the association features of the natural language into the artificial intelligence content generation model;

[0146] Starting from the word vector of the first word, multi-layer linear calculation units are used to perform staggered superposition predictions in sequence to generate predicted word vectors for each word and train the artificial intelligence content generation model.

[0147] The evaluation unit 400 is used to evaluate each predicted word vector through a loss function to enhance the memory of natural language related features.

[0148] According to an embodiment of the present invention, a storage medium is provided for storing computer execution instructions, and the computer execution instructions are used to execute a memory enhancement method for natural language association features provided by the present invention.

[0149] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.< / eos> < / eos>

Claims

1. A method for enhancing memory of natural language association features, characterized in that: The memory enhancement method comprises the following steps: S1, obtaining a natural language word sequence and generating a correlation feature matrix D representing the correlation features of the natural language; S2. constructing an artificial intelligence content generation model, wherein the artificial intelligence content generation model includes multiple layers of linear computing units; S3, inputting the word vector of the first word corresponding to the association feature matrix D representing the association features of natural language into the artificial intelligence content generation model; Starting from the word vector of the first word, multi-layer linear computing units are used to perform offset superposition predictions in sequence to generate predicted word vectors for each word, and train the artificial intelligence content generation model. S4. Each predicted word vector is evaluated through the loss function to enhance the memory of natural language related features.

2. The memory enhancement method according to claim 1, characterized in that: In step S1, generating a correlation feature matrix D representing natural language correlation features includes the following method steps: S11, obtaining a word sequence of a natural language, and generating a first feature matrix A of the natural language with n rows and m columns, wherein n is the number of words in the natural language, and m is the number of encoding bits of each word; S12, cumulatively summing each column of the first characteristic matrix A with n rows and m columns to generate a second characteristic matrix B with n rows and m columns; S13, performing a linear transformation of a neural network on the second characteristic matrix B having n rows and m columns to obtain a third characteristic matrix C having n rows and m columns; S14. Normalize the third feature matrix C with n rows and m columns to obtain a correlation feature matrix D with n rows and m columns; the correlation feature matrix D represents natural language correlation features.

3. The memory enhancement method according to claim 2, characterized in that: In step S11, a unique and non-repetitive ID number is assigned to each word of the natural language to obtain a word sequence of the natural language; For each word ID number, an m-bit code is randomly generated to generate the first feature matrix A of the natural language with n rows and m columns.

4. The memory enhancement method according to claim 2, characterized in that: In step S12, for each element value of each column of the first characteristic matrix A, the previous element value is sequentially accumulated to generate a second characteristic matrix B with n rows and m columns.

5. The memory enhancement method according to claim 2, characterized in that: In step S13, a linear transformation matrix X with m rows and m columns is constructed, and the second characteristic matrix B with n rows and m columns is multiplied by the linear transformation matrix X with m rows and m columns to obtain a third characteristic matrix C with n rows and m columns.

6. The memory enhancement method according to claim 2, characterized in that: In step S14, the value of each element of the association feature matrix D with n rows and m columns ranges from -1 to 1.

7. The memory enhancement method according to claim 2, characterized in that: In step S14, the third characteristic matrix C of n rows and m columns is normalized by using the Min-Max normalization method or the Z-Score normalization method, so that the value of each element of the obtained association characteristic matrix D of n rows and m columns ranges from -1 to 1.

8. The memory enhancement method according to claim 1, characterized in that: In step S1, a natural language word sequence is obtained by taking a character of the natural language as a term, and a correlation feature matrix D representing the correlation features of the natural language is generated.

9. A memory enhancement device for natural language association features, characterized in that: The memory enhancement device comprises: A correlation feature matrix D generating unit, used to obtain a word sequence of a natural language and generate a correlation feature matrix D representing correlation features of the natural language; An artificial intelligence content generation model construction unit, used to construct an artificial intelligence content generation model, wherein the artificial intelligence content generation model includes multiple layers of linear calculation units; A model training unit, for inputting the word vector of the first word corresponding to the association feature matrix D representing the association features of the natural language into the artificial intelligence content generation model; And through multi-layer linear computing units, starting from the word vector of the first word, dislocation superposition prediction is performed in sequence to generate the predicted word vector of each word, and the artificial intelligence content generation model is trained; The evaluation unit is used to evaluate each predicted word vector through a loss function to enhance the memory of natural language related features.

10. A storage medium, characterized in that: The storage medium is used to store computer-executable instructions, and the computer-executable instructions are used to execute the memory enhancement method described in any one of claims 1 to 8.