Memory enhancement method and apparatus for natural language association features and storage medium

Abstract

Provided in the present invention is a memory enhancement method for natural language association features, comprising: S1, acquiring word sequences of natural language, and generating an association feature matrix D representing natural language association features; S2, constructing an artificial intelligence content generation model, the artificial intelligence content generation model comprising multiple layers of linear computing unit; S3, inputting into the artificial intelligence content generation model a word vector of a first word corresponding to said association feature matrix D, and starting from the word vector of the first word, sequentially carrying out misaligned superposition predictions by means of the multiple layers of linear computing units, so as to generate a predicted word vector for each word, so as to train the artificial intelligence content generation model; and S4, by means of a loss function, evaluating each predicted word vector, so as to enhance memories of the natural language association features. On the basis of objective fact-based natural language predictions, the present invention enhances memories of natural language association features, so that the predicted natural language conforms to objective facts.

Description

A method, device and storage medium for memory enhancement of natural language association features Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device and storage medium for enhancing memory of natural language association features. Background Art

[0002] Since there are mutual relationships between n words in natural language, for example, in a sentence: [Tomorrow, the weather is, rainy], "weather" has an important correlation with "rainy day", in order to represent such a correlation, the existing technology generally needs to calculate the correlation features. The current commonly used method is based on the attention mechanism.

[0003] In existing technologies, AI content generation models are mainly based on the Transformer model with attention mechanism. However, the existing Transformer model has the following technical problems:

[0004] (1) The computational complexity of the attention mechanism increases with the length of the word sequence, because each position needs to calculate the attention weight with all other positions. This leads to a significant increase in the computational cost of the model when processing long word sequences, limiting its scalability in practical applications.

[0005] (2) The attention mechanism itself does not directly consider the position information of words, so it may perform poorly when processing disordered input (such as aggregate data). To solve this problem, the Transformer neural network (a deep learning neural network model based on the self-attention mechanism) introduced position encoding to inject position information, but this method does not always capture the accurate relationship between all positions.

[0006] (3) The attention mechanism may become sensitive to a certain degree of noise in the input, resulting in unstable output. Some methods (such as dropout or layer normalization) can alleviate this problem, but they cannot completely solve it.

[0007] (4) Transformer models usually require a lot of computing resources for training and inference, which increases the cost of implementing large models.

[0008] (5) Large Transformer models have tens of billions or even hundreds of billions of parameters, which results in huge storage requirements for the model and takes longer to train.

[0009] (6) The Transformer model requires a large amount of data to achieve good performance during training, which is a challenge for certain specific tasks or low-resource languages.

[0010] (7) The Transformer model has difficulty processing longer text sequences because its self-attention mechanism (Attention) will lead to a significant increase in computational complexity.

[0011] (8) The self-attention mechanism of the Transformer model may be less efficient when processing sparse input.

[0012] The computational complexity of representing natural language association features based on the attention mechanism has seriously affected the training and prediction efficiency of large models of the attention mechanism. Although there have been a lot of explorations in the industry, it has not been possible to improve performance within the scope of linear calculation.

[0013] The Transformer model is well-suited for learning natural language expressions. However, in addition to the aforementioned drawbacks, it also suffers from "hallucination" in practical applications, where the model output lacks authenticity or is even fabricated. However, some scenarios (such as intelligent customer service and knowledge Q&A) require authentic content output from large models.

[0014] Summary of the Invention

[0015] The present invention provides a method, device and storage medium for enhancing the memory of natural language association features to solve the technical problems existing in the Transformer model in the prior art.

[0016] One aspect of the present invention is to provide a method for enhancing memory of natural language association features, the method comprising the following steps:

[0017] S1. Obtain a natural language word sequence and generate a correlation feature matrix D representing the correlation features of the natural language;

[0018] S2. Constructing an artificial intelligence content generation model, wherein the artificial intelligence content generation model includes multiple layers of linear computing units;

[0019] S3. Input the word vector of the first word corresponding to the correlation feature matrix D representing the correlation features of natural language into the artificial intelligence content generation model;

[0020] Starting from the word vector of the first word, multi-layer linear calculation units are used to perform offset superposition predictions in sequence to generate predicted word vectors for each word, thereby training the AI ​​content generation model.

[0021] S4. Each predicted word vector is evaluated through the loss function to enhance the memory of natural language association features.

[0022] In a preferred embodiment, in step S1, generating a correlation feature matrix D representing natural language correlation features includes the following method steps:

[0023] S11. Obtain a word sequence of a natural language and generate a first feature matrix A of the natural language with n rows and m columns, where n is the number of words in the natural language and m is the number of encoding bits for each word;

[0024] S12, cumulatively summing each column of the first characteristic matrix A with n rows and m columns to generate a second characteristic matrix B with n rows and m columns;

[0025] S13, performing a neural network linear transformation on the second characteristic matrix B having n rows and m columns to obtain a third characteristic matrix C having n rows and m columns;

[0026] S14. Normalize the third feature matrix C with n rows and m columns to obtain a correlation feature matrix D with n rows and m columns; the correlation feature matrix D represents natural language correlation features.

[0027] In a preferred embodiment, in step S11, a unique and non-repeating ID number is assigned to each word in the natural language to obtain a word sequence in the natural language;

[0028] For each word ID number, an m-bit code is randomly generated to generate the first feature matrix A of the natural language with n rows and m columns.

[0029] In a preferred embodiment, it is characterized in that, in step S12, for each element value of each column of the first characteristic matrix A, the previous element value is sequentially accumulated to generate a second characteristic matrix B with n rows and m columns.

[0030] In a preferred embodiment, in step S13, a linear transformation matrix X with m rows and m columns is constructed, and the second characteristic matrix B with n rows and m columns is multiplied by the linear transformation matrix X with m rows and m columns to obtain a third characteristic matrix C with n rows and m columns.

[0031] In a preferred embodiment, in step S14, the value of each element of the correlation feature matrix D with n rows and m columns ranges from -1 to 1.

[0032] In a preferred embodiment, in step S14, the Min-Max normalization method or the Z-Score normalization method is used to normalize the third characteristic matrix C with n rows and m columns, so that the value of each element of the obtained association characteristic matrix D with n rows and m columns ranges from -1 to 1.

[0033] In a preferred embodiment, in step S1, a natural language word sequence is obtained by taking a character of the natural language as a word, and a correlation feature matrix D representing the correlation features of the natural language is generated.

[0034] Another aspect of the present invention is to provide a memory enhancement device for natural language association features, the memory enhancement device comprising:

[0035] A correlation feature matrix D generating unit, configured to obtain a natural language word sequence and generate a correlation feature matrix D representing correlation features of the natural language;

[0036] An artificial intelligence content generation model construction unit, configured to construct an artificial intelligence content generation model, wherein the artificial intelligence content generation model includes multiple layers of linear calculation units;

[0037] A model training unit is used to input the word vector of the first word corresponding to the correlation feature matrix D representing the correlation features of natural language into the artificial intelligence content generation model;

[0038] Then, starting from the word vector of the first word, multi-layer linear calculation units are used to perform offset superposition predictions in sequence to generate the predicted word vector for each word, thus training the AI ​​content generation model.

[0039] The evaluation unit is used to evaluate each predicted word vector through the loss function to enhance the memory of natural language related features.

[0040] Another aspect of the present invention is to provide a storage medium, wherein the storage medium is used to store computer-executable instructions, and the computer-executable instructions are used to execute a memory enhancement method for natural language association features provided by the present invention.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention provides a method, device, and storage medium for enhancing the memory of natural language association features, which uses a natural language association feature representation method based on linear calculation, thereby shortening the training cycle of an artificial intelligence content generation model and significantly improving computing performance.

[0043] The present invention provides a method, device and storage medium for enhancing the memory of natural language association features. Natural language prediction is no longer based on the transition probability of natural language, but is based on objective facts. This enhances the memory of natural language association features, and the predicted natural language conforms to objective facts. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] FIG1 is a flow chart of a method for enhancing memory of natural language association features according to the present invention.

[0046] FIG2 is a flow chart of the present invention for generating a correlation feature matrix representing natural language correlation features.

[0047] FIG3 is a structural block diagram of the artificial intelligence content generation model of the present invention.

[0048] FIG4 is a structural block diagram of a memory enhancement device for natural language association features according to the present invention. DETAILED DESCRIPTION

[0049] In order to make the above and other features and advantages of the present invention more clear, the present invention is further described below with reference to the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explaining to those skilled in the art and are only exemplary and not restrictive.

[0050] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0051] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0052] As shown in FIG1 , according to an embodiment of the present invention, a method for enhancing memory of natural language association features is provided, comprising the following steps:

[0053] Step S1: Obtain the word sequence of natural language and generate an association feature matrix D representing the association features of natural language.

[0054] Specifically, as shown in Figure 2, through a representation method of natural language association features based on linear calculation, generate an association feature matrix D representing the association features of natural language, including the following method steps:

[0055] Step S11: Obtain the word sequence of natural language and generate a first feature matrix A with n rows and m columns of natural language, where n is the number of words in natural language and m is the number of encoding bits for each word.

[0056] Specifically, assign a unique and non-repeating ID number to each word of natural language and obtain the word sequence of natural language.

[0057] For the ID number of each word, randomly generate an m-bit encoding to generate a first feature matrix A with n rows and m columns of natural language.

[0058] For example, for a natural language [Tomorrow, 's, weather, is, rainy day], there are n = 5 words in total. Assign a unique and non-repeating ID number to each word. For example, assign ID number 1 to "Tomorrow", ID number 2 to "'s", ID number 3 to "weather", ID number 4 to "is", and ID number 5 to "rainy day", and obtain the natural language sequence [1, 2, 3, 4, 5].

[0059] For the ID number of each word, randomly generate an m-bit encoding. For example, for the ID number of each word, randomly generate an m = 4-bit encoding. Then, the word "Tomorrow" with ID number 1 generates a 4-bit encoded word vector (Embedding): [0.1, 0.2, 0.3, 0.4];

[0060] The word "'s" with ID number 2 generates a 4-bit encoded word vector (Embedding): [0.2, 0.5, 0.1, 0.3];

[0061] The word "weather" with ID number 3 generates a 4-bit encoded word vector (Embedding): [0.5, 0.9, 0.2, 0.3];

[0062] The word "is" with ID number 4 generates a 4-bit encoded word vector (Embedding): [0.3, 0.2, 0.1, 0.7];

[0063] The word "rainy day" with ID number 5 generates a 4-bit encoded word vector (Embedding): [0.1, 0.4, 0.2, 0.9].

[0064] Generate the first feature matrix A with n rows and m columns for the natural language based on the natural language sequence [1, 2, 3, 4, 5] of n words and the word vector (Embedding) encoded with m = 4 bits. For example, for a natural language ["tomorrow", "is", "weather", "it", "rainy day"], generate the first feature matrix A with 5 rows and 4 columns:

[0065] [[0.1, 0.2, 0.3, 0.4]

[0066] [0.2, 0.5, 0.1, 0.3]

[0067] [0.5, 0.9, 0.2, 0.3]

[0068] [0.3, 0.2, 0.1, 0.7]

[0069] [0.1, 0.4, 0.2, 0.9]]。

[0070] It should be noted that the encoding bit number m of the word vector (Embedding) generated for each word is randomly initialized, and the element values of the word vector (Embedding) generated for each word are randomly initialized. The element values of the word vector (Embedding) generated for each word by random initialization will be continuously adjusted during the backpropagation learning process of the neural network to make the first feature matrix A reach the optimal state.

[0071] In some preferred embodiments, take one character of the natural language as one word to obtain the word sequence of the natural language, and generate the associated feature matrix D representing the associated features of the natural language. For example, for a natural language ["tomorrow", "is", "weather", "it", "rainy day"], take one character as one word to obtain a total of n = 9 words, and obtain the natural language sequence [1, 2, 3, 4, 5, 6, 7, 8, 9]

[0072] Step S12: Perform cumulative summation on each column of the first feature matrix A with n rows and m columns to generate the second feature matrix B with n rows and m columns.

[0073] Specifically, for each element value in each column of the first feature matrix A, successively add the previous element value to generate the second feature matrix B with n rows and m columns.

[0074] Taking the first column [0.1, 0.2, 0.5, 0.3, 0.1] of the first feature matrix A with 5 rows and 4 columns generated by a natural language ["tomorrow", "is", "weather", "it", "rainy day"] as an example,

[0075] The first element value 0.1 remains unchanged, and the cumulative sum is itself, that is, 0.1;

[0076] The second element value is 0.2, which is added to the previous element value, that is, 0.2 + 0.1 = 0.3;

[0077] The third element has a value of 0.5, which is added to the previous element value, i.e. 0.5 + 0.3 = 0.8.

[0078] The fourth element value is 0.3, which is the sum of the previous element value, that is, 0.3 + 0.8 = 1.1;

[0079] The fifth element value is 0.1, which is the sum of the previous element value, that is, 0.1+1.1=1.2.

[0080] By analogy, for each element value in each column of the first characteristic matrix A, the previous element value is accumulated in sequence to obtain the second characteristic matrix B with 5 rows and 4 columns:

[0081] [[0.1000,0.2000,0.3000,0.4000]

[0082] [0.3000,0.7000,0.4000,0.7000]

[0083] [0.8000,1.6000,0.6000,1.0000]

[0084] [1.1000,1.8000,0.7000,1.7000]

[0085] [1.2000,2.2000,0.9000,2.6000]].

[0086] Step S13: Perform a neural network linear transformation on the second characteristic matrix B having n rows and m columns to obtain a third characteristic matrix C having n rows and m columns.

[0087] Specifically, a linear transformation matrix X with m rows and m columns is constructed, and the second characteristic matrix B with n rows and m columns is multiplied by the linear transformation matrix X with m rows and m columns to obtain a third characteristic matrix C with n rows and m columns.

[0088] For example, for a natural language [tomorrow, the, weather, is, rainy], according to the second feature matrix B with 5 rows and 4 columns generated in step S12, a linear transformation matrix X with 4 rows and 4 columns is constructed:

[0089] [[0.7260,0.3795,0.6275,0.8122]

[0090] [0.9521,0.5301,0.5481,0.2933]

[0091] [0.1535,0.7780,0.6424,0.1868]

[0092] [0.3251,0.4054,0.7136,0.0900]].

[0093] Each element value of the linear transformation matrix X is randomly initialized.

[0094] The second characteristic matrix B with 5 rows and 4 columns is subjected to a linear transformation of the neural network. Specifically, the second characteristic matrix B is multiplied by the linear transformation matrix X with 4 rows and 4 columns to obtain a third characteristic matrix C with 5 rows and 4 columns, that is, B×X=C.

[0095] [Corrected on 20.10.2023 according to Rule 91] In this embodiment, the obtained 5-row and 4-column third characteristic matrix C is: [[0.4391,0.5395,0.6505,0.2319][1.1732,1.0799,1.3284,0.5867][2.5214,2.0240,2.4780,1.3211][3.1725,2.6054,3.3396,1.7051][3.9492,3.3759,4.3923,2.0220]].

[0096] [Corrected 20.10.2023 in accordance with Article 91] Deleted

[0097] [Corrected 20.10.2023 in accordance with Article 91] Deleted

[0098] [Corrected 20.10.2023 in accordance with Article 91] Deleted

[0099] [Corrected 20.10.2023 in accordance with Article 91] Deleted

[0100] [Corrected 20.10.2023 in accordance with Article 91] Deleted

[0101] Step S14: normalize the third characteristic matrix C with n rows and m columns to obtain a correlation characteristic matrix D with n rows and m columns.

[0102] The association feature matrix D with n rows and m columns represents the natural language association features, and the value of each element of the association feature matrix D with n rows and m columns ranges from -1 to 1.

[0103] Furthermore, the Min-Max (deviation normalization) normalization method or the Z-Score (standard score) normalization method is used to normalize the third characteristic matrix C of n rows and m columns, so that the value of each element of the obtained n rows and m columns correlation characteristic matrix D ranges from -1 to 1.

[0104] For example, for a natural language [tomorrow, the, weather, is, rainy], the 5-row, 4-column third feature matrix C is normalized using the Min-Max normalization method, so that the resulting 5-row, 4-column correlation feature matrix D is:

[0105] [[0.4005,0.3069,0.1792,0.3460]

[0106] [0.5425,0.5025,0.2324,0.5352]

[0107] [1.0000,0.9026,0.4422,0.9927]

[0108] [0.5188,0.4743,0.3340,0.5596]

[0109] [0.6203,0.5705,0.3014,0.6033]].

[0110] The 5-row and 4-column correlation feature matrix D represents the correlation features of a natural language [tomorrow, the, weather, is, rainy day].

[0111] Step S2: construct an artificial intelligence content generation model, wherein the artificial intelligence content generation model includes multiple layers of linear computing units.

[0112] As shown in Figure 3, the artificial intelligence content generation model (AIGC model) constructed by the present invention includes multiple layers of linear computing units. Specifically, the artificial intelligence content generation model includes a first linear computing unit 201 in the first layer, a second linear unit 202 in the second layer, a third linear unit 203 in the third layer, a fourth linear unit 204 in the fourth layer, a fifth linear unit 205 in the fifth layer, ..., and an Nth linear unit 20N in the Nth layer.

[0113] Step S3: The word vector of the first word corresponding to the association feature matrix D representing the natural language association features is input into the artificial intelligence content generation model.

[0114] Starting from the word vector of the first word, multi-layer linear calculation units are used to perform staggered superposition predictions in sequence to generate predicted word vectors for each word and train the artificial intelligence content generation model.

[0115] In conjunction with FIG3 , taking a natural language segment [tomorrow, the weather, is, rainy] as an example, in step S1 , the associated feature matrix D of 5 rows and 4 columns representing the associated features of the natural language is obtained as follows:

[0116] [[0.4005,0.3069,0.1792,0.3460]

[0117] [0.5425,0.5025,0.2324,0.5352]

[0118] [1.0000,0.9026,0.4422,0.9927]

[0119] [0.5188,0.4743,0.3340,0.5596]

[0120] [0.6203,0.5705,0.3014,0.6033]]。

[0121] The word vectors of each word corresponding to the 5-row and 4-column association feature matrix D representing natural language association features are as follows:

[0122] "Tomorrow": [0.4005,0.3069,0.1792,0.3460];

[0123] "Of": [0.5425,0.5025,0.2324,0.5352];

[0124] "Weather": [1.0000,0.9026,0.4422,0.9927];

[0125] "Is": [0.5188,0.4743,0.3340,0.5596]

[0126] "Rainy day": [0.6203,0.5705,0.3014,0.6033].

[0127] Denote the word vector corresponding to the first word "Tomorrow" of the 5-row and 4-column association feature matrix D representing natural language association features as x1, and input it into the artificial intelligence content generation model.

[0128] Starting from the word vector of the first word of the association feature matrix D through the multi-layer linear calculation unit, perform staggered superposition prediction in sequence to generate the predicted word vectors of each word after the first word of the association feature matrix D, and train the artificial intelligence content generation model.

[0129] Table 1: Table of generated predicted word vectors

[0130] As shown in Table 1, the word vector of the first word "Tomorrow" is denoted as x1, and the predicted word vector of the word "Of" is denoted as y2.

[0131] Similarly, the word vector of the word "Tomorrow of" is denoted as x2, and the predicted word vector of the word "Weather" is denoted as y3.

[0132] Similarly, the word vector of the word "tomorrow's weather" is recorded as x3, and the predicted word vector of the word "is" is recorded as y4.

[0133] Similarly, the word vector of the word "tomorrow's weather is" is recorded as x4, and the predicted word vector of the word "rainy day" is recorded as y5.

[0134] The word vector of the natural language "tomorrow's weather is rainy" is recorded as x5, and the added sentence end symbol <eos>, the predicted word vector of the natural language "The weather tomorrow is rainy" is denoted as y6.

[0135] Combined with Figure 3, in a specific embodiment, the 5-row and 4-column association feature matrix D representing the natural language association features corresponding to the word vector x1 of the first word "tomorrow" is input into the first linear calculation unit 201 of the artificial intelligence content generation model. After passing through the first linear calculation unit 201, the predicted word vector y2 of the next word "of" is output.

[0136] The word vector x1 of the word "tomorrow" is superimposed with the predicted word vector y2 of the word "of" to generate the word vector x2 of the word "tomorrow of" = x1 × y2 + x1.

[0137] The word vector x2 of the word "tomorrow of" is input into the second linear calculation unit 202 of the artificial intelligence content generation model. After passing through the second linear calculation unit 202, the predicted word vector y3 of the next word "weather" is output.

[0138] The word vector x2 of the word "tomorrow of" is superimposed with the predicted word vector y3 of the word "weather" to generate the word vector x3 of the word "tomorrow's weather" = x2 × y3 + x2.

[0139] The word vector x3 of the word "tomorrow's weather" is input into the third linear calculation unit 203 of the artificial intelligence content generation model. After passing through the third linear calculation unit 203, the predicted word vector y4 of the next word "is" is output.

[0140] The word vector x3 of the word "tomorrow's weather" is superimposed with the predicted word vector y4 of the word "is" to generate the word vector x4 of the word "tomorrow's weather is" = x3 × y4 + x3.

[0141] The word vector x4 of the word "tomorrow's weather is" is input into the fourth linear calculation unit 204 of the artificial intelligence content generation model. After passing through the fourth linear calculation unit 204, the predicted word vector y5 of the next word "rainy day" is output.

[0142] The word vector x4 of the word "tomorrow's weather is" is superimposed with the predicted word vector y5 of the word "rainy day" to generate the word vector x5 of the natural language "The weather tomorrow is rainy" = x4 × y5 + x4.

[0143] The word vector x5 of the natural language "The weather tomorrow is rainy" is input into the fifth linear calculation unit 205 of the artificial intelligence content generation model. After passing through the fifth linear calculation unit 205, the predicted word vector y6 of the natural language "The weather tomorrow is rainy" is output.

[0144] In this embodiment, a piece of natural language "[Tomorrow, of, weather, is, rainy day]" represents an association feature matrix D of 5 rows and 4 columns for natural language association features, with a total of 5 words: "Tomorrow", "of", "weather", "is", "rainy day".

[0145] In some preferred embodiments, a word sequence of the natural language is obtained by taking one character of the natural language as one word, and an association feature matrix D representing the natural language association features is generated. For example, for a piece of natural language "[Ming, Tian, of, Tian, qi, is, xia, yu, tian]", a word sequence of the natural language is obtained by taking one character as one word, and an association feature matrix D representing the natural language association features is generated, with a total of n = 9 words: "Ming", "Tian", "of", "Tian", "qi", "is", "xia", "yu", "tian".

[0146] As shown in Figure 3, when n = 9, starting from the word vector x1 of the first word of the association feature matrix D, according to x n = x n-1 × y n + x n-1 successively perform staggered superposition prediction to generate the predicted word vector of each word after one word of the association feature matrix D: y n+1 , where n = 1, 2,..., 9, x1 = x0, y1 = 0.

[0147] The word vector corresponding to the first word "Tomorrow" of the association feature matrix D is input into the artificial intelligence content generation model. Starting from the word vector of the first word of the association feature matrix D, through a multi-layer linear calculation unit, successively perform staggered superposition prediction to generate the predicted word vector of each word after one word of the association feature matrix D, and train the artificial intelligence content generation model.

[0148] Step S4: Evaluate each predicted word vector through a loss function to enhance the memory of natural language association features.

[0149] In step S3, the predicted word vector of each generated word is evaluated through a loss function to enhance the memory of natural language association features, enabling the artificial intelligence content generation model to enhance the output of "of", "weather", "is", "rainy day", " <eos>"Memory.

[0150] In one embodiment, the loss function adopts a cross entropy loss function. Further, a CrossEntropyLoss cross entropy loss function or an nllloss cross entropy loss function is adopted.

[0151] The present invention no longer predicts natural language based on the transition probability of natural language, but predicts natural language based on objective facts. By constructing an artificial intelligence content generation model, the word vectors of the association feature matrix representing the association features of natural language based on linear calculation are input into the artificial intelligence content generation model for sequential staggered superposition prediction. The artificial intelligence content generation model is trained, the memory of natural language association features is enhanced, and the predicted natural language conforms to objective facts.

[0152] As shown in FIG4 , according to an embodiment of the present invention, a memory enhancement device for natural language association features is provided, comprising:

[0153] The correlation feature matrix D generating unit 100 is used to obtain a word sequence of a natural language and generate a correlation feature matrix D representing correlation features of the natural language.

[0154] The artificial intelligence content generation model construction unit 200 is used to construct an artificial intelligence content generation model, wherein the artificial intelligence content generation model includes multiple layers of linear calculation units.

[0155] The model training unit 300 is used to input the word vector of the first word corresponding to the correlation feature matrix D representing the natural language correlation features into the artificial intelligence content generation model;

[0156] It also uses multi-layer linear computing units to start from the word vector of the first word, perform staggered superposition predictions in sequence, generate predicted word vectors for each word, and train the artificial intelligence content generation model.

[0157] The evaluation unit 400 is used to evaluate each predicted word vector using a loss function to enhance the memory of natural language related features.

[0158] According to an embodiment of the present invention, a storage medium is provided for storing computer-executable instructions, and the computer-executable instructions are used to execute a memory enhancement method for natural language association features provided by the present invention.

[0159] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.< / eos> < / eos>

Claims

1. A method for enhancing memory of natural language association features, characterized in that: The memory enhancement method comprises the following steps: S1, obtaining a natural language word sequence and generating a correlation feature matrix D representing the correlation features of the natural language; S2. constructing an artificial intelligence content generation model, wherein the artificial intelligence content generation model includes multiple layers of linear computing units; S3, inputting the word vector of the first word corresponding to the association feature matrix D representing the association features of natural language into the artificial intelligence content generation model; Starting from the word vector of the first word, multi-layer linear computing units are used to perform offset superposition predictions in sequence to generate predicted word vectors for each word, and train the artificial intelligence content generation model. S4. Each predicted word vector is evaluated through the loss function to enhance the memory of natural language related features.

2. The memory enhancement method according to claim 1, characterized in that: In step S1, generating a correlation feature matrix D representing natural language correlation features includes the following method steps: S11, obtaining a word sequence of a natural language, and generating a first feature matrix A of the natural language with n rows and m columns, wherein n is the number of words in the natural language, and m is the number of encoding bits of each word; S12, cumulatively summing each column of the first characteristic matrix A with n rows and m columns to generate a second characteristic matrix B with n rows and m columns; S13, performing a linear transformation of a neural network on the second characteristic matrix B having n rows and m columns to obtain a third characteristic matrix C having n rows and m columns; S14, normalize the third characteristic matrix C of n rows and m columns to obtain n rows The correlation feature matrix D has m columns; the correlation feature matrix D represents the natural language correlation features.

3. The memory enhancement method according to claim 2, characterized in that: In step S11, a unique and non-repetitive ID number is assigned to each word of the natural language to obtain a word sequence of the natural language; For each word ID number, an m-bit code is randomly generated to generate the first feature matrix A of the natural language with n rows and m columns.

4. The memory enhancement method according to claim 2, characterized in that: In step S12, for each element value of each column of the first characteristic matrix A, the previous element value is sequentially accumulated to generate a second characteristic matrix B with n rows and m columns.

5. The memory enhancement method according to claim 2, characterized in that: In step S13, a linear transformation matrix X with m rows and m columns is constructed, and the second characteristic matrix B with n rows and m columns is multiplied by the linear transformation matrix X with m rows and m columns to obtain a third characteristic matrix C with n rows and m columns.

6. The memory enhancement method according to claim 2, characterized in that: In step S14, the value of each element of the association feature matrix D with n rows and m columns ranges from -1 to 1.

7. The memory enhancement method according to claim 2, characterized in that: In step S14, the third characteristic matrix C of n rows and m columns is normalized by using the Min-Max normalization method or the Z-Score normalization method, so that the value of each element of the obtained association characteristic matrix D of n rows and m columns ranges from -1 to 1.

8. The memory enhancement method according to claim 1, characterized in that: In step S1, a natural language word sequence is obtained by taking a character of the natural language as a term, and a correlation feature matrix D representing the correlation features of the natural language is generated.

9. A memory enhancement device for natural language association features, characterized in that: The memory enhancement device comprises: A correlation feature matrix D generating unit, used to obtain a word sequence of a natural language and generate a correlation feature matrix D representing correlation features of the natural language; An artificial intelligence content generation model construction unit, used to construct an artificial intelligence content generation model, wherein the artificial intelligence content generation model includes multiple layers of linear calculation units; A model training unit, for inputting the word vector of the first word corresponding to the association feature matrix D representing the association features of the natural language into the artificial intelligence content generation model; And through multi-layer linear computing units, starting from the word vector of the first word, dislocation superposition prediction is performed in sequence to generate the predicted word vector of each word, and the artificial intelligence content generation model is trained; The evaluation unit is used to evaluate each predicted word vector through a loss function to enhance the memory of natural language related features.

10. A storage medium, characterized in that: The storage medium is used to store computer-executable instructions, and the computer-executable instructions are used to execute the memory enhancement method described in any one of claims 1 to 8.

Images