A natural language processing method

Abstract

The present invention discloses a natural language processing method, which specifically includes the following steps: obtaining keywords to be processed, performing permutation and combination processing on the keywords to be processed to obtain a prompt outline, using a language model to generate a natural sentence from the prompt outline according to the perplexity, and building a fact library; using a pre-trained language parsing model to obtain a semantic path corresponding to the natural sentence; based on the semantic structure, generating a set of semantic paths between any two tokens in the natural sentence to build a semantic path library, and storing it in a semantic field database; calculating the similarity of the two semantic fields; evaluating the degree of fact deviation of replacing tokens in the natural sentence; training and initializing a semantic encoding model based on the degree of fact deviation, performing iterative optimization, and obtaining a semantic parsing model; the present invention has significantly improved accuracy performance, stronger model interpretability, and lower computational complexity, and can reduce the computational complexity of a deep learning language model and reduce computing power costs.

Description

Technical Field

[0001] The present invention relates to the field of computer-related technologies, and in particular to a natural language processing method. Background Art

[0002] Knowledge-intensive reasoning uses factual statements—natural sentences describing facts—retrieved from a knowledge base to perform reasoning and make decisions. The most basic type of knowledge-intensive reasoning is commonsense reasoning, which involves forming basic assumptions about everyday cases. Commonsense reasoning is essential for humans to think and interact with the world. Therefore, endowing machines with commonsense reasoning capabilities in practical forms (e.g., question answering, reading comprehension) is fundamental to general artificial intelligence.

[0003] Generative language models are a type of model that can generate text based on input data. These models are typically based on neural networks, such as LSTM or Transformer. They can be used for tasks such as text summarization, conversational bots, and translation. Generative models require a large amount of training data and, after training, can generate corresponding outputs (such as complete articles or conversations) based on given inputs (such as input sentences or summaries). Generally speaking, these models combine an encoder-decoder architecture and an attention mechanism to generate text.

[0004] Large-scale pre-trained language models (LLMs) have strong natural language understanding capabilities. Therefore, they are used as the cornerstone of commonsense reasoning. However, LLMs require an explicit mechanism to handle knowledge-intensive information. As a viable solution for interpreting knowledge-centric data, knowledge graphs (KGs) have been successful in encoding topological features between entities. KGs are indispensable in providing context for LLMs in the form of entities, which are associated with substantive relations to derive answers. Mainstream commonsense reasoning methods are those that couple LLMs with KGs, including KG-BERT, KagNet, QA-GNN, and GreaseLM. They improve accuracy by combining the advantages of natural language understanding and structural knowledge guidance. However, there is still room for improvement in the performance, interpretability, and sustainability of commonsense reasoning.

[0005] Reinforcement Learning-Based Human Feedback Optimized Language Model (RLHF) uses reinforcement learning methods to directly optimize language models with human feedback. RLHF enables language models to begin aligning models trained on a general text data corpus with models of complex human values, guiding the training of intelligent agents through human preferences. Specifically, it requires humans to evaluate the pros and cons of a series of different strategies, and then uses these evaluation results as training data to train the deep neural network of the intelligent agent. In this way, the intelligent agent can learn strategies that are more in line with expectations under the guidance of human preferences. In addition to reducing training time and improving the performance of intelligent agents, RLHF can also play a role in many real-world scenarios, such as game design, autonomous driving, etc. By using human preferences to guide the training of intelligent agents, we can better meet needs and create more intelligent and humane technical applications.

[0006] Syntactic analysis is an important tool for analyzing the semantic components of sentences. It provides auxiliary features for natural language processing tasks. Syntactic analysis can be divided into two categories: component analysis and dependency analysis.

[0007] Constituent syntactic analysis is used to identify phrase structures within a sentence and the hierarchical syntactic relationships between phrases. The main steps are: first, part-of-speech analysis of the words in the sentence, then combining adjacent words into longer phrases, and gradually recursively reducing them to complete sentences. The final representation of constituent syntactic analysis is generally a tree structure (constituent tree): a text is converted into phrases, with non-leaf nodes in the constituent tree representing phrase types and leaf nodes representing words in the sentence.

[0008] Dependency parsing automatically analyzes the dependency syntactic structure in text, enabling accurate understanding of natural language. This technology leverages the dependency relationships between words in a sentence to represent the syntactic structure of each word (e.g., subject-verb, verb-object, attributive, etc.), and uses a tree-like structure to represent the structure of the entire sentence (e.g., subject-verb-object, attributive-adverbial-complement).

[0009] Part-of-speech tagging is used to determine the part of speech of words in a given sentence and tag them. Common part-of-speech tagging methods include the following:

[0010] The basic idea of ​​rule-based part-of-speech tagging is to construct part-of-speech disambiguation rules based on word collocation and context. Early part-of-speech tagging rules were generally constructed manually.

[0011] Part-of-speech tagging methods based on statistical models are primarily used to solve sequence tagging problems. Specifically, given a sequence, this method predicts the part-of-speech of a word based on the tagging results of all words preceding it in the sentence. The most common methods currently used include the Hidden Markov Model (HMM) and Conditional Random Field (CRF).

[0012] Part-of-speech tagging methods based on deep learning, such as BiLSTM+CRF.

[0013] For knowledge reasoning, existing technologies focus more on combining knowledge with semantic features, which are mainly obtained from knowledge graphs and pre-trained neural language models. However, areas such as performance, interpretability, and sustainability still need improvement. For example, although large-scale pre-trained language models have strong natural language processing capabilities, they still require clear mechanisms to handle knowledge-intensive information. Existing methods use a "knowledge as embedding" strategy to aggregate the output of pre-trained language models and knowledge triples into a fixed-dimensional representation. This still has shortcomings such as imperfect and easy loss of information annotation in common sense reasoning tasks. Summary of the Invention

[0014] The purpose of the present invention is to solve the above problems and to design a natural language processing method.

[0015] To achieve the above-mentioned purpose, the technical solution of the present invention is that, further, in the above-mentioned natural language processing method, the natural language processing method includes the following steps:

[0016] Obtaining keywords to be processed, performing permutation and combination processing on the keywords to be processed to obtain a prompt summary, generating natural sentences from the prompt summary using a language model according to the perplexity, and building a fact library;

[0017] Using a pre-trained language parsing model to obtain a semantic path corresponding to the natural sentence;

[0018] Based on the semantic structure, a set of semantic paths between any two tokens in the natural sentence is generated to construct a semantic path library, and the library is stored in a semantic field database;

[0019] Calculating the similarity of two semantic fields, where the semantic field is a set of semantic paths;

[0020] evaluating a degree of factual bias in replacing tokens in the natural sentence;

[0021] The semantic encoding model is initialized based on the degree of factual deviation training, and iterative optimization is performed to obtain the semantic parsing model.

[0022] Furthermore, in the above-mentioned natural language processing method, obtaining keywords to be processed, performing permutation and combination processing on the keywords to be processed to obtain a prompt summary, generating natural sentences from the prompt summary using a language model according to the perplexity, and constructing a fact database include:

[0023] Obtaining a plurality of groups of keywords to be processed, performing permutation and combination processing on the keywords to be processed to obtain a prompt summary, and using a generative language model to expand the prompt summary to generate a complete natural sentence;

[0024] The perplexity of the natural sentences is calculated using a pre-trained language model, and the natural sentences whose perplexity exceeds a preset threshold are eliminated.

[0025] Furthermore, in the above-mentioned natural language processing method, after removing the natural sentences whose perplexity exceeds a preset threshold, the method further includes:

[0026] Obtaining the eliminated natural sentences, and manually sorting the eliminated natural sentences based on a reinforcement learning method to obtain a sorting result;

[0027] Based on the ranking results, improving the generative language model according to the RLHF method;

[0028] Obtaining a prompt summary, and inputting the prompt summary into the improved generative language model to obtain a natural sentence;

[0029] A fact base is constructed based on the prompt summary and the corresponding natural sentence, and is stored in the fact base.

[0030] Furthermore, in the above-mentioned natural language processing method, the obtaining of the semantic path corresponding to the natural sentence by using a pre-trained language parsing model includes:

[0031] Obtaining a pre-trained language parsing model, wherein the language parsing model is a component syntax parsing model, a dependency syntax parsing model, and a part-of-speech tagging model;

[0032] The natural sentence is input into the pre-trained language parsing model to obtain a parsing result, and the parsing result is sorted to obtain a semantic structure.

[0033] Furthermore, in the above-mentioned natural language processing method, based on the semantic structure, a set of semantic paths directly between any two tokens in the natural sentence is generated to construct a semantic path library, and the set is stored in a semantic field database, including:

[0034] Suppose there is a set of N morphemes w1, w2...w N , mark the natural sentence of length k composed of morphemes in a specific order as

[0035] Read the semantic path library and determine the natural sentences in the fact library Does it contain a certain word?

[0036] If so, the index keys corresponding to the natural statement are read including

[0037] The index key is input into the language parsing model to obtain a set of semantic paths, and the

[0038] Stored as index keys in the semantic field database.

[0039] Furthermore, in the above natural language processing method, the calculating the similarity between two semantic fields includes:

[0040] Get index keys Corresponding semantic fields and index keys a corresponding semantic field, wherein the semantic field is a set of semantic paths;

[0041] The similarity between the two semantic fields is calculated by a similarity matching algorithm, wherein the similarity matching algorithm evaluates the similarity between two constituent elements by a trainable metric function.

[0042] Furthermore, in the above-mentioned natural language processing method, the evaluating the degree of factual deviation of replacing tokens in the natural sentence includes:

[0043] Calculate the natural sentences and The similarity of the corresponding semantic fields is evaluated using an evaluation mechanism to obtain a final value, wherein the evaluation mechanism is to take the maximum value or the average value.

[0044] Furthermore, in the above-mentioned natural language processing method, the initialization of the semantic encoding model based on the fact deviation degree training and the iterative optimization to obtain the semantic parsing model include:

[0045] Call to initialize the semantic encoding model, where the initialized semantic encoding model is the LSTM or Transformer semantic encoding model;

[0046] Input a natural sentence, train the initialized semantic encoding model, and output a relationship result between tokens in the natural sentence, wherein the relationship result is a vector of a specific dimension;

[0047] Use the deep learning framework to perform backpropagation training on the trained initialized semantic encoding model, replace a word token in the natural sentence with another word, and determine the deviation of the natural sentence. The larger the deviation, the higher the loss value of the corresponding loss function.

[0048] The loss value is determined by the deviation of the natural sentence, and the loss value is iteratively optimized on the randomly initialized semantic encoding model using the back propagation algorithm to obtain the semantic parsing model.

[0049] Its beneficial effects are that, by obtaining keywords to be processed, the keywords to be processed are arranged and combined to obtain a prompt outline, and a language model is used to generate a natural sentence from the prompt outline according to the perplexity, and a fact library is constructed; a pre-trained language parsing model is used to obtain the semantic path corresponding to the natural sentence; based on the semantic structure, a set of semantic paths directly between any two tokens in the natural sentence is generated to construct a semantic path library, and stored in a semantic field database; the similarity of two semantic fields is calculated, wherein the semantic field is a semantic path set; the degree of fact deviation of replacing the tokens in the natural sentence is evaluated; the semantic encoding model is initialized based on the fact deviation degree, and it is iteratively optimized to obtain a semantic parsing model; the accuracy performance of the present invention is significantly improved, the model is more interpretable, and users are supported to adjust the content in the fact library to customize the model for specific scenarios. The computational complexity is low, and the computational complexity of large-scale deep learning language models can be reduced, reducing computing power costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Various other advantages and benefits will become apparent to those skilled in the art by reading the following detailed description of the preferred embodiment.The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0051] Figure 1 Schematic diagram of a natural language processing method according to an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of a first embodiment of building a semantic path library based on a pre-trained model in an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of a second embodiment of building a semantic path library based on a pre-trained model in an embodiment of the present invention;

[0054] Figure 4 Schematic diagram of an embodiment of constructing a dynamic reasoning mechanism based on a pre-trained model in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0056] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a," "an," "said," and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0057] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 As shown, a natural language processing method comprises the following steps:

[0058] Step 101: Obtain keywords to be processed, perform permutation and combination processing on the keywords to obtain a prompt summary, generate natural sentences from the prompt summary using a language model based on the perplexity, and build a fact database;

[0059] In this embodiment, several groups of keywords to be processed are obtained, and the keywords to be processed are arranged and combined to obtain a prompt summary. The prompt summary is expanded using a generative language model to generate a complete natural sentence; the perplexity of the natural sentence is calculated using a pre-trained language model, and natural sentences whose perplexity exceeds a preset threshold are eliminated.

[0060] In this embodiment, the eliminated natural sentences are obtained, and the eliminated natural sentences are manually sorted based on the reinforcement learning method to obtain the sorting results; based on the sorting results, the generative language model is improved according to the RLHF method; the prompt summary is obtained, and the prompt summary is input into the improved generative language model to obtain natural sentences; based on the prompt summary and the corresponding natural sentences, a fact library is constructed and stored in the fact library.

[0061] In this embodiment, the method can generate natural sentences that conform to common sense based on keywords.

[0062] Step 1: Arrange and combine several keywords to generate a prompt summary, and then use a generative language model (such as the GENIUS model) to expand the prompt summary into a complete natural sentence.

[0063] Step 2: Use a pre-trained language model (such as the BERT model) to calculate the perplexity of the natural sentences generated in the previous step, and remove natural sentences whose perplexity exceeds a specific threshold (such as 80).

[0064] Step 3: If the perplexity of natural sentences generated by various permutations and combinations of a set of keywords is generally high, a reinforcement learning method based on human feedback (RLHF) is introduced. These high-perplexity sentences are first manually sorted according to their common sense rationality. The sorting results are then used as rewards. The generative language model is improved based on the RLHF method to generate natural sentences with higher common sense rationality.

[0065] Step 4: Use the prompt summary as the index key, and then use the natural sentences generated by the prompt summary with high common sense rationality as the storage content (value) corresponding to the index, and store them in the fact database for subsequent training and retrieval.

[0066] Step 102: Using a pre-trained language parsing model to obtain a semantic path corresponding to a natural sentence;

[0067] In this embodiment, a pre-trained language parsing model is obtained, wherein the language parsing model is a component syntax analysis model, a dependency syntax analysis model, and a part-of-speech tagging model; a natural sentence is input into the pre-trained language parsing model to obtain a parsing result, and the parsing result is sorted to obtain a semantic structure.

[0068] In this embodiment, the function is briefly described as follows: converting natural sentences into multi-level semantic structure graphs.

[0069] Step 1: Use pre-trained language parsing models, including but not limited to: constituency parsing model, dependency parsing model, and part-of-speech tagging model, to perform semantic analysis on the current natural sentence.

[0070] Step 2: Arrange the parsing results. For parsing models whose parsing form is triples (e.g., component parsing model, dependency parsing model), that is, such models can obtain the relationship between each two different tokens, thereby forming a basic parsing element in the form of "token-relation-token" (e.g., Figure 2 Structure1 and Structure2 in ).

[0071] Step 3: If it is not possible to ensure that there is a path between all tokens (for example, a part-of-speech tagging model that can only connect each token to one relationship), you can build a root node (ROOT) and connect all relationships (such as Figure 2 Structure3 in ).

[0072] Step 4: Based on the semantic structure obtained in the previous step, the set of semantic paths between any two tokens in the current natural sentence can be obtained, such as Figure 3 There are three different semantic paths between the word tokens who and wear, corresponding to three different language parsing models (component syntactic analysis model, dependency syntactic analysis model, and part-of-speech tagging model).

[0073] Step 5: Use "token 1-token 2-sentence ID" as the index key, "language parsing model ID" as the sub-index key, and then store the semantic path between token 1 and token 2 in the form of a list as the storage content corresponding to the index and sub-index to build a semantic path library.

[0074] Step 103: Based on the semantic structure, a set of semantic paths between any two tokens in the natural sentence is generated to construct a semantic path library, and the library is stored in a semantic field database.

[0075] In this embodiment, suppose there are N morpheme sets w1, w2...w N , mark the natural sentence of length k composed of morphemes in a specific order as Read the semantic path library and judge the natural sentences in the fact library Whether to include If so, the index keys corresponding to the natural statement are read including Input the index key into the language parsing model to obtain the semantic path set, and use Stored as index keys in the semantic field database.

[0076] In this embodiment, the function is to judge a natural sentence S (j) A word in

[0077] with another who may not be in S (j) The tokens in Whether there is knowledge-level equivalence in a specific fact base.

[0078] Step 1: First, build the knowledge base of the scenario, that is, according to the construction of the fact base, use the given keywords (corresponding to the common sense that should be regarded as true propositions) to generate a specific fact base (marked as K), which contains M natural sentences.

[0079] Step 2: Use the semantic path library based on the pre-training model to combine the natural sentences in the fact base generated in step 1 and S (j) Transformed into a semantic field and formed a semantic path library.

[0080] Step 3: All the semantic path libraries The index key of the form "token 1-token 2-sentence ID" and its corresponding storage content are read out as "token 1". Specifically, if a natural sentence in the fact database Include The index keys related to this sentence that need to be read include:

[0081] Step 4: For ease of explanation, build a semantic field database and replace the index key in step 3 with The set of semantic paths (i.e., semantic fields) obtained in different language parsing models is Stored as index keys in the semantic field database.

[0082] Step 104: Calculate the similarity between two semantic fields, where a semantic field is a set of semantic paths;

[0083] In this embodiment, the index key is obtained Corresponding semantic fields and index keys The corresponding semantic field, where the semantic field is a set of semantic paths;

[0084] The similarity between two semantic fields is calculated by a similarity matching algorithm, where the similarity matching algorithm evaluates the similarity of two constituent elements through a trainable metric function.

[0085] In this embodiment, step five: index key The corresponding semantic field and index key " Similarity of corresponding semantic fields Simplified mark Calculated by a specific similarity matching algorithm, which includes but is not limited to dynamic programming algorithms, such as Figure 4 This example is modified from the LCS (Longest Common Subsequence) algorithm to calculate the similarity between two sequence paths. The algorithm also considers the similarity of the constituent elements in the paths to be matched. That is, a trainable metric function g is used to evaluate the similarity between the two constituent elements.

[0086] Step 105: Evaluate the degree of factual deviation of replacing tokens in natural sentences;

[0087] In this embodiment, the natural sentences and The similarity of the corresponding semantic fields is evaluated to obtain the final value, which is the maximum value or the average value.

[0088] In this embodiment, step six: using step five, all the sentences S in the fact base are (i) and The semantic field similarity is calculated and we get: Then use the evaluation mechanism (including but not limited to: taking the maximum value, taking the average value) to get the final value, which represents the natural sentence S (j) The tokens in With a certain word The higher the value of knowledge equivalence in the fact base K, the more equivalent the two are.

[0089] Step 106: Initialize the semantic encoding model based on the degree of factual deviation training, perform iterative optimization, and obtain a semantic parsing model.

[0090] In this embodiment, an initialized semantic coding model is called, wherein the initialized semantic coding model is a semantic coding model of LSTM or Transformer; a natural sentence is input, the initialized semantic coding model is trained, and the relationship results between the word tokens in the natural sentence are output, wherein the relationship results are vectors of a specific dimension; the trained initialized semantic coding model is back-propagated using a deep learning framework, a word token in the natural sentence is replaced with another word, and the deviation of the natural sentence is determined. If the deviation is large, the loss value of the corresponding loss function is higher; the loss value is determined by the deviation of the natural sentence, and the loss value is iteratively optimized on the randomly initialized semantic coding model using a back-propagation algorithm to obtain a semantic parsing model.

[0091] In this embodiment, step 1: obtaining the semantic relationships between tokens in natural sentences without relying on a pre-trained semantic parsing model. Several semantic encoding models, such as LSTM or Transformer, are randomly initialized (the input is a natural sentence, and the output is the relationship between tokens in the sentence, expressed as a vector of a specific dimension). Each randomly initialized model corresponds to a semantic structure. Subsequent operations are consistent with the above method.

[0092] Step 2: Use a deep learning framework (such as PyTorch) to perform backpropagation training on the current model. The training mechanism is to replace a word in a factual sentence with another word. If the sentence deviates from common sense more after the replacement, the loss value of the corresponding loss function will be higher. The loss value is then used to iteratively optimize the randomly initialized semantic encoding model using the backpropagation algorithm.

[0093] In an embodiment of the present invention, keywords to be processed are obtained, and the keywords to be processed are arranged and combined to obtain a prompt outline. A language model is used to generate a natural sentence from the prompt outline according to the perplexity, and a fact library is constructed; a pre-trained language parsing model is used to obtain the semantic path corresponding to the natural sentence; based on the semantic structure, a set of semantic paths directly between any two tokens in the natural sentence is generated to construct a semantic path library, and stored in a semantic field database; the similarity of two semantic fields is calculated, where the semantic field is a set of semantic paths; the degree of fact deviation of replacing tokens in the natural sentence is evaluated; the semantic encoding model is initialized based on the fact deviation degree, and iterative optimization is performed to obtain a semantic parsing model; the present invention has significantly improved accuracy performance, and the model is more interpretable. It supports users to adjust the content in the fact library to customize the model for specific scenarios, and has low computational complexity. It can reduce the computational complexity of large-scale deep learning language models and reduce computing power costs.

[0094] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A natural language processing method, characterized in that: The natural language processing method comprises the following steps: Obtaining keywords to be processed, performing permutation and combination processing on the keywords to be processed to obtain a prompt summary, generating natural sentences from the prompt summary using a language model according to the perplexity, and building a fact library; Using a pre-trained language parsing model to obtain a semantic path corresponding to the natural sentence to obtain a semantic structure; Based on the semantic structure, a set of semantic paths between any two tokens in the natural sentence is generated to construct a semantic path library, and the library is stored in a semantic field database; Calculating the similarity of two semantic fields, where the semantic field is a set of semantic paths; evaluating a degree of factual bias in replacing tokens in the natural sentence; Based on the degree of factual deviation, the semantic encoding model is initialized and iteratively optimized to obtain the semantic parsing model; The step of generating a set of semantic paths between any two tokens in the natural sentence based on the semantic structure to construct a semantic path library and storing the library in a semantic field database includes: Suppose there is a set of N morphemes w1,w2...w N , mark the natural sentence of length k composed of morphemes in a specific order as Read the semantic path library and determine the natural sentences in the fact library Whether to include If so, the index keys corresponding to the natural statement are read including The index key is input into the language parsing model to obtain a set of semantic paths, and the Stored as index keys in the semantic field database.

2. A natural language processing method according to claim 1, characterized in that: The step of obtaining keywords to be processed, performing permutation and combination processing on the keywords to be processed to obtain a prompt summary, generating a natural sentence from the prompt summary using a language model according to the perplexity, and constructing a fact database includes: Obtaining a plurality of groups of keywords to be processed, performing permutation and combination processing on the keywords to be processed to obtain a prompt summary, and using a generative language model to expand the prompt summary to generate a complete natural sentence; The perplexity of the natural sentences is calculated using a pre-trained language model, and the natural sentences whose perplexity exceeds a preset threshold are eliminated.

3. A natural language processing method according to claim 2, characterized in that: After removing the natural sentences whose perplexity exceeds a preset threshold, the method further includes: Obtaining the eliminated natural sentences, and manually sorting the eliminated natural sentences based on a reinforcement learning method to obtain a sorting result; Based on the ranking results, improving the generative language model according to the RLHF method; Obtaining a prompt summary, and inputting the prompt summary into the improved generative language model to obtain a natural sentence; A fact base is constructed based on the prompt summary and the corresponding natural sentence, and is stored in the fact base.

4. A natural language processing method according to claim 1, characterized in that: The method of using a pre-trained language parsing model to obtain a semantic path corresponding to the natural sentence and a semantic structure includes: Obtaining a pre-trained language parsing model, wherein the language parsing model is a component syntax parsing model, a dependency syntax parsing model, and a part-of-speech tagging model; The natural sentence is input into the pre-trained language parsing model to obtain a parsing result, and the parsing result is sorted to obtain a semantic structure.

5. A natural language processing method according to claim 1, characterized in that: The calculating the similarity between two semantic fields includes: Get index keys Corresponding semantic fields and index keys a corresponding semantic field, wherein the semantic field is a set of semantic paths; The similarity between the two semantic fields is calculated by a similarity matching algorithm, wherein the similarity matching algorithm evaluates the similarity between two constituent elements by a trainable metric function.

6. A natural language processing method according to claim 1, characterized in that: The evaluation of the degree of factual deviation of replacing tokens in the natural sentence includes: Calculate the natural sentences and The similarity of the corresponding semantic fields is evaluated using an evaluation mechanism to obtain a final value, wherein the evaluation mechanism is to take the maximum value or the average value.

7. A natural language processing method according to claim 1, characterized in that: The method of initializing the semantic encoding model based on the degree of factual deviation and performing iterative optimization to obtain a semantic parsing model includes: Call to initialize the semantic encoding model, where the initialized semantic encoding model is the LSTM or Transformer semantic encoding model; Input a natural sentence, train the initialized semantic encoding model, and output a relationship result between tokens in the natural sentence, wherein the relationship result is a vector of a specific dimension; Use the deep learning framework to perform backpropagation training on the trained initialized semantic encoding model, replace a word token in the natural sentence with another word, and determine the deviation of the natural sentence. The larger the deviation, the higher the loss value of the corresponding loss function. The loss value is determined by the deviation of the natural sentence, and the loss value is iteratively optimized on the randomly initialized semantic encoding model using the back propagation algorithm to obtain the semantic parsing model.

Images