A semantic error elimination method and system based on a knowledge element model

Abstract

The application discloses a kind of based on knowledge element model's semantic error elimination method and system, it is related to knowledge element technical field, the present application is introduced structured knowledge from knowledge element to supplement more extensive semantic information, the hierarchy of context input text is used to describe the meaning of word and phrase, and constructs hybrid encoder algorithm based on context and interpretation, introduces graph attention network to reduce the noise information in context input text, to improve the error elimination accuracy of phrase form target word, and finally improve the error elimination effect of method.The error elimination accuracy of the present application is superior to comparative algorithm in most cases, and better results are obtained.

Description

Technical Field

[0001] This invention relates to the field of semantic error elimination, specifically to a semantic error elimination method and system based on a knowledge meta-model. Background Technology

[0002] Natural language is mostly ambiguous, but when a word appears in a specific context, its meaning is limited by that context. Therefore, the meaning of a target word can be interpreted based on its context. The goal of error elimination is to find the precise meaning of ambiguous words in a specific context. Once the scope of the context is determined, the choice of error elimination method becomes crucial for semantic error elimination.

[0003] Semantic error elimination has two main categories of effective methods: supervised methods and knowledge-based methods. Supervised methods treat semantic error elimination as a classification problem, using extensively annotated semantic data to categorize specific words into dictionary-defined meanings and employing machine learning and neural networks to achieve semantic error elimination. Knowledge-based methods, on the other hand, use existing lexical resources, such as knowledge bases and semantic networks, as the knowledge base for error elimination, and then determine the meaning of words based on context. Contextual representations derived from neural language models have proven effective in representing the differences between different meanings of the same word. However, these contextual representations are not easily connected to semantic networks, thus easily overlooking information from the knowledge base itself. Compared to knowledge-based methods, supervised methods currently produce better error elimination accuracy, but their flexibility is less than that of knowledge-based methods in the overall semantic error elimination task. Supervised methods use short text datasets, resulting in a limited amount of information in most corpora, making it difficult to distinguish semantics in different scenarios, which limits the application and development speed of supervised methods. Furthermore, in semantic error elimination tasks, the connections between words and phrases in the input text can affect the model's error elimination performance. When the target word for error elimination is a word in a phrase, the connection between the phrase and the word may lead to errors in word meaning judgment, thus affecting the error elimination effect. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a semantic error elimination method and system based on a knowledge element model. By introducing structured knowledge from knowledge elements to supplement a wider range of semantic information, the hierarchical structure of the context input text is used to describe the meaning of words and phrases, and a hybrid encoder algorithm based on context and paraphrasing is constructed. A graph attention network is introduced to reduce noise information in the context input text, thereby improving the accuracy of error elimination for target words in phrase form and ultimately improving the error elimination effect of the method.

[0005] This invention first discloses a semantic error elimination method based on a knowledge meta-model, the specific steps of which are as follows:

[0006] Step 1: Obtain the target word information to be eliminated. Construct a knowledge element embedding matrix based on the relationship between words and semantics in WordNet. Embed the knowledge elements into the annotation vector representation of the target words through the adjacency matrix, thereby obtaining the structured information and annotation data of the knowledge elements.

[0007] Step 2: Construct a hierarchical structure of phrases and related words based on the context of the input text;

[0008] Step 3: Construct a hybrid encoder, which consists of two parts: a context encoder and a paraphrasing encoder. The hierarchical context input text constructed in Step 2 is fed into the context encoder. The structured information and annotation data of the knowledge elements extracted in Step 1 are input into the paraphrasing encoder. Text feature vectors and structural feature vectors are extracted from the outputs of the context encoder and the paraphrasing encoder, respectively.

[0009] Step 4: The text feature vector extracted from the context encoder is weighted and aggregated with neighboring nodes through a graph attention network to obtain a weighted aggregated feature vector, thereby eliminating the influence of noisy neighboring nodes;

[0010] Step 5: The final error elimination score is obtained by performing a dot product operation between the weighted aggregated feature vector output in Step 4 and the structural feature vector output from the interpretive encoder.

[0011] Furthermore, to better utilize the structured information and annotation data of knowledge elements, this method also includes:

[0012] The constructed knowledge element embedding matrix A is multiplied by the annotation vector representation matrix of the target word to be eliminated, its weight matrix, and the bias vector to obtain the final knowledge element vector matrix Q.

[0013] Q = (HO + b) q A T +(HO+b q )

[0014] Where Q represents the obtained knowledge element vector matrix; H represents the annotation vector representation matrix formed by all annotation data of the target word to be corrected; and O represents the weight matrix of the target word to be corrected in the knowledge element; b q This represents the bias vector.

[0015] Furthermore, step 2 specifically involves:

[0016] Divide the sentence c in the context into any two parts; in each iteration, select one part and divide it into smaller partitions until the smallest segmentation unit of the sentence is reached; evaluate the relevance between the new partitions after each iteration and update the relevance set; finally, a relevance set and a hierarchical structure set are obtained; the relevance set contains all text partitions generated in each partition and the number of relevance links between words and phrases in each partition; the hierarchical structure set contains all partitions of the sentence c in the context in each partition.

[0017] Furthermore, step 3 specifically involves:

[0018] The contextual input text is processed through a hierarchical structure and then fed into the context encoder; when the target word for error elimination is a single word, start and end symbols are added to the input text and then it is fed into the pre-trained language model encoder; when the target word for error elimination is a phrase, the average vector of each word in the phrase is used to represent the phrase.

[0019]

[0020] Where j and q represent the first and last words in the segmentation of this phrase, respectively; T c [l] indicates that each word is sequentially fed into the context encoder; r ws This represents the text feature vector obtained from the input text after processing by the context encoder;

[0021] In the paraphrasing encoder, the annotation data g of the target word s to be error-eliminated s Used as input; the paraphrasing encoder generates the paraphrasing vector g of the target word s to be error-eliminated. (s) The details are as follows:

[0022] g (s) =T g (g s [0]

[0023] Among them, T g Indicates the encoder's meaning;

[0024] For each synonym of the target word, a vector score is calculated based on the structured information of the knowledge element, and the paraphrased vector score is q. s It is obtained by adding the vector score of the target word to be eliminated to the weight of the edges between the target word and all synonyms;

[0025] q s =z s +∑ s'∈V|<s'，s>∈E w(<s',s> )·Z s'

[0026] in,<s',s> q represents the edge between each target word s and its synonym s'; s w( represents the weighted combination of the vector scores of all target words;<s',s> ) represents the weight of the edges between the target word s to be eliminated and all its synonyms s'; V represents the set of all vertices in the knowledge element, and E represents all edges; Z s' The vector matrix representation of the synonym s'; z s The vector matrix representation of the target word s to be eliminated is as follows:

[0027] z s =h T g (s) +b T g (s)

[0028] Among them, h T and b T These represent the transpose of the annotation vector representation matrix and the bias vector, respectively.

[0029] The explanatory vector fraction q s It is concatenated with the knowledge element vector matrix Q to obtain the structural feature vector output by the interpretive encoder.

[0030] Furthermore, step 4 specifically involves:

[0031] The graph attention network performs weighting and aggregation in two steps: The first step is to calculate the attention coefficients. For each target word to be eliminated, the similarity coefficient between it and its neighboring nodes is calculated. Then, the attention coefficients are calculated through normalization. The second step is to obtain a new feature vector by weighted summation of the attention coefficients and perform feature enhancement transformation on it to obtain the weighted aggregated feature vector of the encoder's final output.

[0032] Furthermore, the specific process for calculating the attention coefficient is as follows:

[0033] The specific process for calculating the attention coefficient is as follows:

[0034] For the target word s to be error-eliminating, calculate the similarity coefficient between it and its neighboring node j one by one. The similarity coefficient is calculated as follows:

[0035]

[0036] Where, r ws r represents the text feature vector of the target word s obtained from the output of the context encoder; wj The text feature vector representing neighbor node j; Wr represents the set of all neighboring nodes of the target word to be eliminated;ws Indicates that for r ws The value obtained by performing a linear transformation; h represents the value obtained by performing a linear transformation; Wr ws ‖Wr wj ] indicates that in r ws and r wj The concatenation operation after feature transformation; LeakyReLU represents the activation function.

[0037] Furthermore, a new feature vector is obtained by weighted summation of the attention coefficients as follows:

[0038] Based on the obtained similarity coefficient α sj The attention coefficient β is calculated through a normalization operation. sj The specific calculation process is as follows:

[0039]

[0040] The attention coefficient β is obtained by weighted summation. sj To obtain a new feature vector, specifically:

[0041]

[0042] in, σ(·) represents the new feature vector output by the graph attention network for each word s after fusing neighborhood information; σ(·) represents the activation function.

[0043] Furthermore, the feature enhancement transformation specifically includes:

[0044] New feature representation obtained by weighted summation After feature enhancement transformation, the obtained It is the weighted aggregated feature vector that is the final output of the encoder. The specific process is as follows:

[0045]

[0046] Where K represents the number of heads in the multi-head attention mechanism; W k Let be the weight matrix for the k-th attention head; This represents the attention coefficient between the target word s and its neighbor node j in the k-th attention head.

[0047] On the other hand, the present invention also provides a semantic error elimination system based on a knowledge meta-model, the system comprising:

[0048] At least one processor; and a memory communicatively connected to said at least one processor; wherein,

[0049] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a semantic error elimination method for a knowledge meta-model as described above.

[0050] On the other hand, the present invention also provides a computer storage medium storing a semantic error elimination method based on a knowledge meta-model, wherein the method, when executed by at least one processor, implements the semantic error elimination method based on a knowledge meta-model as described above.

[0051] This invention relates to a semantic error elimination method and system based on a knowledge meta-model. It proposes a hybrid encoder semantic error elimination method combining a knowledge meta-model and a text hierarchical structure. By introducing structured knowledge from knowledge meta-models to supplement broader semantic information, the hierarchical structure of the contextual input text is used to describe the meaning of words and phrases. A hybrid encoder algorithm based on context and paraphrasing is constructed, and a graph attention network is introduced to reduce noise information in the contextual input text, thereby improving the accuracy of error elimination for phrase-form target words and ultimately enhancing the error elimination effect of the method. This invention also processes words and phrases in manually annotated contextual input text, making the dataset more suitable for the application of the semantic error elimination model presented in this paper. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of a hybrid encoder;

[0053] Figure 2 This is a schematic diagram of a semantic error elimination system based on a knowledge meta-model, provided as an embodiment of the present invention. Detailed Implementation

[0054] The detailed description that follows, taken in conjunction with the accompanying drawings, is intended to describe various exemplary embodiments of the invention and is not intended to represent only embodiments in which the invention can be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form to avoid obscuring the concept of the invention.

[0055] In the description of this invention, it should be noted that the embodiments described in this invention are only some embodiments of this invention, not all embodiments; all other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0056] The terms "first," "second," etc., used in this specification, claims, and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0057] The detailed description that follows, taken in conjunction with the accompanying drawings, is intended to describe various exemplary embodiments of the invention and is not intended to represent only embodiments in which the invention can be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form to avoid obscuring the concept of the invention.

[0058] Supervised semantic error elimination methods are extremely time-consuming and labor-intensive when acquiring large amounts of annotated data. Therefore, this invention proposes embedding structured information from knowledge elements into the annotated data and using existing knowledge metadata to alleviate the difficulty of acquiring large amounts of data. Supervised semantic error elimination methods treat semantic error elimination as a labeling and classification problem, similar to lexical annotation and named entity recognition tasks in natural language processing. Achieving semantic error elimination requires generating a vector representation of the target word in a given context and using it to generate the probability distribution of all its possible semantics. In this invention, knowledge elements are embedded into the annotated data in the form of an adjacency matrix.

[0059] In the field of natural language processing, semantic errors often lead to difficulties in understanding and information extraction. Supervised semantic error elimination methods offer an efficient solution by treating semantic error elimination as a label classification problem. However, traditional methods are both time-consuming and labor-intensive when acquiring large amounts of annotated data, which limits their widespread application.

[0060] To address this challenge, this invention proposes a novel method that alleviates the difficulty of acquiring large amounts of data by embedding structured information from knowledge elements into annotation data. This process essentially utilizes existing knowledge metadata, transforming its structured information into a format suitable for annotation.

[0061] 1. Semantic error elimination is considered a classification problem:

[0062] The method of this invention transforms semantic error elimination into a label classification task, similar to common lexical annotation and named entity recognition tasks in natural language processing. This setup can be handled using existing machine learning algorithms.

[0063] 2. Generate vector representation:

[0064] The key to achieving semantic error elimination lies in generating a vector representation of the target word in a given context, which can effectively capture the semantic information of the target word. By establishing a semantic vector model, a probability distribution of all possible semantics of the target word can be generated.

[0065] 3. Embedding of knowledge elements:

[0066] In this invention, knowledge elements are embedded into the annotation data in the form of an adjacency matrix. The use of adjacency matrix allows structured information to be intuitively and efficiently integrated into data processing, providing rich contextual information and thus improving the accuracy of semantic error elimination.

[0067] Example 1

[0068] In this invention, the knowledge element embedding matrix A is constructed based on the relationships between words and semantics in WordNet. Since there are hierarchical relationships between words, the hierarchical relationships between words are considered when constructing the knowledge element embedding matrix A. The knowledge element embedding matrix is ​​constructed through experiments using only the highest level, only the subordinate level, and simultaneously using hypernyms and hyponyms.

[0069] Step 1: Embedding structured information of knowledge elements

[0070] To embed knowledge elements, this invention performs a dot product operation between the constructed knowledge element embedding matrix A and the annotation vector representation matrix, weight matrix, and bias vector of the target word to be error-eliminated, to obtain the final knowledge element vector matrix Q, specifically:

[0071] Q = (HO + b) q A T +(HO+b q )

[0072] Where Q represents the obtained knowledge element vector matrix; since each word to be eliminated has multiple annotations, each annotation vector has a vector representation, H represents the annotation vector representation matrix formed by all annotation data of the target word to be eliminated, and O represents the weight matrix of the target word to be eliminated in the knowledge element; b q This represents the bias vector.

[0073] Step 2: Text Hierarchy

[0074] When selecting target words from input text, a target word may be a single word or a phrase, but phrases often contain multiple words, which frequently affects the judgment of the meaning of the words within the phrase. Therefore, this invention constructs a hierarchical structure by recursively detecting the degree of interaction between words and phrases and then dividing larger text partitions into smaller text partitions based on this degree of influence.

[0075] For the word sense error elimination task, the input is a sentence c containing a set of words s to be eliminated. c is defined as: c = c0, c1, ..., c i ,…,c n Among them, c i This refers to the target word at position i in the sentence, where the error needs to be eliminated. Additionally, the partitioning... A sentence partition defined as having a length of n. These represent the words that mark the endpoints of each sentence partition. The basic process of partitioning is to divide a given large partition into smaller parts. Divided into two smaller partitions, and Where j represents the dividing point, and i <j<j+1。

[0076] Starting with the entire sentence, the algorithm first divides the context sentence c into two parts. In the next iteration, it selects one of these two parts and further divides it into smaller partitions until the smallest segment of the sentence is reached. The final step of each iteration is to evaluate the relevance between the new partitions and update the relevance set R; the final output of the algorithm includes the relevance set R and the hierarchical structure set H. The relevance set R contains all the text partitions generated in each partition and the number of relevance links between words and phrases between the partitions, while the hierarchical structure set H contains all the partitions of the context sentence c in each partition.

[0077] Step 3: Hybrid Encoder

[0078] Depend on Figure 1 As can be seen, the hybrid encoder structure encodes both the context and the semantic meaning of the target word, with each encoder initialized using a pre-trained language model. Therefore, the encoder input requires start and end symbols specific to the language model. The part of the hybrid encoder structure that processes the context is called the context encoder, while the part that processes the semantic meaning is called the semantic encoder.

[0079] The contextual input text is processed through a hierarchical structure and then fed into the context encoder. When the target word for error cancellation is a single word, start and end symbols are added to the input text before it is fed into the pre-trained language model encoder. When the target word for error cancellation is a phrase, the phrase is represented using the average vector of each word in the phrase.

[0080]

[0081] Where j and q represent the first and last words in the segmentation of this phrase, respectively; T c [l] indicates that each word is sequentially fed into the context encoder; r ws This represents the text feature vector obtained from the input text after processing by the context encoder;

[0082] In the paraphrasing encoder, the annotation data g of the target word s to be error-eliminated s Used as input; the paraphrasing encoder generates the paraphrasing vector g of the target word s to be error-eliminated. (s) The details are as follows:

[0083] g (s) =T g (g s [0]

[0084] Among them, T g This indicates a delimited encoder; the first vector output by the delimited encoder (corresponding to the start marker of the input) is considered a global vector.

[0085] To achieve semantic error elimination, a vector representation of the target word to be eliminated needs to be generated within a given context, and this vector representation is used to generate the probability distribution of all possible semantics of that word. To utilize information from knowledge elements, this is denoted as G =<V,E,w> Where V represents the set of all vertices in the knowledge element, E represents all edges, and w represents the weight of the edge between two vertices.

[0086] For each synonym of the target word, a vector score is calculated based on the structured information of the knowledge element, and the paraphrased vector score is q. s It is obtained by adding the vector matrix of the target word to be eliminated to the weights of the edges between the target word and all its synonyms;

[0087] q s =z s +Σ s'∈V|<s'，s>∈E w(<s',s> )·Z s'

[0088] in,<s',s> q represents the edge between each target word s and its synonym s'; s w( represents the weighted combination of the vector scores of all target words;<s',s> Z represents the weight of the edges between the target word s to be eliminated and all its synonyms s'; s' The vector matrix representing the synonym s'; z s The vector matrix representing the target word s to be eliminated is as follows:

[0089] z s =h T g (s) +b T g (s)

[0090] Among them, h T and b T These represent the transpose of the annotation vector representation matrix and the bias vector, respectively.

[0091] The explanatory vector fraction q s The knowledge element vector matrix Q is concatenated with it to obtain the structural feature vector r output by the paraphrasing encoder. gs .

[0092] Step 4: Weighted aggregation of neighbor nodes

[0093] Since the target word to be eliminated has many neighboring words in its context, the noise information in these neighboring words can affect the accuracy of the model in eliminating the error for that word. This makes the output of the context encoder model less robust. Therefore, this paper uses a graph attention network to perform weighted aggregation of neighboring nodes to eliminate the influence of noisy neighbors on accuracy. The graph attention network used in this invention performs weighting and aggregation in two main steps. The first step is to calculate the attention coefficient. For the target word s to be eliminated, the similarity coefficient between it and its neighboring node j is calculated one by one. The similarity coefficient calculation is as follows:

[0094]

[0095] Where, r ws r represents the text feature vector of the target word s obtained from the output of the context encoder; wj N represents the text feature vector of neighbor node j; s Wr represents the set of all neighboring nodes of the target word to be eliminated; ws Indicates that for r ws The value obtained by performing a linear transformation; h represents the value obtained by performing a linear transformation; Wr ws ‖Wr wj ] indicates that in r ws and r wj The concatenation operation after feature transformation; LeakyReLU represents the activation function;

[0096] Once the similarity coefficient α is obtained... sj The attention coefficient β is then calculated through a normalization operation. sj The specific calculation process is as follows:

[0097]

[0098] In the first step, obtain the attention coefficient β. sj Then, in the second step, the attention coefficient β is weighted and summed. sj To obtain a new feature vector; the calculation process is as follows:

[0099]

[0100] in, This represents the new feature vector output by the graph attention network for each word s after fusing neighborhood information; σ(·) represents the activation function.

[0101] The new eigenvector obtained by weighted summation After feature enhancement transformation, the results are made more robust; the final result is... It is the weighted aggregated feature vector output by the encoder; the calculation process is as follows:

[0102]

[0103] Where K represents the number of heads in the multi-head attention mechanism; W k Let be the weight matrix for the k-th attention head; This represents the attention coefficient between the target word s and its neighbor node j in the k-th attention head.

[0104] Step 5: Error Elimination Accuracy Calculation

[0105] The final feature vector output by the context encoder is represented as follows: The final output feature vector of the paraphrasing encoder is represented as r. gs The final error elimination accuracy is achieved by... and r gs The result is obtained by performing a dot product operation. The calculation process is as follows:

[0106]

[0107] The semantic score of the target word in a given context; when When the value exceeds a preset threshold, the correct meaning of the target word is determined, thereby eliminating the error of the target word.

[0108] The model is trained using cross-entropy loss on the candidate semantic scores of the words s to be error-eliminated; the loss function used is shown in the following formula:

[0109]

[0110] Among them, (s,g i () represents words and their meanings.

[0111] Experiments demonstrate that the error elimination accuracy of the model proposed in this invention outperforms state-of-the-art models in most cases. The main reason for this superiority lies in its use of not only a pre-trained language model but also dual encoders to encode contextual and semantic information separately. Furthermore, for the contextual input text itself, it possesses explicit hierarchical representations between words and phrases, enhancing the model's ability to eliminate phrase errors. In addition, the model integrates structured information from knowledge elements and introduces a graph attention mechanism for noisy data in the input text, resulting in superior error elimination accuracy compared to other models.

[0112] To understand the impact of each module in the model on error cancellation accuracy, we analyzed the influence of each module on the model by examining each part of the error cancellation model separately. Simulation results show that the context and annotation encoders have the greatest impact on the model's error cancellation accuracy and are the main components of the model. On the entire dataset, the hierarchical module significantly reduces the model's error cancellation accuracy, indicating that the impact of the hierarchical module on the model's error cancellation accuracy is affected by the number of phrases in the dataset, which also has a greater impact on the model's error cancellation accuracy. The knowledge meta-structured information embedding module and the graph attention mechanism module have some impact on all datasets.

[0113] Example 2

[0114] like Figure 2 As shown, this embodiment provides a semantic error elimination system based on a knowledge element model. The system may include a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0115] Memory, used to store computer programs;

[0116] When a processor executes a computer program stored in memory, it implements the semantic error elimination method for a knowledge meta-model described in Embodiment 1 above.

[0117] As one embodiment of the present invention, the communication bus mentioned in the terminal above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0118] In one embodiment of the present invention, the memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0119] As one embodiment of the present invention, the processor described above may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0120] Meanwhile, the present invention also provides a computer storage medium, characterized in that: the computer storage medium stores a semantic error elimination method based on a knowledge meta-model, and when the method is executed by at least one processor, it implements the semantic error elimination method based on a knowledge meta-model described above.

[0121] In summary, this invention supplements broader semantic information by introducing structured knowledge from knowledge elements, uses the hierarchical structure of the contextual input text to describe the meaning of words and phrases, constructs a hybrid encoder algorithm based on context and paraphrasing, and introduces a graph attention network to reduce noise information in the contextual input text, thereby improving the accuracy of error elimination for target words in phrase form and ultimately enhancing the error elimination effect of the method.

[0122] In the embodiments provided herein, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, apparatuses, or units, or they may be electrical, mechanical, or other forms of connection.

[0123] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments described herein, depending on actual needs.

[0124] Furthermore, the functional units in the various embodiments of this document can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0125] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this paper, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this paper. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A semantic error elimination method based on a knowledge element model, the specific steps being as follows: Step 1: Obtain target word information to be error-corrected, construct a knowledge element embedding matrix based on the relationship between words and semantics in WordNet, embed the knowledge element into the annotation vector representation of the target word through an adjacency matrix, and thus obtain structured information and annotation data of the knowledge element; Step 2: Construct a hierarchical structure of phrases and related words based on the context input text; the method for constructing the hierarchical structure is to divide large text partitions into small text partitions by recursively detecting the interaction degree between words and phrases and dividing the large text partitions according to the interaction degree; Step 3: Construct a hybrid encoder, which consists of two parts, including a context encoder and a paraphrase encoder, wherein the context input text with a hierarchical structure constructed in step 2 is input into the context encoder; the structured information and annotation data of the knowledge element extracted in step 1 are input into the paraphrase encoder; and text feature vectors and structure feature vectors are extracted from the output results of the context encoder and the paraphrase encoder, respectively; Step 4: The text feature vectors extracted from the context encoder are weighted and aggregated by a graph attention network to obtain weighted aggregation feature vectors, so as to eliminate the influence of noise adjacent nodes; Step 5: The final error elimination score is obtained by performing dot product operation on the weighted aggregation feature vectors output in step 4 and the structure feature vectors output from the paraphrase encoder.

2. The semantic error elimination method based on the knowledge element model according to claim 1, in order to better utilize the structured information and annotation data of the knowledge element, the method further comprises: Performing dot product operation on the constructed knowledge element embedding matrix A, the annotation vector representation matrix of the target word to be error-corrected, and the weight matrix and bias vector thereof to obtain a final knowledge element vector matrix Q; Q = (H0 + b q ) A T + (H0 + b q ) Wherein, Q represents the obtained knowledge element vector matrix; H represents the annotation vector representation matrix formed by all annotation data of the target word to be error eliminated, O represents the weight matrix of the target word to be error eliminated in the knowledge element; b q represents a bias vector.

3. The method of claim 1, wherein, Step 2 is specifically: Divide the sentence c in the context into any two parts; in each iteration, select one part and divide it into smaller partitions until the smallest partition unit of the sentence is reached; After each iteration, evaluate the correlation between the new partitions and update the correlation set; finally obtain the correlation set and the hierarchical structure set; wherein the correlation set contains all text partitions generated in each division and the number of correlation links between words and phrases in each partition; the hierarchical structure set contains all partitions of the sentence c in the context in each division.

4. The semantic error elimination method based on the knowledge element model according to claim 2, wherein step 3 is specifically: Input the constructed context input text with a hierarchical structure into the context encoder; when the target word to be error-corrected is a single word, add start and end symbols to the input text and then input it into the pre-trained language model encoder; when the target word to be error-corrected is a phrase, use the average vector of each word in the phrase to represent the phrase; wherein j and q represent the first and last words in the span of the phrase, respectively; N represents the number of words contained in the phrase; T c represents the context encoder; [l] represents sequentially putting each word into the context encoder; r ws represents the text feature vector obtained from the input text after processing by the context encoder; In the paraphrase encoder, the annotation data g of the target word s to be error-corrected s is used as input; the paraphrase encoder generates the paraphrase vector g of the target word s to be error-corrected (s) , as follows: g (s) = T g (g s )[0] wherein T g denotes the disambiguation encoder; For each synonym of the target word, a vector score is computed from the structured information of the knowledge element, the paraphrase vector score q s is obtained by adding the vector matrix of the target word to be error-corrected to the weight of the edge between the target word and all synonyms; q s = z s +∑ s'∈V|<s',s>∈E w(<s',s>)·Z s' where <s',s> represents the edge between each target word s and its synonym s'; q s represents the weighted combination of the vector scores of all target words; w(<s',s>) represents the weight of the edge between the target word s to be error-corrected and all its synonyms s'; V represents the set of all vertices in the knowledge element, and E represents all edges; Z s' represents the vector matrix of the synonym s'; z s represents the vector matrix of the target word s to be error-corrected, specifically: z s = h T g (s) + b T g (s) where h T and b T denote the transpose of the annotation vector representation matrix and bias vector, respectively; The paraphrase vector score q s and the knowledge element vector matrix Q are spliced together to obtain a structural feature vector r of the paraphrase encoder output gs .

5. The semantic error elimination method based on the knowledge element model according to claim 1, wherein step 4 is specifically: The graph attention network is divided into two steps for weighting and aggregation: the first step is to calculate the attention coefficient, and the similarity coefficient between the target word to be error-corrected and each neighbor node is calculated; then the attention coefficient is calculated through normalization operation; the second step is to obtain a new feature vector by weighted summation of the attention coefficient, and the weighted aggregation feature vector of the final output of the encoder is obtained by feature enhancement transformation.

6. The semantic error correction method based on the knowledge element model according to claim 5, wherein the specific process of calculating the attention coefficient is: The similarity coefficient between the target word to be error-corrected and each neighbor node is calculated. wherein, r ws represents the text feature vector of the target word s obtained from the context encoder output; r wj represents the text feature vector of the obtained neighbor node j; represents the set of all neighboring nodes of the target word to be error-canceled; Wr ws represents the value obtained by performing linear transformation on r ws ; Wr wj represents the value obtained by performing linear transformation on r wj ; h represents the value obtained by performing linear transformation; [Wr ws ‖Wr wj ] represents the concatenation operation after feature transformation on r ws and r wj ; LeakyReLU represents the activation function.

7. The semantic error correction method based on the knowledge element model according to claim 6, wherein the specific process of obtaining a new feature vector by weighted summation of the attention coefficient is: According to the obtained similarity coefficient α sj , the attention coefficient β sj is calculated through a normalization operation, and the specific calculation process is as follows: wherein, a sk denotes the similarity coefficient between the target word s and all neighboring nodes k; A new feature vector is obtained by weighted sum of the attention coefficients β sj Specifically, the new feature vector is: wherein, denotes the new feature vector output by the representation graph attention network for each word s after fusing neighborhood information; σ(·) denotes an activation function.

8. The semantic error correction method based on the knowledge element model according to claim 7, wherein the specific process of feature enhancement transformation is: the new feature vector obtained by weighted summation the new feature vector obtained by the feature enhancement transformation the weighted aggregation feature vector finally output by the encoder, and the specific process is as follows: wherein K represents the number of heads in the multi-head attention mechanism; W k is the weight matrix of the kth attention head; represents the attention coefficient of the target word s and the neighbor node j in the kth attention head.

9. A knowledge element model based semantic error elimination system, characterized in that, The system comprises: at least one processor; and a memory connected in communication with the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the semantic error correction method based on the knowledge element model according to any one of claims 1-8.

10. A computer storage medium, characterized in that: The computer storage medium stores a semantic error correction method based on the knowledge element model, and the method is executed by at least one processor to realize the semantic error correction method based on the knowledge element model according to any one of claims 1-8.

Images