Synonym judgment method and device based on bayesian network
By calculating synonym probabilities using a Bayesian network model, this method solves the problem of distinguishing between technical terms and proper nouns in scientific and technological literature using pre-trained language models, achieving efficient and resource-saving synonym identification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA UNIONPAY
- Filing Date
- 2022-06-01
- Publication Date
- 2026-04-28
AI Technical Summary
Existing coreference resolution methods rely on pre-trained language models, which are difficult to effectively distinguish between technical terms and proper nouns in scientific and technological literature. They also require large amounts of computing resources and corpora, and the results are poorly interpretable.
A Bayesian network is used for synonym identification. Feature variables are extracted through data preprocessing, and the similarity level is calculated and the probability of synonyms is determined using a Bayesian network model, thereby reducing the need for computing and corpus resources.
It improves the accuracy and interpretability of synonym identification, saves computing and corpus resources, and is suitable for distinguishing professional terms and proper nouns in scientific and technological literature.
Smart Images
Figure CN115526167B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to computer technology, and more specifically to a synonym determination method and device based on Bayesian networks. Background Technology
[0002] In scientific and technological intelligence analysis, it is necessary to extract knowledge entities such as topics, names, and author affiliations from scientific and technological documents. Since synonyms often appear among these entities, manual comparison is required, making graph database construction time-consuming. This manual comparison process is known as coreference resolution, which aims to identify different representations pointing to the same entity. It is an NP-hard problem (NP-hard refers to problems that can be encountered in polynomial time complexity), and it has significant implications for applications such as knowledge graph construction, semantic search, question answering, and recommender systems.
[0003] Existing coreference resolution methods are mainly based on pre-trained language models and consist of four stages: word feature extraction, referent feature extraction, referent identification, and coreference prediction. A crucial stage in these methods is determining the relevance of word pairs using the generated word vectors. The effectiveness of this method depends entirely on the training performance of the pre-trained language model, resulting in significant uncertainty. Specifically, it suffers from the following drawbacks:
[0004] Pre-trained models are the foundation for generating word vectors, and current major pre-trained models (such as BERT and GPT-2) are trained on general domain corpora. However, scientific and technological literature contains a large number of specialized terms from different fields, as well as proper nouns such as names of people and authors' affiliations, which existing pre-trained language models struggle to effectively distinguish.
[0005] Pre-trained models can improve their recognition accuracy in vertical domains by re-pre-training and fine-tuning, but this process requires a lot of computing resources and corpus resources.
[0006] After obtaining word vectors through a pre-trained model, it is possible to determine whether word pairs are synonyms. However, the interpretability of the results is poor, and the direction and methods for optimizing and improving the model performance are unclear. This leads to a great deal of uncertainty in the targeted optimization effect of the pre-trained model in vertical fields. Summary of the Invention
[0007] In view of the above problems, the present invention aims to propose a Bayesian network-based synonym determination method and device that can distinguish proper nouns and save computational and corpus resources.
[0008] The synonym determination method based on Bayesian networks in one aspect of the present invention includes:
[0009] The data preprocessing step involves extracting features from the words to be disambiguated to obtain feature variables;
[0010] The Bayesian network training steps include: calculating the similarity between the words to be disambiguated based on their feature variables; discretizing the calculated similarity to obtain similarity levels; inputting these similarity levels into a pre-trained Bayesian network model; and using the Bayesian network model to obtain the synonym probabilities corresponding to the similarity levels between the words.
[0011] The synonym determination step involves determining synonyms based on the aforementioned synonym probabilities.
[0012] Optionally, the pre-trained Bayesian network model is trained through the following steps:
[0013] Collect keywords;
[0014] Feature extraction is performed on the topic words to obtain the feature variables of each topic word;
[0015] Based on the characteristic variables of the subject terms, the specified similarity between pairs of subject terms with the same characteristics is calculated;
[0016] Discretize the defined similarity of the common features between the topic word pairs to obtain the similarity level between the topic word pairs; and
[0017] To annotate the probability of synonyms between topic word pairs, the similarity level between the topic word pairs and the probability of synonyms between the topic word pairs are used as a training sample set. Based on the training sample set, a prescribed learning algorithm is used to learn the network structure to obtain a Bayesian network model.
[0018] Optionally, the specified similarity is any of the following:
[0019] Cosine similarity, Jaccard similarity, and log-likelihood similarity.
[0020] Optionally, learning a Bayesian network model by using a prescribed learning algorithm on the network structure based on the training sample set includes:
[0021] The network structure is learned using a prescribed learning algorithm based on the training sample set, resulting in a directed acyclic graph of the Bayesian network structure; and
[0022] The parameters of each node are obtained by counting the training samples in the training sample set.
[0023] Optionally, the specified learning algorithm may be a learning algorithm based on dependency statistical analysis or a learning algorithm based on score search.
[0024] Optionally, the feature variables include:
[0025] Individual word features used to represent the characters that make up a word, the order of the characters, and attributes;
[0026] Contextual features used to represent word features surrounding the word to be disambiguated in the text; and
[0027] Text label features are used to represent the structured information of the text containing the disambiguation term.
[0028] Optionally, the contextual features can be extracted using any of the following methods:
[0029] Word frequency, TF-IDF weights, mutual information, and N-Gram.
[0030] Alternatively, the text label features can be extracted using any of the following methods:
[0031] Word frequency, TF-IDF weights, mutual information, and N-Gram.
[0032] Optionally, the specified algorithm can be any of the following algorithms:
[0033] PC algorithm, TPDA algorithm, MDL algorithm and K2 algorithm.
[0034] Optionally, the pre-trained Bayesian network model is trained through the following steps:
[0035] Randomly select a specified number of keywords;
[0036] Calculate the Jaccard similarity between the character vectors of all topic words. For the Jaccard similarity between the character vectors that is greater than a specified threshold, they are selected topic word pairs. For the selected topic word pairs, mark the probability of synonyms between the topic word pairs.
[0037] For the selected keyword pairs, calculate six feature variables: the word vector of the sentence containing each keyword, the word vector of the sentence containing the keyword, the word vector of the author's name where the keyword appears, the word vector of the institution where the keyword appears, the word vector of the classification number where the keyword appears, and the word vector of the publication time of the related papers.
[0038] Calculate the cosine similarity of the six features between the topic word pairs;
[0039] The cosine similarity of the six features is discretized to obtain the similarity level between topic word pairs; and
[0040] The similarity levels between the topic word pairs and the probabilities of synonyms between the labeled topic word pairs are used as training sample sets. Based on the training sample sets, the K2 algorithm is used to learn the Bayesian network structure to obtain a Bayesian network model.
[0041] Optionally, the learning of the Bayesian network structure includes:
[0042] Assuming that there is a dependency between the Jaccard similarity and the cosine similarity of the 6 features, represented by connecting edges, a complete latent graph is constructed to obtain the probability of similarity.
[0043] The fully potential graph is pruned to reduce computational complexity;
[0044] Set the initial variable order and the upper limit value k of the variable parent node, where k is a natural number less than 7;
[0045] For each node, starting from an empty node set, calculate the K2 score of the candidate parent node combination based on the pruned potential graph, and gradually increase the number of parent nodes. The algorithm ends when the number of parent nodes increases but the score does not increase or the number of parent nodes reaches k.
[0046] Based on the above network structure, parameter learning is performed, and the conditional probability distribution table of each node is learned according to the obtained similarity probabilities; and
[0047] By changing the initial variable order, the Bayesian network is trained multiple times. The network structure with the highest accuracy is selected, and the parameters are estimated according to the training sample set.
[0048] A synonym determination device based on Bayesian networks according to one aspect of the present invention includes:
[0049] The data preprocessing module extracts features from the words to be disambiguated and obtains feature variables;
[0050] The Bayesian network training module calculates the similarity between the words to be disambiguated based on their feature variables. It then discretizes the calculated similarity to obtain similarity levels. These similarity levels are input into a pre-trained Bayesian network model, which uses the model to obtain the synonym probabilities corresponding to the similarity levels.
[0051] The synonym determination module determines synonyms based on the synonym probabilities.
[0052] Optionally, the data preprocessing module includes:
[0053] The individual word feature extraction submodule is used to extract features from the input word to extract individual word features that represent the characters that make up the word, the order of the characters, and the attributes.
[0054] The context feature extraction submodule is used to extract features from the input word to extract context features representing the features of words surrounding the word to be disambiguated in the text; and
[0055] The text label feature extraction submodule is used to extract features from the input words to extract text label features that represent the structured information of the text containing the word to be disambiguated.
[0056] Optionally, the Bayesian network training module includes:
[0057] The similarity calculation submodule is used to calculate the similarity between the words to be disambiguated based on the feature variables of the words to be disambiguated.
[0058] A discretization submodule is used to discretize the similarity between the words to be disambiguated calculated by the similarity calculation submodule, thereby obtaining the similarity levels between the words to be disambiguated; and
[0059] The Bayesian computation submodule is used to input the similarity level between the words to be disambiguated into a pre-trained Bayesian network model, and use the Bayesian network model to obtain the synonym probability corresponding to the similarity level between the words to be disambiguated.
[0060] Optionally, the pre-trained Bayesian network model is obtained through training as follows:
[0061] The data preprocessing module performs feature extraction on the collected keywords to obtain feature variables for each keyword; and
[0062] The Bayesian network training module calculates the specified similarity of the same features between topic word pairs based on the feature variables of the topic words. It discretizes the specified similarity of the same features between topic word pairs to obtain the similarity level between topic word pairs. It also labels the probability of synonyms between topic word pairs. The similarity level between topic word pairs and the probability of synonyms between topic word pairs are used as training sample sets. The network structure is learned using a specified learning algorithm based on the training sample set to obtain the Bayesian network model.
[0063] Optionally, the specified similarity is any of the following:
[0064] Cosine similarity, Jaccard similarity, and log-likelihood similarity.
[0065] Optionally, the specified learning algorithm may be a learning algorithm based on dependency statistical analysis or a learning algorithm based on score search.
[0066] Optionally, the context feature extraction submodule extracts context features using any one of word frequency, TF-IDF weights, mutual information, and N-Gram.
[0067] Optionally, the text label feature extraction submodule extracts text label features using any one of the following methods: word frequency, TF-IDF weights, mutual information, and N-Gram.
[0068] Optionally, the specified algorithm can be any of the following algorithms.
[0069] PC algorithm, TPDA algorithm, MDL algorithm and K2 algorithm.
[0070] A computer-readable medium of the present invention stores a computer program thereon, wherein the computer program, when executed by a processor, implements the aforementioned Bayesian network-based synonym determination method.
[0071] A computer device according to one aspect of the present invention includes a storage module, a processor, and a computer program stored on the storage module and executable on the processor, wherein the processor executes the computer program to perform the Bayesian network-based synonym determination. Attached Figure Description
[0072] Figure 1 This is a flowchart illustrating the synonym determination method based on Bayesian networks of the present invention.
[0073] Figure 2 This is a block diagram illustrating the structure of the Bayesian network-based synonym determination device of the present invention. Detailed Implementation
[0074] The following are some embodiments of the present invention, intended to provide a basic understanding of the invention. They are not intended to identify key or decisive elements of the invention or to limit the scope of protection sought.
[0075] For purposes of brevity and illustrative purposes, the principles of the invention are described herein primarily with reference to exemplary embodiments thereof. However, those skilled in the art will readily recognize that the same principles are equivalently applicable to all types of Bayesian network-based synonym determination methods and devices, and that these same principles can be implemented therein, with any such variations not departing from the true spirit and scope of this patent application.
[0076] Furthermore, reference is made in the following description to the accompanying drawings, which illustrate specific exemplary embodiments. Electrical, mechanical, logical, and structural modifications may be made to these embodiments without departing from the spirit and scope of the invention. Moreover, while features of the invention are disclosed in conjunction with only one of several embodiments, this feature may be combined with one or more other features of other embodiments if desired and / or advantageous for any given or identifiable function. Therefore, the following description should not be considered limiting in any sense, and the scope of the invention is defined by the appended claims and their equivalents.
[0077] Terms such as “possessing” and “comprising” indicate that, in addition to having units (modules) and steps that are directly and explicitly stated in the specification and claims, the technical solution of the present invention does not exclude the presence of other units (modules) and steps that are not directly or explicitly stated.
[0078] Before describing the Bayesian network-based synonym determination method and the Bayesian network-based synonym determination device of the present invention, the relevant technical terms will be explained.
[0079] (1) Bayesian network
[0080] Bayesian networks are probabilistic graphical models first proposed by Judea Pearl in 1985. They are models for handling uncertainty in causal relationships during human reasoning, consisting of a directed acyclic graph (DAG) and a conditional probability table (CPT).
[0081] (2) Jaccard similarity
[0082] Jaccard similarity is used to compare the similarity and differences between a finite set of samples. The higher the Jaccard similarity, the greater the sample similarity.
[0083] (3) Cosine similarity
[0084] Cosine similarity, also known as cosine similarity, assesses the similarity between two vectors by calculating the cosine of the angle between them. Cosine similarity plots the vectors in a vector space, such as the most common two-dimensional space, based on their coordinate values.
[0085] (4) TF-IDF
[0086] TF-IDF (term frequency–inverse document frequency) is a commonly used weighting technique for information retrieval and data mining. TF stands for Term Frequency, and IDF stands for Inverse Document Frequency.
[0087] TF-IDF is a statistical method used to assess the importance of a word to a document within a set of documents or a corpus. A word's importance increases proportionally to its frequency in a document, but decreases inversely proportionally to its frequency in the corpus. Various weighted forms of TF-IDF are commonly used by search engines as a measure or ranking of the relevance between documents and user queries.
[0088] (5) N-Gram
[0089] N-grams are a commonly used language model in large-vocabulary continuous speech recognition. In Chinese, we call it the Chinese Language Model (CLM). The CLM utilizes collocation information between adjacent words in context to achieve automatic conversion to Chinese characters.
[0090] (6) K2 algorithm
[0091] The K2 algorithm is a classic algorithm for learning Bayesian network structures. In essence, it is a comprehensive algorithm that combines hill-climbing algorithm and Bayesian scoring algorithm.
[0092] First, the invention of a synonym determination method based on Bayesian networks is explained.
[0093] Figure 1 This is a flowchart illustrating the synonym determination method based on Bayesian networks of the present invention.
[0094] like Figure 1 As shown, the synonym determination method based on Bayesian networks of the present invention includes:
[0095] Data preprocessing step S100: Perform word segmentation on the text containing the words to be disambiguated to obtain the words to be disambiguated, and extract feature variables from the words to be disambiguated;
[0096] Bayesian network training step S200: Calculate the similarity of identical features between different words to be disambiguated based on their feature variables; discretize the calculated similarity to obtain similarity levels; input these similarity levels into a pre-trained Bayesian network model; and use the Bayesian network model to obtain the synonym probabilities corresponding to the similarity levels between the words to be disambiguated.
[0097] Synonym determination step S300: Perform synonym determination based on the synonym probabilities.
[0098] First, the data preprocessing step S100 will be explained.
[0099] In the data preprocessing step S100, the full-text information including the words to be disambiguated is preprocessed. First, the text to be processed is segmented according to dictionary resources, and then different categories of feature variables are extracted. The dictionary resources used here include a professional term dictionary and a stop word dictionary. By using dictionary resources, the segmentation effect can be improved, that is, the professional terms that may be segmented can be identified. In this invention, these different categories of feature variables include: individual word features, context features, and text tag features.
[0100] (1) Features of individual words
[0101] Individual word features refer to features that are independent of context and whose main information comes from the characters that make up the word and their order, as well as the attributes derived from them (such as word length, word glyphs (uppercase and lowercase), word prefixes and suffixes).
[0102] When the word pair to be disambiguated is in English, the first step is to extract lemmas and stems. Lemma reduction and stem extraction methods are used for lemma extraction and stemming, aiming to simplify or reduce the inflectional or derived forms of English words to the basic form of the stem or prototype. Finally, the constituent characters of the word pair to be disambiguated and their order are extracted to form individual word features.
[0103] (2) Contextual features
[0104] Contextual features refer to the word features surrounding the word to be disambiguated in the text. The extraction window for contextual features is at least the sentence containing the word to be disambiguated, and at most the paragraph containing the word to be disambiguated (excluding the word itself).
[0105] The context features consist of keywords within a window. The extraction methods mainly include word frequency, TF-IDF weights, mutual information, N-Gram, etc. The extraction results are keywords and their weights, which form the context word features of the word to be disambiguated.
[0106] Optionally, the text should be preprocessed before extraction, such as removing stop words and punctuation marks.
[0107] Optionally, the average relative position of the keyword and the disambiguation term can be calculated as the keyword weight (the closer the distance, the greater the weight).
[0108] Optionally, when the word to be disambiguated belongs to text label features (such as author name, institution name, etc.), the context features can be directly extracted from the abstract, keywords, or specific parts of the text.
[0109] (3) Text label features
[0110] Text label features refer to the structured information of the text containing the term to be disambiguated, which mainly includes structured features such as author name, author affiliation, co-authors, contact information, publication time, and classification number.
[0111] The extraction methods include word frequency, TF-IDF weight, mutual information, N-Gram, etc. The extraction results are keywords and their weights, which form a text label feature set of the words to be disambiguated.
[0112] Before extraction, the text should be processed as necessary, such as standardizing proper nouns and time dimensions (year, month, etc.).
[0113] Next, the Bayesian network training step S200 will be explained.
[0114] First, the training process of the Bayesian network model will be explained.
[0115] A Bayesian network is a probabilistic graphical model that simulates the uncertainty of causal relationships in human reasoning. It consists of a directed acyclic graph (DAG) and a conditional probability table (CPT).
[0116] In a Bayesian network, if two variables X and Y are directly connected, it means that they have a direct dependency relationship, and information can be transmitted between the two directly connected nodes. If two variables X and Y are not directly connected, then information needs to be transmitted between them through other variables. If all information channels between X and Y are blocked, then information cannot be transmitted between them.
[0117] In this invention, the training process of the Bayesian network model includes the following steps:
[0118] (1) Based on the individual word features, context features and text label features of the words to be disambiguated, the similarity of the same features between different words to be disambiguated is calculated.
[0119] Individual word features can be represented as word vectors, while contextual word features and features in text labels can be represented as word vectors using a bag-of-words model, thereby calculating the cosine similarity of the word pairs to be disambiguated (or Jaccard similarity, log-likelihood similarity, etc. can also be used).
[0120] (2) Discretize the similarity to obtain the similarity level of each feature (for example, equal width method, K-means clustering method or distribution method can be used).
[0121] (3) Construct a training sample set based on the similarity level distribution of each feature variable between each word pair. Specifically, the feature data used for model training is the similarity level of each feature variable between each word pair, and the label corresponding to the feature data used for model training is the probability of synonyms between word pairs. The similarity level of each feature variable between each word pair and the corresponding probability of synonyms between word pairs are used as the training sample set. The training sample set can be obtained by randomly sampling the population multiple times.
[0122] (4) Based on the training sample set, the network structure is learned by using a learning algorithm based on dependency statistical analysis (PC, TPDA, etc.) or a learning algorithm based on rating search (such as MDL, K2, etc.) to obtain the Bayesian network structure DAG. The parameters of each node are obtained by counting the training samples (i.e., parameter estimation is performed for the Bayesian network). Thus, the Bayesian network model is obtained.
[0123] Optionally, the causal relationships between some network nodes (variables) can be determined in advance by domain experts, thereby eliminating meaningless topological structures and reducing the algorithm's search space.
[0124] Using the Bayesian network model obtained in (4), variable elimination is used to infer whether the target word pair is a synonym. Specifically, the input to the Bayesian network model is the similarity level of each variable of the word pair, and the output of the Bayesian network model is the probability of synonyms between the word pairs.
[0125] Finally, in the synonym judgment step S300, the synonyms are judged based on the obtained synonym probabilities.
[0126] As described above, the Bayesian network-based synonym determination method (i.e., coreference resolution method) of the present invention can construct feature variables based on word stems, context, and text labels, calculate the similarity of feature variables between different words, train a Bayesian network model, and finally identify coreference relationships between words using the trained Bayesian network model. This invention expands the available information by integrating text features and text label features, helping to distinguish professional terms, proper nouns such as names and author affiliations in scientific and technological literature. Furthermore, by introducing a Bayesian network for synonym determination, significant computational and corpus resources can be saved, while also exhibiting good interpretability, allowing for manual optimization and parameter tuning of the model, thus facilitating the development of a tool for long-term use.
[0127] The following describes a synonym determination method based on Bayesian networks according to an embodiment of the present invention.
[0128] In this embodiment, the coreference disambiguation task of Chinese paper subject terms is used as an example for illustration.
[0129] First, the training process of the Bayesian network model in this embodiment will be explained.
[0130] The training process of the Bayesian network model in this embodiment includes the following steps:
[0131] (1) Collect Chinese papers according to the requirements and build a database of papers to be analyzed.
[0132] The paper text is segmented using dictionary resources, and then unsupervised keyword extraction methods (such as TF-IDF, TextRank, etc.) are used to extract the subject terms from all papers to form a candidate subject term library.
[0133] The candidate thesaurus is initially filtered by combining rules (such as the keywords must appear in the paper title and the keywords must be the paper's keywords) and expert experience, retaining the keywords that have practical significance.
[0134] (2) Randomly select a certain number of topic words, calculate the Jaccard similarity (frequency weight) S1 between the word vectors of all topic words, and for topic word pairs with similarity greater than a pre-set threshold, such as 60%, experts will label the synonyms in the above topic word pairs. Here, the purpose of expert labeling is to judge the word pairs with a high probability of being synonyms. For example, 60% Jaccard similarity is taken as the synonym probability threshold. Experts will label the word pairs that are confirmed to be synonyms to form a training sample set.
[0135] (3) For each keyword, calculate and obtain the following features: the word vector of the sentence where the keyword is located (TF-IDF weight) S2, the word vector of the sentence where the keyword is located (relative position distance weight) S3, the word vector of the author's name where the keyword appears (frequency weight) S4, the word vector of the institution where the keyword appears (frequency weight) S5, the word vector of the classification number where the keyword appears (frequency weight) S6, and the word vector of the publication time (year) of the related papers (frequency weight) S7.
[0136] (4) Calculate the cosine similarity of the above 6 features among different keywords.
[0137] (5) Discretize the obtained cosine similarity. Specifically, divide the cosine similarity into five levels (very low, low, medium, high, very high) using the equal width method. The probabilities of the above six features can be calculated from the existing samples using the maximum likelihood estimation method to form a training sample set.
[0138] (6) The Bayesian network structure is learned using the K2 algorithm based on the training sample set. First, it is assumed that there are dependencies among the above 7 variables, which are represented by connecting edges, and a complete latent graph is established.
[0139] (7) Prune the fully potential graph according to expert opinions to reduce computational complexity.
[0140] (8) Input the initial variable order and the upper limit value k of the parent node of the variable according to the expert opinion (the default value is 7).
[0141] (9) For each node, starting from the empty node set, calculate the K2 score of the candidate parent node combination based on the pruned potential graph, and gradually increase the number of parent nodes. When the number of parent nodes is increased but the score value does not increase or the number of parent nodes reaches 7, the algorithm ends.
[0142] (10) Based on the above network structure, perform parameter learning and learn the conditional probability distribution table of each node according to the probability obtained in (5).
[0143] (11) Change the order of initial variables according to expert opinions, train the Bayesian network multiple times, select the network structure with the highest accuracy, and estimate the parameters according to the training sample set.
[0144] Next, the application of the Bayesian network model constructed through the above steps will be explained.
[0145] The Bayesian network model trained through the above steps can be used to determine whether new paper topic terms are synonyms. This mainly includes the following steps:
[0146] (1) Extract the sentences in which the topic words appear, segment them and calculate the TF-IDF weights to obtain the word vector S2 of the sentence in which the topic words appear, then remove stop words and calculate the relative position distance weights to obtain the word vector S3 of the sentence in which the topic words appear.
[0147] (2) Extract the structural features of the text in which the keywords appear, calculate the frequency of the structural information, and obtain the word vectors of author name S4, institution S5, classification number S6, and paper publication year S7.
[0148] (3) Calculate the cosine similarity of the six features (S2~S7) between the topic word pairs of the paper and discretize them into five levels {very low, low, medium, high, very high}.
[0149] (4) As input to the Bayesian network model, the similarity level of the above 6 features of the topic word pair of the paper is input. As output to the Bayesian network model, the probability of whether the topic word pair of the paper is a synonym is output. That is, the probability of belonging to each result in the discrimination result set {synonym, non-synonym} is output. Based on the pre-set threshold (e.g., 75%), it can be determined whether the topic word pair to be disambiguated is a synonym (i.e., there is a coreference relationship).
[0150] Secondly, a variation of the synonym determination method based on Bayesian networks of the present invention will be described.
[0151] The generation of word vectors and similarity calculation can be modified as follows:
[0152] 1) As a method for calculating word vector weights, TF-IDF weights can be replaced by common word weights such as word frequency, mutual information, and N-Gram.
[0153] 2) As a method for calculating word vector similarity, cosine similarity can also be replaced by common methods such as adjusted cosine similarity, Euclidean distance, and Hamming distance.
[0154] The following variations can be performed during Bayesian network training, including:
[0155] 1) As a Bayesian network structure learning method, the K2 score search method can be replaced by BD score, MDL score, BIC score, AIC score, etc.
[0156] 2) Bayesian networks can also be trained multiple times through multiple random sampling based on the incremental Bayes formula, thereby continuously improving the robustness of the model.
[0157] Next, the Bayesian network-based synonym determination device of the present invention will be described.
[0158] Figure 2This is a block diagram illustrating the structure of the Bayesian network-based synonym determination device of the present invention.
[0159] like Figure 2 As shown, the Bayesian network-based synonym determination device of the present invention includes:
[0160] The data preprocessing module 100 receives the training dataset and the words to be disambiguated, performs feature extraction on the topic words and the words to be disambiguated in the training dataset, and obtains feature variables.
[0161] The Bayesian network training module 200 calculates the similarity between the words to be disambiguated based on their feature variables. It then discretizes the calculated similarity to obtain similarity levels. These similarity levels are input into a pre-trained Bayesian network model, which uses the model to obtain the synonym probabilities corresponding to the similarity levels.
[0162] The synonym judgment module 300 performs synonym judgment based on the synonym probability.
[0163] The data preprocessing module 100 includes:
[0164] The single-word feature extraction submodule 110 is used to extract features from the input word to extract single-word features representing the characters that make up the word, the order of the characters, and the attributes.
[0165] The context feature extraction submodule 120 is used to extract features from the input word to extract context features representing the features of words surrounding the word to be disambiguated in the text; and
[0166] The text label feature extraction submodule 130 is used to extract features from the input words to extract text label features that represent the structured information of the text containing the word to be disambiguated.
[0167] The Bayesian network training module 200 includes:
[0168] The similarity calculation submodule 210 is used to calculate the similarity between the words to be disambiguated based on the feature variables of the words to be disambiguated.
[0169] Discretization submodule 220 is used to discretize the similarity between the words to be disambiguated calculated by the similarity calculation submodule to obtain the similarity level between the words to be disambiguated; and
[0170] The Bayesian computation submodule 230 is used to input the similarity level between the words to be disambiguated into a pre-trained Bayesian network model, and use the Bayesian network model to obtain the synonym probability corresponding to the similarity level between the words to be disambiguated.
[0171] The Bayesian network model is obtained through the following training:
[0172] The data preprocessing module 100 performs feature extraction on the collected keywords to obtain the feature variables of each keyword; and
[0173] The Bayesian network training module 200 calculates the prescribed similarity of the same features between topic word pairs based on the feature variables of the topic words, discretizes the prescribed similarity of the same features between topic word pairs to obtain the similarity level between topic word pairs, and labels the probability of synonyms between topic word pairs. The similarity level between topic word pairs and the probability of synonyms between topic word pairs are used as training sample set. The network structure is learned using a prescribed learning algorithm based on the training sample set to obtain the Bayesian network model.
[0174] The specified similarity can be any of the following: cosine similarity, Jaccard similarity, or log-likelihood similarity.
[0175] The specified learning algorithm is either a learning algorithm based on dependency statistical analysis or a learning algorithm based on score search.
[0176] The context feature extraction submodule 120 extracts context features using any one of word frequency, TF-IDF weights, mutual information, and N-Gram. The text label feature extraction submodule 130 extracts text label features using any one of word frequency, TF-IDF weights, mutual information, and N-Gram.
[0177] The specified algorithm is any one of the following: PC algorithm, TPDA algorithm, MDL algorithm, and K2 algorithm.
[0178] The Bayesian network-based synonym identification method and device of this invention can construct feature variables based on word stems, context, and text labels, calculate the similarity of feature variables between different words, train a Bayesian network model, and finally identify the core reference relationships between words using the trained model. This integrates text features and text label features, expanding the available information and helping to distinguish proper nouns such as names and author affiliations in scientific and technological literature. By using Bayesian networks for synonym identification, computer and corpus resources can be saved, and the accuracy and interpretability of the identification can be improved.
[0179] The present invention also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned Bayesian network-based synonym determination method.
[0180] The present invention also provides a computer device, including a storage module, a processor, and a computer program stored on the storage module and executable on the processor, wherein the processor executes the computer program to perform the Bayesian network-based synonym determination.
[0181] The above examples primarily illustrate the Bayesian network-based synonym determination method and the Bayesian network-based synonym determination device of the present invention. Although only some specific embodiments of the present invention have been described, those skilled in the art should understand that the present invention can be implemented in many other forms without departing from its spirit and scope. Therefore, the examples and embodiments shown are considered illustrative rather than restrictive, and the present invention may encompass various modifications and substitutions without departing from the spirit and scope of the invention as defined in the appended claims.
Claims
1. A synonym determination method based on Bayesian networks, characterized in that, include: The data preprocessing step involves extracting features from the words to be disambiguated to obtain feature variables; The Bayesian network training steps include: calculating the similarity between the words to be disambiguated based on their feature variables; discretizing the calculated similarity to obtain similarity levels; inputting these similarity levels into a pre-trained Bayesian network model; and using the Bayesian network model to obtain the synonym probabilities corresponding to the similarity levels between the words. The synonym determination step involves determining synonyms based on the aforementioned synonym probabilities. The pre-trained Bayesian network model is obtained through the following steps: Randomly select a specified number of keywords; Calculate the Jaccard similarity between the character vectors of all topic words. For the Jaccard similarity between the character vectors that is greater than a specified threshold, they are selected topic word pairs. For the selected topic word pairs, mark the probability of synonyms between the topic word pairs. For the selected keyword pairs, the following six feature variables are obtained: TF-IDF weights are calculated to obtain the sentence word vectors of each keyword, stop words are removed and relative position distance weights are calculated to obtain the sentence word vectors of the keyword, the author's name of the keyword, the institution of the keyword, the classification number of the keyword, and the publication time of the related papers. Calculate the cosine similarity of the six features between the topic word pairs; The cosine similarity of the six features is discretized to obtain the similarity level between topic word pairs; and The similarity levels between the topic word pairs and the probabilities of synonyms between the labeled topic word pairs are used as the training sample set. Based on the training sample set, the K2 algorithm is used to learn the Bayesian network structure to obtain the Bayesian network model. The learning of the Bayesian network structure includes: Assuming that there is a dependency between the Jaccard similarity and the cosine similarity of the 6 features, represented by connecting edges, a complete latent graph is constructed to obtain the probability of similarity. The fully potential graph is pruned to reduce computational complexity; Set the initial variable order and the upper limit value k of the variable parent node, where k is a natural number less than 7; For each node, starting from an empty node set, calculate the K2 score of the candidate parent node combination based on the pruned potential graph, and gradually increase the number of parent nodes. The algorithm ends when the number of parent nodes increases but the score does not increase or the number of parent nodes reaches k. Based on the above network structure, parameter learning is performed, and the conditional probability distribution table of each node is learned according to the obtained similarity probability; and By changing the initial variable order, the Bayesian network is trained multiple times. The network structure with the highest accuracy is selected, and the parameters are estimated based on the training sample set. The feature variables include: Individual word features used to represent the characters that make up a word, the order of the characters, and attributes; Contextual features used to represent word features surrounding the word to be disambiguated in the text; and Text label features are used to represent the structured information of the text containing the disambiguation term.
2. The synonym determination method based on Bayesian networks as described in claim 1, characterized in that, Extract the contextual features using any of the following methods: Word frequency, TF-IDF weights, mutual information, and N-Gram.
3. The synonym determination method based on Bayesian networks as described in claim 1, characterized in that, Extract the text label features using any of the following methods: Word frequency, TF-IDF weights, mutual information, and N-Gram.
4. A synonym determination device based on Bayesian networks, characterized in that, include: The data preprocessing module extracts features from the words to be disambiguated and obtains feature variables; The Bayesian network training module calculates the similarity between the words to be disambiguated based on their feature variables. It then discretizes the calculated similarity to obtain similarity levels. These similarity levels are input into a pre-trained Bayesian network model, which uses the model to obtain the synonym probabilities corresponding to the similarity levels. The synonym determination module performs synonym determination based on the synonym probabilities. The pre-trained Bayesian network model is obtained through the following steps: Randomly select a specified number of keywords; Calculate the Jaccard similarity between the character vectors of all topic words. For the Jaccard similarity between the character vectors that is greater than a specified threshold, they are selected topic word pairs. For the selected topic word pairs, mark the probability of synonyms between the topic word pairs. For the selected keyword pairs, the following six feature variables are obtained: TF-IDF weights are calculated to obtain the sentence word vectors of each keyword, stop words are removed and relative position distance weights are calculated to obtain the sentence word vectors of the keyword, the author's name of the keyword, the institution of the keyword, the classification number of the keyword, and the publication time of the related papers. Calculate the cosine similarity of the six features between the topic word pairs; The cosine similarity of the six features is discretized to obtain the similarity level between topic word pairs; and The similarity levels between the topic word pairs and the probabilities of synonyms between the labeled topic word pairs are used as the training sample set. Based on the training sample set, the K2 algorithm is used to learn the Bayesian network structure to obtain the Bayesian network model. The learning of the Bayesian network structure includes: Assuming that there is a dependency between the Jaccard similarity and the cosine similarity of the 6 features, represented by connecting edges, a complete latent graph is constructed to obtain the probability of similarity. The fully potential graph is pruned to reduce computational complexity; Set the initial variable order and the upper limit value k of the variable parent node, where k is a natural number less than 7; For each node, starting from an empty node set, calculate the K2 score of the candidate parent node combination based on the pruned potential graph, and gradually increase the number of parent nodes. The algorithm ends when the number of parent nodes increases but the score does not increase or the number of parent nodes reaches k. Based on the above network structure, parameter learning is performed, and the conditional probability distribution table of each node is learned according to the obtained similarity probability; and By changing the initial variable order, the Bayesian network is trained multiple times. The network structure with the highest accuracy is selected, and the parameters are estimated based on the training sample set. The data preprocessing module includes: The individual word feature extraction submodule is used to extract features from the input word to extract individual word features that represent the characters that make up the word, the order of the characters, and the attributes. The context feature extraction submodule is used to extract features from the input word to extract context features representing the features of words surrounding the word to be disambiguated in the text; and The text label feature extraction submodule is used to extract features from the input words to extract text label features that represent the structured information of the text containing the word to be disambiguated.
5. The synonym determination device based on Bayesian networks as described in claim 4, characterized in that, The context feature extraction submodule extracts context features using any one of the following methods: word frequency, TF-IDF weights, mutual information, and N-Gram.
6. The synonym determination device based on Bayesian networks as described in claim 4, characterized in that, The text label feature extraction submodule extracts text label features using any one of the following methods: word frequency, TF-IDF weights, mutual information, and N-Gram.
7. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the synonym determination method based on Bayesian networks as described in any one of claims 1 to 3.
8. A computer device, comprising a storage module, a processor, and a computer program stored on the storage module and executable on the processor, characterized in that, When the processor executes the computer program, it implements the synonym determination method based on Bayesian networks as described in any one of claims 1 to 3.