Word semantic similarity measurement method and device integrating SAO and Bayesian model

Abstract

The invention provides a word semantic similarity measurement method and device integrating SAO and a Bayesian model, and relates to the technical field of word semantic similarity calculation, the method is an innovative word semantic similarity measurement method, and SAO (subject-action-object) structure extraction and the Bayesian model are integrated; firstly, SAO structures are extracted from a text, and then the occurrence frequency of similar words is counted based on the structures and serves as a measure of word similarity. Meanwhile, semantic similarity parameters of the words are obtained by utilizing a semantic knowledge base such as WordNet and the like, the parameters and statistical results are integrated through a Bayesian model, and the semantic similarity posterior probability of the words is calculated. The method is characterized in that the advantages of a method based on statistics are utilized, a method based on knowledge is combined, and a new perspective is provided to understand and quantify the similarity between words.

Description

Technical Field

[0001] The present invention relates to the technical field of word semantic similarity calculation, and in particular to a word semantic similarity measurement method and device integrating SAO and Bayesian model. Background Art

[0002] In the digital age, with the development of artificial intelligence and natural language processing technology, accurate measurement of semantic similarity of words in text has become particularly important. This measurement has a far-reaching impact on a variety of intelligent applications, including but not limited to text classification, knowledge retrieval, service matching and many other fields, and is a key link. The core challenge of measuring semantic similarity of words is how to accurately capture and quantify the subtle differences and connections between words, which is crucial to improving the performance of natural language processing (NLP) systems.

[0003] At present, in this field, although researchers have proposed a variety of methods to solve the problem of measuring word semantic similarity, they each have their own advantages and limitations. Specifically, knowledge-based methods rely on expert experience and predefined semantic relationships, which may lead to strong subjectivity of the results and difficulty in adapting to new fields. Corpus-based methods rely on large-scale text data. Although the data is rich, they face the problems of data sparsity and noise interference, which affects the accuracy of the measurement results. The word vector method simplifies the calculation process by converting words into points in the vector space, but this method may not fully capture the deep semantic information of words. In addition, although multi-method fusion can combine the advantages of various technologies, it has high computational complexity and needs to solve the compatibility problem between different models. In short, although these methods can provide useful similarity measurement results in specific cases, they still have limitations in practical applications. For example, knowledge-based methods may not cover all semantic relationships, and corpus-based methods may be limited by the quality and scope of available data. In addition, although word vector-based methods can provide fast similarity measurement in some cases, they may not accurately reflect the complex semantic relationships between words.

[0004] Therefore, although a variety of methods have been proposed to solve the problem of measuring word semantic similarity, this field still needs new methods and technologies to improve the accuracy and reliability of the measurement. These new methods need to be able to overcome the limitations of existing technologies and provide more accurate and comprehensive semantic similarity measurement results to meet the growing needs of NLP applications.

[0005] In view of this, this application is filed. Summary of the invention

[0006] The present invention provides a word semantic similarity measurement method and device integrating SAO and Bayesian model, which can at least partially improve the above problems.

[0007] To achieve the above object, the present invention adopts the following technical solutions: A word semantic similarity measurement method integrating SAO and Bayesian model, comprising: Based on semantic part-of-speech tagging, the SAO structure of the words to be compared is extracted from the preset corpus, and the SAO structure is statistically preprocessed to obtain statistical results; According to the statistical results, the SAO similarity of the words is calculated to obtain a SAO similarity score; Calculating the semantic similarity experience similarity between the comparison word and the word to be compared according to a preset knowledge base to generate the experience similarity, wherein the knowledge base is a wordnet knowledge base; Substitute the SAO similarity score and the empirical similarity into the Bayesian formula to obtain the final calculation result of the word semantic similarity.

[0008] The present invention also provides a word semantic similarity measurement device integrating SAO and Bayesian model, which comprises: An extraction statistics unit is used to extract the SAO structure of the words to be compared from a preset corpus based on semantic part-of-speech tagging, and perform statistical preprocessing on the SAO structure to obtain statistical results; An SAO similarity unit, used to calculate the SAO similarity of the words according to the statistical results to obtain a SAO similarity score; An experience similarity unit, used for calculating the semantic similarity experience similarity between the comparison word and the word to be compared according to a preset knowledge base, and generating experience similarity, wherein the knowledge base is a wordnet knowledge base; The Bayesian calculation unit is used to substitute the SAO similarity score and the empirical similarity into the Bayesian formula to obtain the final word semantic similarity calculation result.

[0009] In summary, the word semantic similarity measurement method integrating SAO and Bayesian model proposes a word semantic similarity measurement model integrating SAO and Bayesian model. This method uses word SAO similarity as the basis for determining word similarity, and combines the knowledge base to obtain semantic consistency parameters, which can simultaneously combine experience-based and statistical-based word semantic similarity calculation methods; and proposes a new similarity measurement based on the Bayesian model. Specifically, the SAO co-occurrence statistics are used as the SAO similarity probability, the knowledge base semantic consistency parameters are integrated, and the posterior probability of word similarity is calculated. Through this study, a new idea is provided for the judgment of word semantic similarity, the accuracy of similarity judgment is improved, and a reference is provided for solving the problem of similarity calculation. In short, the word semantic similarity measurement method integrating SAO and Bayesian model can combine the corpus-based statistical method and the knowledge base-based method in the hope of obtaining a more comprehensive and accurate similarity measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flow chart of a method for measuring word semantic similarity by integrating SAO and Bayesian model provided by the first embodiment of the present invention; Figure 2 It is a schematic diagram of a model flow of a word semantic similarity measurement method integrating SAO and Bayesian model provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of the wordnet structure provided by an embodiment of the present invention; Figure 4 Schematic diagram of a Bayesian model of word semantic similarity provided by an embodiment of the present invention; Figure 5 is a schematic diagram of node score distribution provided by an embodiment of the present invention; Figure 6 is a schematic diagram of a curve fitting condition provided by an embodiment of the present invention; Figure 7 The different methods provided in the embodiments of the present invention are used to calculate and compare the results; Figure 8 It is a module schematic diagram of a device for measuring word semantic similarity integrating SAO and Bayesian model provided by the second embodiment of the present invention. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0012] refer to Figure 1As shown, the first embodiment of the present invention discloses a word semantic similarity measurement method integrating SAO and Bayesian model, which can be executed by a word semantic similarity measurement device integrating SAO and Bayesian model (hereinafter referred to as measurement device), and in particular, executed by one or more processors in the measurement device to implement the following method: The word semantic similarity measurement method integrating SAO and Bayesian model is an innovative word semantic similarity measurement method, wherein semantic similarity is to measure the degree of interchangeability of a word in the context of another word, and the semantic relationship between words is mainly divided into: 1. hyponymy, 2. synonymy, 3. antonymy, 4. inclusion. In general, synonymy, hyponymy and inclusion will obtain higher semantic similarity scores, and antonymy will have lower scores. However, in special cases, the semantic similarity of antonym word pairs will have special values, and this situation will be analyzed based on cases in this embodiment.

[0013] The word semantic similarity measurement method integrating SAO and Bayesian model provides a word semantic similarity calculation model integrating SAO and naive Bayesian model, and the model is mainly divided into three parts: the first part uses semantic annotation to extract keyword SAO structure from the corpus, and the SAO structure similarity of the word pairs to be compared is statistically analyzed to obtain the SAO similarity score; the second part obtains the semantic similarity empirical similarity of the word pairs to be compared based on the knowledge base; the third part integrates the SAO similarity score and empirical similarity into the Bayesian model to obtain the final semantic similarity score. The specific implementation steps and the Bayesian model construction process of word semantic similarity are as follows: Figure 2 shown.

[0014] Specific: S1, extracting the SAO structure of the words to be compared from the preset corpus based on semantic part-of-speech tagging, and performing statistical preprocessing on the SAO structure to obtain statistical results; Specifically, step S1 includes: performing sentence processing on the text in the preset corpus to obtain multiple sentences, and performing semantic part-of-speech tagging on the words in each sentence; Perform sentence SAO extraction processing according to the marked word parts of speech, extract the SAO structures of the two words to be compared, perform word form merging processing on the extracted SAO structures, and convert other word forms of the words to be compared into prototypes; The occurrence frequencies of the SAO structures corresponding to the two words to be compared in the text are counted respectively, and compared to obtain the SAO structures that appear in the two words to be compared and their occurrence frequencies, and generate statistical results.

[0015] In this embodiment, the text in the corpus is divided into sentences, and then the semantic part-of-speech tagging is continued for the words in the sentences, and the sentence SAO is extracted according to the word part-of-speech. The SAO structures of the two words to be compared are taken. Since some words have multiple word forms during the extraction process, such as past tense, present tense, future tense or plural form, it is necessary to merge the word forms of the obtained SAO; for the convenience of statistics and understanding, it is also necessary to convert other word forms of the words into prototypes.

[0016] The frequency of occurrence of the SAO corresponding to the words to be compared in the text is counted respectively, and the SAO structures of the two groups of words to be compared are compared to obtain the SAO structures and their frequencies that co-occur in the two groups. When counting the co-occurring predicates, if the frequency of the common predicate is too high and the influence of other predicates is too small, the common predicate is removed.

[0017] S2, performing SAO similarity calculation on the words according to the statistical results to obtain a SAO similarity score; Specifically, step S2 includes: based on the statistical results, when it is determined that the comparison word and the word to be compared are subjects, the corresponding word SAO similarity is calculated according to the predicate, object, and frequency that co-occur between the comparison word and the word to be compared; When it is determined that the comparison word and the word to be compared are predicates, the SAO similarity of the corresponding words is calculated according to the subjects, objects and their frequencies that appear together between the comparison word and the word to be compared; When it is determined that the comparison word and the word to be compared are objects, the SAO similarity of the corresponding words is calculated based on the subjects, predicates, and frequencies that appear together between the comparison word and the word to be compared; When it is determined that there is no co-occurring SAO structure between the comparison word and the word to be compared, the corresponding SAO similarity is 0, and the Laplace smoothing algorithm is used to adjust the word SAO similarity.

[0018] Preferably, the calculation formula for calculating the word SAO similarity is: , , ,in, is the word-subject SAO similarity, is the word-predicate SAO similarity, is the word object SAO similarity, is the probability of co-occurrence of subject of word w1 and word w2, is the probability of word w2 co-occurring with the subject of word w1, is the sum of the probabilities of subject co-occurrence between words, is the probability of word w1 and word w2 co-occurring as predicates, is the probability of word w2 co-occurring with word w1 as a predicate, is the sum of the probabilities of predicate co-occurrence between words, is the probability of co-occurrence of word w1 and word w2 object, is the probability of word w2 co-occurring with the object of word w1, is the sum of the probabilities of object co-occurrence between words.

[0019] It should be noted that due to the different calculation methods, and , and , and , are not always equal.

[0020] In this embodiment, the SAO structure refers to a triple extracted from a text sentence, in which the subject and the object appear as the subject, representing the executor and the executor in the event description; the predicate, as the action between the subject and the object, links the two together. As a structure based on part of speech and semantic acquisition, the SAO structure can explain the logical relationship between words and avoid confusion of word features. Specifically, the advantages of using SAO as a word information feature include: avoiding interference from other irrelevant words in the context, and being able to accurately describe the associated information of the words; most importantly, reducing the workload of statistics and calculations, and improving the accuracy of the semantic similarity algorithm.

[0021] In this embodiment, in the word semantic similarity measurement method integrating SAO and Bayesian model, the word SAO similarity is determined by the ratio of the SAO of words with less information contained in the SAO of words with more information.

[0022] Based on the calculation formula of word SAO similarity calculation, assuming that word 1 contains i groups of SAO, and word 2 contains j groups of SAO, , , , and its calculation formula is: , , , , , , , , ,in, Indicates the co-occurring subject of word 1 and word 2, Equally so; Indicates the subject of word 1, The same goes for this.

[0023] S3, calculating the semantic similarity experience similarity between the comparison word and the word to be compared according to a preset knowledge base to generate the experience similarity, wherein the knowledge base is a wordnet knowledge base; Specifically, step S3 includes: calculating the path distance similarity experience score wu_simlarity based on the path and the semantic similarity experience score lin_simarity based on the concept content between the comparison word and the word to be compared according to the preset knowledge base, so as to determine the highest score between the comparison word and the word to be compared; Calculate the average of the path distance similarity experience score wu_simlarity and the semantic similarity experience score lin_simarity to get the experience similarity. The calculation formula is: ,in, is the semantic similarity experience score calculated based on the path distance similarity experience score wu_similarity, is the semantic similarity experience score calculated based on the semantic similarity experience score lin_similarity; When it is determined that the scores of the path distance similarity experience score wu_simlarity and the semantic similarity experience score lin_simarity are both 1, the dissimilar experience similarity is 0, and the Laplace smoothing algorithm is used to adjust the experience similarity.

[0024] Preferably, the calculation formula of the Laplace smoothing algorithm is: , ,in, is the smoothing parameter, is the observation result of the j-th characteristic result of random variable a with value l, K is the number of characteristic types, is the number of feature classifications, N is the number of statistical samples, is an indicator function, indicating that in the i-th sample, the j-th feature The value is equal to , and category Y is equal to When , the function value is 1, otherwise it is 0. is the kth category, is the i-th category, is the jth feature in the i-th sample.

[0025] In this embodiment, the is an indicator function, meaning that in the i-th sample, the j-th feature The value is equal to And category Y is equal to When , the function value is 1, otherwise it is 0. Sum this indicator function over all samples and we get Next, Features The value is The number of samples. It is also a sum indicator function, indicating that the category Y is equal to Then add ,in is the jth feature This is also part of the Laplace smoothing, which is used to adjust the denominator to keep the probability smooth.

[0026] Preferably, the calculation formula of the path distance similarity empirical score wu_simlarity is: ,in, is the highest common parent node of word w1 and word w2 in the knowledge base, is the shortest path length from word w1 to word w2, is the depth of the word in the knowledge base hierarchy tree.

[0027] Preferably, the calculation formula of the semantic similarity empirical score lin_simarity is: ,in, is the probability that the information content of the nearest common parent node of word w1 and word w2 accounts for the total description information content, is the probability that the information content of word w1 accounts for the total information content, is the probability that the amount of information in word w2 accounts for the total amount of information, is the information content of the highest common parent node of word w1 and word w2 in the knowledge base, is the information content of word w1, is the information content of word w2.

[0028] In this embodiment, wordnet, which has more comprehensive information and more accurate calculation results, is used as the knowledge base. By calculating the wu_simlarity (hereinafter referred to as wu) based on the path of the comparison word and the word to be compared and the lin_simarity (hereinafter referred to as lin) based on the concept content, the highest score of the two is obtained. Further, the average of wu and lin is calculated to obtain the empirical similarity. It should be noted that since the empirical similarity is to be put into the Bayesian model, when the scores of wu and lin are both 1, the dissimilar empirical similarity is judged to be 0, which will cause the final empirical score to lose its effect. To solve this problem, this method uses Laplace smoothing to adjust the empirical similarity.

[0029] For details, please refer to Figure 3Wordnet is an English semantic knowledge base widely used in the field of NLP. It forms a semantic network based on the semantic relationship between words. The semantic relationship between words is expressed through links, which makes wordnet have a good conceptual hierarchy and tree structure. Taking the hyponym set of the noun channel as an example, the semantic relationship between words in Wordnet is as follows: Figure 3 As shown. Among them, Figure 3 The node at the upper level is the parent node of the node at the lower level, usually indicating a hierarchical relationship; nodes with the same parent node are sibling nodes, usually indicating a near-synonymous or synonymous relationship; for each definition, there is a word set definition that expresses the specific meaning of the word set.

[0030] Specifically, in this embodiment, the semantic similarity calculation model derived from wordnet is mainly based on three considerations: based on the path distance between word pairs, based on the amount of information (IC), and based on attribute characteristics. Due to the good network performance of wordnet, this method mainly focuses on the number of connecting edges between two concepts. The number of connecting edges is inversely proportional to the semantic similarity. The more connecting edges, the lower the semantic similarity; the number of connecting edges is directly proportional to the semantic distance. The more connecting edges, the greater the semantic distance. The word semantic similarity measurement method integrating SAO and Bayesian model selects Wu_simlarity as the path distance similarity empirical score. The calculation formula of Wu_simlarity is: .by Figure 3 Take the schematic diagram in as an example, here is "channel", yes Arrived via (line, channel) Length, is the depth of the channel in the level. Among them, the path distance similarity empirical score Wu_simlarity believes that the lower the level of the concept, the smaller the similarity; when calculating, not only the concept path is considered, but also the depth of the highest common parent node in the classification tree. Therefore, the accuracy is better than other algorithms, and the calculation complexity is low.

[0031] In addition, Wordnet not only has good network performance, but also contains rich conceptual information; the semantic similarity calculation method based on IC mainly focuses on the information content of the concept. The word semantic similarity measurement method integrating SAO and Bayesian model uses Lin_simlarity method as the semantic similarity empirical score based on IC. The calculation formula of Lin_simlarity is: .by Figure 3 Take the schematic diagram as an example. The nearest common parent node of and contact is "line", then the Lin_simlarity between the two is the ratio of twice the amount of information of "line" to the sum of the amount of information of the two.

[0032] Finally, based on the results of the above two empirical scoring algorithms, taking words 1 and 2 as examples, the empirical similarity calculation formula of words based on the knowledge base is obtained: .

[0033] In this embodiment, since the calculation of semantic similarity in this paper is based on the Bayesian model, zero-probability events will lead to extreme conditions (0 or 1) in the final calculation result. To avoid such extreme conditions, Laplace smoothing technology is used to adjust zero-probability events. , which is usually set to 1, which ensures that the numerator will not be 0.

[0034] Specifically, in the word semantic similarity measurement method integrating SAO and Bayesian model, there are two cases where the Laplace smoothing algorithm needs to be introduced, namely, the case where the score is 1 when calculating the empirical similarity, and the case where there is no co-occurrence SAO when calculating the SAO similarity.

[0035] When calculating the experience score, if the words are under the same parent node, the experience score of the word similarity is 1, and the experience score of the dissimilar words is 0. When the experience score is introduced into the Bayesian model, the semantic similarity calculation is completely dependent on the SAO similarity, which obviously violates the original intention of this method. Therefore, Laplace smoothing is used for the case where the semantic similarity experience score is 1; the method of use is: the experience score is divided into two categories, namely =2, assuming the number of statistical samples N=10, =1, at this time, the number of statistically similar samples is adjusted to: similarity equals 10+1, and dissimilarity equals 1. , ,After Laplace adjustment, the situation that the experience score is 0 and becomes ineffective is avoided, which is beneficial to ensure the accuracy of the algorithm.

[0036] In the statistical co-occurrence SAO, especially when the words are antonyms or different domains, there is often no co-occurrence SAO. , and Once it is 0, the result of the entire Bayesian model is also 0, causing the algorithm to fail. To avoid this situation, Laplace smoothing can be used to adjust the SAO co-occurrence statistics. Based on the original statistical results, the statistical results are divided into two categories, namely co-occurrence and non-co-occurrence. =2, let =1, at this time, , , where T represents the total number of statistical samples and B represents the total number of samples that do not contribute to the statistics. After the above adjustment, the situation where the probability of statistical co-occurrence SAO is 0 is avoided, which removes obstacles for the Bayesian model to calculate similarity and is conducive to improving the accuracy of the algorithm.

[0037] S4, substitute the SAO similarity score and the empirical similarity into the Bayesian formula to obtain the final word semantic similarity calculation result.

[0038] Specifically, step S1 includes: using the SAO similarity score as the probability of judging similarity between word pairs, and using the empirical similarity as a regulation parameter; The Bayesian formula is used to calculate the SAO similarity score and the empirical similarity to obtain the final word semantic similarity calculation result. The calculation formula is: , .

[0039] In this embodiment, according to the Bayesian formula, the SAO similarity score of a word pair is used as the probability of judging similarity between word pairs, and the probability of judging dissimilarity is "1-SAO similarity score". At the same time, considering that statistical words and similarity methods are inevitably biased, the empirical similarity between words based on the corpus and the similarity method is used as an adjustment parameter. Finally, the above similarity score and adjustment parameter are substituted into the Bayesian formula to obtain the calculation result of the semantic similarity of words calculated by the Bayesian model.

[0040] In this embodiment, the SAO statistical model is used as the SAO similarity probability of the Bayesian model, and the similarity experience score based on the knowledge base is used as the semantic consistency parameter to finally obtain the similarity calculation result of the words. Among them, the Bayesian model is as follows Figure 4 As shown. Assuming that the marginal distribution is P(Y), then P(Ssim|Y), P(Asim|Y), and P(Osim|Y) represent the probability distribution of determining the semantic similarity between word pairs. Setting the prior probabilities P(Y=T) and P(Y=F) to 0.5, the posterior probabilities of determining word similarity based on SAO similarity are P(Y =T| Ssim), P(Y =T| Asim), and P(Y =T| Osim), and the posterior probabilities of determining word dissimilarity are P(Y =F| Ssim), P(Y =F| Asim), and P(Y =F| Osim). Assuming that word 1 and word 2 are subjects, then by the attribute Asim( ) and the attribute Osim( ) The calculation formula of semantic similarity sim(w1,w2) between word 1 and word 2 under joint constraint is as follows: shown.

[0041] From the conditional probability formula we can get: , . From the joint probability formula, we can get: ,for ,because and Do not affect each other, so .

[0042] Similarly, , Combining P(Y=T)=P(Y=F)=0.5 with the above formula, we get: , In addition, if word 1 and word 2 are predicates, they are represented by the attribute Ssim( ) and the attribute Osim( ) joint constraint to calculate semantic similarity; if word 1 and word 2 are objects, then the attributes Ssim( ) and the attribute Asim( )Combined constraints to calculate semantic similarity.

[0043] Specifically, in this embodiment, bathroom texts are selected as corpora for experimental comparison, wherein the text structure comparison is shown in Table 1.

[0044] Table 1 Text structure comparison

[0045] Based on the existing bathroom overview text, the sentence SAO structure is extracted to form a database. Since the bathroom overview text SAO database contains more than 1.7 million SAO data, the probability of accidental events is reduced and the credibility of the database is improved. Some SAO structures are shown in Table 2.

[0046] Table 2 SAO structure of bathroom text

[0047] Select a pair of words that need to be compared, and extract the SAO structure of the words according to whether the words are subjects, predicates or objects. Because some words have multiple word forms, such as dispenser, dispensers, dispensering and dispensation, it is necessary to convert multiple word forms into general forms. This embodiment uses the WordNetLemmatizer method in the python NLTK library to merge word forms. After word form merging, the SAOs of similar words are extracted respectively, and the total number of SAO statistics and co-occurrence SAOs are marked; finally, the co-occurrence SAOs are counted and calculated to obtain the SAO similarity. At the same time, the empirical score based on the knowledge base is calculated for the word pair. This embodiment uses the python wordnet module to calculate the wu_simlarity and lin_simlarity between word pairs, and calculates the average of the two algorithms. The calculation results of the SAO similarity of word pairs and the empirical similarity calculation results based on the knowledge base are shown in Table 3.

[0048] Table 3 SAO similarity calculation results of word pairs and empirical similarity calculation results based on knowledge base

[0049] Finally, the SAO similarity and the experience score are integrated into the Bayesian model to obtain the word pair similarity value. In this embodiment, forty pairs of words are selected for verification, and the final experience score and SAO similarity calculation results are shown in the "Similarity" column of Table 4.

[0050] Among them, the correlation coefficient is a measure of the strength of the relationship between two variables. In statistics, the main correlation coefficients used are the Pearson correlation coefficient and the Spearman correlation coefficient. The Pearson correlation coefficient mainly measures the degree of linear correlation between two variables, and its value is between [-1,1]. The closer the correlation coefficient is to 1, the more linear the correlation between the variables is. The closer the correlation coefficient is to -1, the more linear the negative correlation is, and 0 indicates nonlinear correlation. The main function of the Spearman correlation coefficient is to use a monotonic function to describe the degree of relationship between two variables. Its calculation method is mainly based on the ranking of the variables rather than the original data. The calculation formulas for the Pearson and Spearman correlation coefficients are: , ,in, represents the Pearson correlation coefficient between variable x and variable y, and n represents the number of observations; represents the spearman correlation coefficient, Indicates the rank difference of the corresponding variable.

[0051] In this embodiment, the SimLax-999 data set is selected as the manual judgment standard, and compared with the calculation results of the word semantic similarity measurement method integrating SAO and Bayesian model. The SimLax-999 data set score mainly measures the similarity between word pairs, and eliminates word pairs that can form phrases with mixed parts of speech (such as "yellow-hair", "play golf") to avoid such noise interference. It has a high reference value and is suitable for the evaluation of semantic similarity calculation. In addition, in addition to the word pairs in simlax-999, the word semantic similarity measurement method integrating SAO and Bayesian model also selects some data in wordsim-353 and word pairs with high semantic similarity to add to the verification set. The calculation results of the word semantic similarity measurement method integrating SAO and Bayesian model are shown in Table 4 below.

[0052] Table 4 Calculation results of the word semantic similarity measurement method integrating SAO and Bayesian model and the calculation results of other methods

[0053] The final Pearson correlation coefficient is obtained by comparing the word semantic similarity measurement method integrating SAO and Bayesian model with the semantic similarity calculation method based on knowledge base, the semantic similarity calculation method based on word vector and the manual verification set as shown in Table 5. Among them, Wu&lin is the average value of Wu_simlarity method and Lin_simlarity method based on wordnet knowledge base; CBOW is the cosine similarity calculation result of word vector based on bag of words model; Skip Gram is the cosine similarity calculation result of word vector based on neural network; Similarity is the calculation result of this method; Manual rating is manual rating. The spearman correlation coefficient is shown in Table 6.

[0054] Table 5 Pearson coefficients of calculation results of each method

[0055] Table 6 Spearman coefficients of calculation results of each method

[0056] In this embodiment, the node selection and the distribution of scores of the artificial verification set and the curve fitting status are as follows: Figure 5 As shown, the curve fitting condition is as follows Figure 6As shown. The Pearson correlation coefficient reaches 0.823466 and the p value is 6.84e-11, which shows that the word semantic similarity measurement method integrating SAO and Bayesian model has a strong correlation and significant effect with the manual scoring results. The spearman correlation coefficient reaches 0.819447, which shows that the word semantic similarity measurement method integrating SAO and Bayesian model has a strong dependence and consistency with the manual judgment results. It can be concluded from the curve fitting results and the node distribution status that the word semantic similarity measurement method integrating SAO and Bayesian model has better semantic similarity calculation effect than the semantic similarity calculation method based on knowledge base and the calculation method based on word vector. Based on the data in Table 2 and the manual annotation scoring results as the benchmark, the comparison results of various algorithms are shown in the following figure. Figure 7 As shown by Figure 7 It can be seen that the word semantic similarity measurement method integrating SAO and Bayesian model is closer to the result of manual judgment.

[0057] According to the calculation results in Table 4, the semantic similarity calculation model has good results when the semantic similarity of the word pairs is high or low, and the calculation effect is poor when the semantic similarity score is in the range of [0.4, 0.5], and there are obvious outliers; such as "achieve-try", "child-adult", "effort-difficulty", "second-minute", "capability-strength", etc. The main reason for this problem is that simlax-999 mainly scores the semantic similarity of word pairs. The above word pairs have low semantic similarity, but high semantic relevance, and their SAO co-occurrence opportunities are more. This view can be verified from the scoring results based on the knowledge base semantic similarity algorithm. This outlier phenomenon also leads to a slightly lower correlation coefficient score in this paper.

[0058] In summary, the word semantic similarity measurement method integrating SAO and Bayesian model aims to solve the challenge of word semantic similarity calculation in the field of natural language processing by integrating SAO structure and Bayesian model. The method first extracts the SAO structure of the text from the sample set, and performs statistical calculation on the SAO of similar words to obtain the SAO similarity statistical probability; then, the semantic similarity parameters of the words are obtained using semantic knowledge bases such as WordNet, and the Laplace smoothing technology is used to adjust the semantic consistency parameters. Finally, the semantic similarity probability of the words is calculated by combining the similarity parameters and the SAO similarity statistical posterior probability through the Bayesian model.

[0059] The word semantic similarity measurement method integrating SAO and Bayesian model can combine subjective judgment based on experience with objective analysis based on statistics, thereby improving the accuracy and reliability of word semantic similarity measurement. In addition, in the case study of the above-mentioned bathroom supplies text dataset, the method showed good feasibility and effectiveness. Through this method, the similarity between words can be more accurately understood and quantified, providing a new perspective and tool for the development of natural language processing technology. The word semantic similarity measurement method integrating SAO and Bayesian model has brought new breakthroughs in the field of semantic similarity measurement, and is expected to promote the advancement and application of related technologies.

[0060] See also Figure 8 The second embodiment of the present invention provides a word semantic similarity measurement device integrating SAO and Bayesian model, which includes: An extraction and statistics unit 201 is used to extract the SAO structure of the words to be compared from a preset corpus based on semantic part-of-speech tagging, and perform statistical preprocessing on the SAO structure to obtain statistical results; The SAO similarity unit 202 is used to calculate the SAO similarity of the words according to the statistical results to obtain a SAO similarity score; An experience similarity unit 203 is used to calculate the semantic similarity experience similarity between the comparison word and the word to be compared according to a preset knowledge base to generate experience similarity, wherein the knowledge base is a wordnet knowledge base; The Bayesian calculation unit 204 is used to substitute the SAO similarity score and the empirical similarity into the Bayesian formula to obtain the final word semantic similarity calculation result.

[0061] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A word semantic similarity measurement method integrating SAO and Bayesian model, characterized in that: include: Based on semantic part-of-speech tagging, the SAO structure of the words to be compared is extracted from the preset corpus, and the SAO structure is statistically preprocessed to obtain statistical results; According to the statistical results, the SAO similarity of the words is calculated to obtain a SAO similarity score; Calculating the semantic similarity experience similarity between the comparison word and the word to be compared according to a preset knowledge base to generate the experience similarity, wherein the knowledge base is a wordnet knowledge base; Substitute the SAO similarity score and the empirical similarity into the Bayesian formula to obtain the final calculation result of the word semantic similarity.

2. The word semantic similarity measurement method integrating SAO and Bayesian model according to claim 1, characterized in that: Based on semantic part-of-speech tagging, the SAO structure of the words to be compared is extracted from the preset corpus, and the SAO structure is statistically preprocessed to obtain statistical results, specifically: Sentence the text in the preset corpus to obtain multiple sentences, and perform semantic part-of-speech tagging on the words in each sentence; Perform sentence SAO extraction processing according to the marked word parts of speech, extract the SAO structures of the two words to be compared, perform word form merging processing on the extracted SAO structures, and convert other word forms of the words to be compared into prototypes; The occurrence frequencies of the SAO structures corresponding to the two words to be compared in the text are counted respectively, and compared to obtain the SAO structures that appear in the two words to be compared and their occurrence frequencies, and generate statistical results.

3. The word semantic similarity measurement method integrating SAO and Bayesian model according to claim 1, characterized in that: According to the statistical results, SAO similarity calculation is performed to obtain a SAO similarity score, which is specifically: Based on the statistical results, when it is determined that the comparison word and the word to be compared are subjects, the SAO similarity of the corresponding words is calculated according to the predicate, object and frequency co-occurring between the comparison word and the word to be compared; When it is determined that the comparison word and the word to be compared are predicates, the SAO similarity of the corresponding words is calculated according to the subjects, objects and their frequencies that appear together between the comparison word and the word to be compared; When it is determined that the comparison word and the word to be compared are objects, the SAO similarity of the corresponding words is calculated based on the subjects, predicates, and frequencies that appear together between the comparison word and the word to be compared; When it is determined that there is no co-occurring SAO structure between the comparison word and the word to be compared, the corresponding SAO similarity is 0, and the Laplace smoothing algorithm is used to adjust the word SAO similarity.

4. The word semantic similarity measurement method integrating SAO and Bayesian model according to claim 3 is characterized in that: The calculation formula for word SAO similarity calculation is: , , ,in, is the word-subject SAO similarity, is the word-predicate SAO similarity, is the word object SAO similarity, is the probability of co-occurrence of subject of word w1 and word w2, is the probability of co-occurrence of subject of word w1 and word w2, is the sum of the probabilities of subject co-occurrence between words, is the probability of word w1 and word w2 co-occurring as predicates, is the probability of word w2 co-occurring with word w1 as a predicate, is the sum of the probabilities of predicate co-occurrence between words, is the probability of co-occurrence of word w1 and word w2 object, is the probability of word w2 co-occurring with the object of word w1, is the sum of the probabilities of object co-occurrence between words.

5. The word semantic similarity measurement method integrating SAO and Bayesian model according to claim 3, characterized in that: The calculation formula of the Laplace smoothing algorithm is: , ,in, is the smoothing parameter, is the observation result of the j-th characteristic result of random variable a with value l, K is the number of characteristic types, is the number of feature classifications, N is the number of statistical samples, is an indicator function, indicating that in the i-th sample, the j-th feature The value is equal to , and category Y is equal to When , the function value is 1, otherwise it is 0. is the kth category, is the i-th category, is the jth feature in the i-th sample.

6. The word semantic similarity measurement method integrating SAO and Bayesian model according to claim 4, characterized in that: The semantic similarity experience similarity between the comparison word and the word to be compared is calculated according to the preset knowledge base to generate the experience similarity, which is specifically: According to the preset knowledge base, the path distance similarity experience score wu_simlarity based on the path and the semantic similarity experience score lin_simarity based on the concept content of the comparison word and the word to be compared are calculated to determine the highest score between the comparison word and the word to be compared; Calculate the average of the path distance similarity experience score wu_simlarity and the semantic similarity experience score lin_simarity to get the experience similarity. The calculation formula is: ,in, is the semantic similarity experience score calculated based on the path distance similarity experience score wu_similarity, is the semantic similarity experience score calculated based on the semantic similarity experience score lin_similarity; When it is determined that the scores of the path distance similarity experience score wu_simlarity and the semantic similarity experience score lin_simarity are both 1, the dissimilar experience similarity is 0, and the Laplace smoothing algorithm is used to adjust the experience similarity.

7. The word semantic similarity measurement method integrating SAO and Bayesian model according to claim 6, characterized in that: The calculation formula of the path distance similarity empirical score wu_simlarity is: ,in, is the highest common parent node of word w1 and word w2 in the knowledge base, is the shortest path length from word w1 to word w2, is the depth of the word in the knowledge base hierarchy tree.

8. The word semantic similarity measurement method integrating SAO and Bayesian model according to claim 6, characterized in that: The calculation formula of the semantic similarity empirical score lin_simarity is: ,in, is the probability that the information content of the nearest common parent node of word w1 and word w2 accounts for the total description information content, is the probability that the information content of word w1 accounts for the total information content, is the probability that the amount of information in word w2 accounts for the total amount of information, is the information content of the highest common parent node of word w1 and word w2 in the knowledge base, is the information content of word w1, is the information content of word w2.

9. The word semantic similarity measurement method integrating SAO and Bayesian model according to claim 6, characterized in that: Substituting the SAO similarity score and the empirical similarity into the Bayesian formula, we can obtain the final word semantic similarity calculation result, which is: The SAO similarity score is used as the probability of judging similarity between word pairs, and the empirical similarity is used as a tuning parameter; The Bayesian formula is used to calculate the SAO similarity score and the empirical similarity to obtain the final word semantic similarity calculation result. The calculation formula is: , .

10. A word semantic similarity measurement device integrating SAO and Bayesian model, characterized in that: include: An extraction statistics unit is used to extract the SAO structure of the words to be compared from a preset corpus based on semantic part-of-speech tagging, and perform statistical preprocessing on the SAO structure to obtain statistical results; An SAO similarity unit, used to calculate the SAO similarity of the words according to the statistical results to obtain a SAO similarity score; An experience similarity unit, used for calculating the semantic similarity experience similarity between the comparison word and the word to be compared according to a preset knowledge base, and generating experience similarity, wherein the knowledge base is a wordnet knowledge base; The Bayesian calculation unit is used to substitute the SAO similarity score and the empirical similarity into the Bayesian formula to obtain the final word semantic similarity calculation result.

Images