Word sense disambiguation method based on bi-lstm and graphsage

Abstract

The present application relates to a word sense disambiguation method based on BiLSTM and GraphSAGE. The present application firstly carries out word segmentation, part-of-speech tagging, semantic class labeling and traditional Chinese character labeling on Chinese sentences containing ambiguous words. The word form, part-of-speech and semantic class of the sentence containing ambiguous words and the four word units with the highest correlation degree to the ambiguous words are taken as disambiguation features, the disambiguation features are taken as nodes to construct a disambiguation feature graph, the Word2Vec tool and the Doc2Vec tool are used for vectorization processing of the features as the input of the GraphSAGE model, and the BERT encoder is used for vectorization processing of the word form, part-of-speech, semantic class and traditional Chinese character as the input of the BiLSTM model. The BiLSTM+GraphSAGE model is optimized by using training corpus, the optimized BiLSTM+GraphSAGE model is used for word sense disambiguation of test corpus, and the probability distribution sequence of the ambiguous words in each semantic category is obtained. The semantic category with the maximum probability is the semantic category of the ambiguous word. The present application has good word sense disambiguation effect and can more accurately judge the true meaning of the ambiguous word.

Description

Technical fields:

[0001] This invention relates to a word sense disambiguation method based on BiLSTM and GraphSAGE, which has been widely applied in the field of natural language processing. Background technology:

[0002] Word sense disambiguation is an important task in the field of natural language processing, aiming to determine the precise meaning of words in a given context. This is because many words have multiple possible meanings, and information from the context can provide clues to resolve ambiguities. Word sense disambiguation plays a crucial role in text classification, information retrieval, and machine translation; finding the correct meaning of ambiguous words will greatly benefit these fields.

[0003] Currently, machine learning algorithms are commonly used to disambiguate ambiguous words, such as Naive Bayes, association rule-based methods, and statistical classification methods. However, these machine learning algorithms have many shortcomings; they can only extract local disambiguation features, resulting in poor classification performance. With the widespread application of deep learning algorithms in natural language processing, the disambiguation performance of extracted features has improved significantly. This paper utilizes the BERT encoder and BiLSTM model to extract disambiguation features from lexical form, part-of-speech tagging, semantic class, and traditional Chinese characters. The GraphSAGE model is then used to extract disambiguation features from sentences and the inclusion relationships between lexical form, part-of-speech tagging, and semantic class. Finally, the disambiguation features from the two models are fused. For ambiguous words, this model can better disambiguate word meanings and correctly classify semantics. Summary of the Invention:

[0004] To address the ambiguity problem of Chinese vocabulary in the field of natural language processing, this invention discloses a word meaning disambiguation method based on BiLSTM and GraphSAGE.

[0005] Therefore, the present invention provides the following technical solution:

[0006] 1. A word sense disambiguation method based on BiLSTM and GraphSAGE, characterized in that the method includes the following steps:

[0007] Step 1: Perform word segmentation, part-of-speech tagging, semantic class tagging, and traditional Chinese character tagging on all Chinese sentences contained in the SemEval-2007:Task#5 corpus. Use the Word2Vec tool to convert words into word vectors and calculate the correlation between ambiguous words and other words. Select the word form, part-of-speech, semantic class, and traditional Chinese characters of the four word units with the highest correlation with the ambiguous words in the sentence, as well as the sentences containing the ambiguous words, as disambiguation features.

[0008] Step 2: Use the Doc2Vec tool to vectorize the extracted sentences, and use the Word2Vec tool to vectorize the lexical form, part-of-speech tag, and semantic class. Use the processed training corpus from SemEval-2007:Task#5 as the training data for GraphSAGE, and use the processed test corpus from SemEval-2007:Task#5 as the test data for GraphSAGE. Use the BERT encoder to vectorize the lexical form, part-of-speech tag, semantic class, and traditional Chinese characters. Use the processed training and test corpora from SemEval-2007:Task#5 as the training and test data for BiLSTM, respectively.

[0009] Step 3: Use the extracted sentence and the word form, part of speech, and semantic class of the four word units on the left and right with the highest correlation to the ambiguous words as graph nodes to construct the disambiguation feature map of GraphSAGE.

[0010] Step 4: During the training phase, the disambiguation feature map constructed using the training data is used to optimize the GraphSAGE model, and the training data is used to optimize the BiLSTM model.

[0011] Step 5: The testing phase is a semantic classification process. The disambiguation feature map constructed from the test data is input into the optimized GraphSAGE model, and the test data is input into the optimized BiLSTM model. The outputs of the two models are concatenated, and the probability distribution of the ambiguous word under each semantic category is calculated. The semantic category with the highest probability is the semantic category of the ambiguous word.

[0012] 2. The word sense disambiguation method based on BiLSTM and GraphSAGE according to claim 1, characterized in that, in step 1, the Chinese sentence containing ambiguous words w is segmented, labeled with parts of speech, semantically labeled, and labeled with traditional Chinese characters to extract its disambiguation features, the specific steps of which are as follows:

[0013] Step 1-1: Use a Chinese word segmentation tool to segment the Chinese sentence into words;

[0014] Steps 1-2: Use Chinese part-of-speech tagging tools to tag the words with their parts of speech;

[0015] Steps 1-3: Use Chinese semantic annotation tools to perform semantic category annotation on words;

[0016] Steps 1-4: Use a traditional Chinese character annotation tool to annotate the vocabulary with traditional Chinese characters;

[0017] Steps 1-5 calculate the degree of association between the ambiguous word w and the surrounding words, as shown below:

[0018]

[0019] Where n is the dimension of the word vector, and a = (a1, a2, ..., a... n Let ) be the Word2Vec word vector of the ambiguous word w, and b = (b1, b2, ..., bb2) n ) represents the Word2Vec word vectors of other words t in the context, and dist(w,t) represents the distance between words w and t. The word form, part of speech, semantic class, traditional Chinese characters, and sentences containing ambiguous words are selected as disambiguation features from the four word units with the highest correlation to the ambiguous word w.

[0020] 3. The word sense disambiguation method based on BiLSTM and GraphSAGE according to claim 1, characterized in that, in step 2, the sentence is vectorized, and the word form, part of speech, semantic class, and traditional Chinese characters are vectorized to obtain training data and test data, the specific steps of which are as follows:

[0021] Step 2-1: Use the Doc2Vec tool to vectorize the extracted sentence text T1 to obtain a 200-dimensional feature vector F. T1 As shown below:

[0022] F T1 =Doc2Vec(T1)

[0023] The extracted text T2, containing lexical, part-of-speech, and semantic categories, was vectorized using the Word2Vec tool to obtain a 2400-dimensional feature vector F. T2 As shown below:

[0024] F T2 =Word2Vec(T2)

[0025] Step 2-2 uses the BERT encoder to vectorize the extracted text T3 containing lexical forms, parts of speech, semantic classes, and traditional Chinese characters, resulting in a 12288-dimensional feature vector F. T3 As shown below:

[0026] F T3 =BERT(T3)

[0027] Step 2-3: Use the training corpus of SemEval-2007:Task#5 processed in Step 2-1 as the training data of the GraphSAGE model, denoted as training data 1; and use the test corpus of SemEval-2007:Task#5 processed in Step 2-1 as the test data of the GraphSAGE model, denoted as test data 1.

[0028] Step 2-4: Use the training corpus of SemEval-2007:Task#5 processed in Step 2-2 as the training data of the BiLSTM model, denoted as training data 2; and use the test corpus of SemEval-2007:Task#5 processed in Step 2-2 as the test data of the BiLSTM model, denoted as test data 2.

[0029] 4. The word sense disambiguation method based on BiLSTM and GraphSAGE according to claim 1, characterized in that, in step 3, constructing the disambiguation feature map specifically involves the following steps:

[0030] Step 3-1 takes the sentence containing the ambiguous word w and the word form, part of speech, and semantic class of the four word units with the highest correlation to w as nodes of the disambiguation feature map of GraphSAGE;

[0031] Step 3-2 uses the feature vector F from step 2-1 T1 and F T2 Weighted embedding is performed on the nodes of the disambiguation feature map, as shown below:

[0032] embedding(v d ) = F T1

[0033] embedding(v c ) = F T2

[0034] Among them, v d v is the feature vector node of the sentence. c These are feature vector nodes representing word form, part-of-speech, and semantic classes; embedding is the weighted embedding.

[0035] Step 3-3: Establish edges between nodes based on the inclusion relationship between word form, part of speech, semantic class, and sentence.

[0036] 5. The word sense disambiguation method based on BiLSTM and GraphSAGE according to claim 1, characterized in that, in step 4, the GraphSAGE and BiLSTM models are optimized, specifically through the following steps:

[0037] Step 4-1: Input the disambiguation feature map constructed from training data 1 into the initialized GraphSAGE model, and obtain the new feature vector t' of sentence node v through neighbor sampling and feature aggregation. v The aggregation formula is as follows:

[0038]

[0039] Where v is the sentence node, k is the k-th order neighbor sampling level, u is the neighbor node of node v, and N(v) represents the sampled neighbor set of node v. W is the feature vector of node v in the k-th level. mean is the weight coefficient matrix of the mean aggregator, MEAN represents the element-wise mean operation, and σ is the non-linear activation function;

[0040] Step 4-2: Convert the feature vector F of training data 2 T3 The input is fed into the LSTM model to obtain the forward hidden state of sentence node v. and backward hidden state As shown below:

[0041]

[0042]

[0043] Connecting the forward and backward hidden states yields the global feature h of sentence node v. t As shown below:

[0044]

[0045] Step 4-3 concatenates the outputs of the two model output layers to obtain x, as shown below:

[0046] x = concat(t' v ,h t )

[0047] After the fully connected layer in step 4-4x, the Softmax function is used to calculate the semantic category s of the ambiguous word w. i Predicted probabilities for (i = 1, 2, ..., n):

[0048]

[0049] Among them, a i This represents the input data for the Softmax function;

[0050] Steps 4-5 calculate the loss using the cross-entropy loss function, perform backpropagation based on the loss, and update the model parameters θ layer by layer. The update process is as follows:

[0051]

[0052] Where θ represents the parameter set, θ' represents the updated parameter set, and α is the learning rate;

[0053] Steps 4-6 repeat steps 4-1 to 4-5 until the number of iterations for training is reached, resulting in the optimized GraphSAGE and BiLSTM.

[0054] 6. The word sense disambiguation method based on BiLSTM and GraphSAGE according to claim 1, characterized in that, in step 5, semantic classification is performed on the ambiguous word w, specifically through the following steps:

[0055] Step 5-1: Input the disambiguation feature map constructed from test data 1 into the optimized GraphSAGE model, and obtain the new feature vector t' of sentence node v through neighbor sampling and feature aggregation. v The aggregation formula is shown below:

[0056]

[0057] Where v is the sentence node, k is the k-th order neighbor sampling level, u is the neighbor node of node v, and N(v) represents the sampled neighbor set of node v. W is the feature vector of node v in the k-th level. mean is the weight coefficient matrix of the mean aggregator, MEAN represents the element-wise mean operation, and σ is the non-linear activation function;

[0058] Step 5-2: Convert the feature vector F of test data 2 T3 The input is fed into the LSTM model to obtain the forward hidden state of sentence node v. and backward hidden state As shown below:

[0059]

[0060]

[0061] Connecting the forward and backward hidden states yields the global feature h of sentence node v. t As shown below:

[0062]

[0063] Step 5-3 concatenates the outputs of the two model output layers to obtain x, as shown below:

[0064] x = concat(t' v ,h t )

[0065] After the fully connected layer in step 5-4x, the Softmax function is used to calculate the semantic category s of the ambiguous word w. i Predicted probabilities for (i = 1, 2, ..., n):

[0066]

[0067] Among them, a i This represents the input data for the Softmax function;

[0068] Step 5-5: From P(s1|w), P(s2|w), ..., P(s n The semantic class with the highest probability in |w) is selected as the semantic category of the ambiguous word w:

[0069]

[0070] Beneficial effects:

[0071] 1. This invention is a word sense disambiguation method based on BiLSTM and GraphSAGE. It performs lexical segmentation, part-of-speech tagging, semantic class tagging, and traditional Chinese character tagging on Chinese sentences. The disambiguation features are vectorized using Doc2Vec, Word2Vec, and a BERT encoder, resulting in high-quality extracted disambiguation features.

[0072] 2. The model used in this invention is a BiLSTM+GraphSAGE model. The biggest feature of the BiLSTM model is that by stacking two LSTM layers, the model overcomes the limitation of predicting the output of the next time step based solely on the temporal information of previous time steps, and can better combine context for output. The biggest feature of GraphSAGE is that it uses a graph structure to iterate node features, with each node sampling only a portion of its neighboring nodes to iteratively update its features. By fusing the outputs of the BiLSTM model and the GraphSAGE model and then performing classification, a good classification result can be obtained.

[0073] 3. The classifier used in this invention is the Softmax classifier, which can not only solve the word sense disambiguation problem of two-word meanings, but also the word sense disambiguation problem of multiple-word meanings.

[0074] 4. Gradient descent is used to update parameters during model training. By calculating the error, the model parameters are updated iteratively to generate an optimized BiLSTM+GraphSAGE model, resulting in higher disambiguation accuracy. Attached image description:

[0075] Figure 1 This is a flowchart of the word sense disambiguation method based on BiLSTM and GraphSAGE in an embodiment of the present invention;

[0076] Figure 2 This is a disambiguation feature map of the word sense disambiguation method based on BiLSTM and GraphSAGE in the embodiments of the present invention;

[0077] Figure 3 is the training process of the word sense disambiguation method based on BiLSTM and GraphSAGE in the embodiments of the present invention;

[0078] Figure 4 is the sampling and aggregation process of the word sense disambiguation method based on BiLSTM and GraphSAGE in the embodiments of the present invention;

[0079] Figure 5 is the test process of the word sense disambiguation method based on BiLSTM and GraphSAGE in the embodiments of the present invention; DETAILED DESCRIPTION:

[0080] To clearly and completely describe the technical solutions in the implementations of the present invention, the present invention is further elaborated in detail by taking the test sentence containing the ambiguous word "children" "Under the difficult living conditions, Ji Rongzhen and her children depended on each other for survival until they were raised to adulthood" as an example. The ambiguous word "children" has two sense categories, s1: children, s2: young men and women. There are 60 sentences in the training corpus of the word "children", and 20 sentences in the test corpus of the word "children".

[0081] The flowchart of the word sense disambiguation method based on BiLSTM and GraphSAGE implemented in the present invention is as Figure 1 shown, which includes the following steps.

[0082] Step 1: Process the Chinese sentence containing the ambiguous word "children" "Under the difficult living conditions, Ji Rongzhen and her children depended on each other for survival until they were raised to adulthood" and extract disambiguation features, the steps are as follows:

[0083] Step 1-1: Perform word segmentation on the Chinese sentence using a Chinese word segmentation tool, which is specifically as follows:

[0084] Word segmentation result: Under the difficult living conditions, Ji Rongzhen and her children depended on each other for survival until they were raised to adulthood

[0085] Step 1-2: Perform part-of-speech tagging on the words using a Chinese part-of-speech tagging tool, which is specifically as follows:

[0086] Part-of-speech tagging: Under / p difficult / a the / u life / vn environment / n under / f Ji / nr Rongzhen / nr and / p children / n depend on each other / i until / v take / p them / r raise / v up / a adult / v

[0087] Step 1-3: Perform semantic category tagging on the words using a Chinese semantic tagging tool, which is specifically as follows:

[0088] Semantic class annotation: In the difficult living environment, Ji Rongzhen depended on her children for survival until they were raised to adulthood

[0089] Steps 1-4 use a traditional Chinese character annotation tool to perform traditional Chinese character annotation on the vocabulary, as follows specifically:

[0090] Traditional Chinese character annotation: In / difficult / difficult of / of life / life environment / environment under / under Ji / Ji Rongzhen / Rongzhen and / and children / children depend on each other for survival / depend on each other for survival until / until raise / raise them / them raise / raise up / up adult / adult

[0091] Steps 1-5 calculate the correlation degree between "children" and other vocabulary, as follows specifically:

[0092] exist 0.05913563072681427 difficult 0.12750311195850372 of 0.06507641822099686 Life -0.05107709765434265 environment 0.04072391614317894 Down -0.16034643352031708 Ji 0.060225050896406174 Rongzhen 0.021088464185595512 and -0.047791771590709686 Dependent on each other for survival -0.014990204945206642 until -0.030753014609217644 Bundle 0.09281620383262634 them -0.04130595922470093 keep 0.06638385355472565 big -0.04940113425254822 aldult 0.19641277194023132

[0093] Select the sentence where the ambiguous vocabulary "children" is located, as well as the word form, part of speech, semantic class and traditional Chinese characters of the 4 vocabulary units on the left and right with the highest correlation degree with the ambiguous vocabulary "children" as disambiguation features, as follows specifically:

[0094]

[0095] Step 2 Acquire training data and test data, the steps are as follows:

[0096] Step 2-1 Use the Doc2Vec tool to perform vectorization processing on the text T1 of the extracted sentence, to obtain a 200-dimensional feature vector F T1 , as follows specifically:

[0097] F T1 =Doc2Vec(In the difficult living environment, Ji Rongzhen depended on each other for survival until they were raised to adulthood)

[0098] Feature vector F T1 is as follows specifically:

[0099]

[0100] The extracted text T2, containing lexical, part-of-speech, and semantic categories, was vectorized using the Word2Vec tool to obtain a 2400-dimensional feature vector F. T2 The details are as follows:

[0101] F T2 =[f1,f2,f3,f4,f5,f6,f7,f8,f9,f 10 ,f 11 ,f 12 ]

[0102] f1 = Word2Vec (raise), f2 = Word2Vec (take), f3 = Word2Vec (difficult), f4 = Word2Vec (adult).

[0103] f5=Word2Vec(v), f6=Word2Vec(p), f7=Word2Vec(a), f8=Word2Vec(v)

[0104] f9 = Word2Vec(Jd01), f 10 =Word2Vec(Hc10)

[0105] f 11 =Word2Vec(Ef14),f 12 =Word2Vec(Ib01)

[0106] eigenvector F T2 Specifically as follows:

[0107]

[0108]

[0109] Step 2-2 uses the BERT encoder to vectorize the extracted text T3 containing lexical forms, parts of speech, semantic classes, and traditional Chinese characters, resulting in a 12288-dimensional feature vector F. T3 The details are as follows:

[0110] F T3 =[f1',f2',f3',f4',f5',f6',f7',f8',f9',f 10 ',f 11 ',f 12 ',f 13 ',f 14 ',f 15 ',f 16 ']

[0111] f1'=BERT(nourish),f2'=BERT(hold),f3'=BERT(difficult),f4'=BERT(adult)

[0112] f5'=BERT(v),f6'=BERT(p),f7'=BERT(a),f8'=BERT(v)

[0113] f9'=BERT(Jd01),f 10 '=BERT(Hc10)

[0114] f 11 '=BERT(Ef14),f 12 '=BERT(Ib01)

[0115] f 13 '=BERT(nourish),f 14 '=BERT(hold)

[0116] f 15 '=BERT(difficult),f 16 '=BERT(adult)

[0117] Feature vector F T3 is specifically as follows:

[0118]

[0119]

[0120] Step 2-3: Take the processed "children" training corpus from step 2-1 as the training data of the GraphSAGE model, denoted as training data 1, and take the processed "children" test corpus from step 2-1 as the test data of the GraphSAGE model, denoted as test data 1;

[0121] Step 2-4: Take the processed "children" training corpus from step 2-2 as the training data of the BiLSTM model, denoted as training data 2, and take the processed "children" test corpus from step 2-2 as the test data of the BiLSTM model, denoted as test data 2;

[0122] Step 3: Construct a disambiguation feature graph, as Figure 2 shown, the steps are as follows:

[0123] Step 3-1 takes the sentence containing the ambiguous word "children" ("In a difficult living environment, Ji Rongzhen and her children depended on each other until they raised them to adulthood") and the word forms "raise", "raise", "raise", "difficult", and "adulthood" of the four word units with the highest relevance to "children", the parts of speech "v", "p", "a", and "v", and the semantic classes "Jd01", "Hc10", "Ef14", and "Ib01" as nodes in the disambiguation feature map of GraphSAGE.

[0124] Step 3-2 uses the feature vector F from step 2-1 T1 and F T2 Weighted embedding is performed on the nodes of the disambiguation feature map, as shown below:

[0125] embedding(v d ) = F T1

[0126] embedding(v c ) = F T2

[0127] Among them, v d v is the feature vector node of the sentence. c These are feature vector nodes representing word form, part-of-speech, and semantic classes; embedding is the weighted embedding.

[0128] Step 3-3: Establish edges between nodes based on the inclusion relationship between word form, part of speech, semantic class, and sentence.

[0129] Step 4: Optimize the GraphSAGE and BiLSTM models using the training data. The training process is as follows: Figure 3 As shown, the steps are as follows:

[0130] Step 4-1: Input the disambiguation feature map constructed from training data 1 into the initialized GraphSAGE model, and obtain the new feature vector representation t' of sentence node v through neighbor sampling and feature aggregation. v The sampling and aggregation process is as follows Figure 4 As shown, the aggregation formula is as follows:

[0131]

[0132] Where v is the sentence node, k is the k-th order neighbor sampling level, u is the neighbor node of node v, and N(v) represents the sampled neighbor set of node v. W is the feature vector of node v in the k-th level. mean is the weight coefficient matrix of the mean aggregator, MEAN represents the element-wise mean operation, and σ is the non-linear activation function;

[0133] Step 4-2 Process the feature vector F of training data 2 T3 Input it into the LSTM model to obtain the forward hidden state of the sentence node v and the backward hidden state as shown below:

[0134]

[0135]

[0136] Concatenate the forward and backward hidden states to obtain the global feature h of the sentence node v t , as shown below:

[0137]

[0138] Step 4-3 Splice the output results of the output layers of the two models to obtain x, as shown below:

[0139] x=concat(t' v ,h t )

[0140] Step 4-4 After x passes through the fully connected layer, use the Softmax function to calculate the prediction probabilities P(s1|children) and P(s2|children) of the ambiguous word "children" under the semantic categories s1=children, s2=young_men_and_women;

[0141] Step 4-5 Use the cross-entropy loss function to calculate the loss loss 儿女 , the calculation result is as follows:

[0142] loss 儿女 =0.0003

[0143] Perform back propagation according to loss 儿女 , and update the model parameter θ layer by layer. The update process is as follows:

[0144]

[0145] Step 4-6 Repeat steps 4-1 to 4-5 for iteration until the number of iterative training is reached, so as to obtain the optimized GraphSAGE and BiLSTM;

[0146] Step 5 The test process is as Figure 5 shown, and the semantic classification of the ambiguous word "children" is carried out. The steps are as follows:

[0147] Step 5-1 Input the disambiguation feature graph constructed from the test sentence containing the ambiguous word "children and young people" "In the difficult living environment, Ji Rongzhen and her children depended on each other for life until they were raised up" into the optimized GraphSAGE model, and obtain the new feature vector representation t' of the sentence node v through neighbor sampling and feature aggregation v , the sampling and aggregation process is as Figure 4 shown, and the aggregation formula is as follows:

[0148]

[0149] where v is the sentence node, k is the number of k-hop neighbor sampling layers, u is the neighbor node of node v, N(v) represents the sampled neighbor set of node v, is the feature vector of node v in the k-th layer, W mean is the weight coefficient matrix of the mean aggregator, MEAN denotes the element-wise mean operation, and σ is the nonlinear activation function;

[0150] Step 5-2 Extract the feature vector F from the test sentence containing the ambiguous word "children and young people" "In the difficult living environment, Ji Rongzhen and her children depended on each other for life until they were raised up" T3 and input it into the LSTM model to obtain the forward hidden state of the sentence node v and the backward hidden state as shown below:

[0151]

[0152]

[0153] Concatenate the forward and backward hidden states to obtain the global feature h of the sentence node v t , as shown below:

[0154]

[0155] Step 5-3 Splice the output results of the output layers of the two models to obtain x, as shown below:

[0156] x=concat(t' v ,h t )

[0157] Step 5-4 After x passes through the fully connected layer, use the Softmax function to calculate the prediction probability of the ambiguous word "children and young people" under the semantic categories s1=children, s2=young_man_and_woman:

[0158] P(children|children and young people)=0.65

[0159] P(young_man_and_woman|érnǚ)=0.35

[0160] Herein, P(children|érnǚ) represents the prediction probability of the ambiguous word "érnǚ" under the semantic category s1=children, and P(young_man_and_woman|érnǚ) represents the prediction probability of the ambiguous word "érnǚ" under the semantic category s2=young_man_and_woman;

[0161] Step 5-5: Select the semantic category with the maximum probability from P(children|érnǚ) and P(young_man_and_woman|érnǚ) as the semantic category of "érnǚ":

[0162]

[0163] The optimized BiLSTM+GraphSAGE model is used to perform word sense disambiguation on the ambiguous word "érnǚ" in the Chinese sentence "In the difficult living environment, Ji Rongzhen and her children depended on each other for life until they were raised to adulthood", and the semantic category corresponding to the ambiguous word "érnǚ" is children.

[0164] The optimized BiLSTM+GraphSAGE model is used to disambiguate the test data containing the ambiguous word "érnǚ", and the accuracy acc of word sense disambiguation is:

[0165] acc=17 / 20≈0.85

[0166] The word sense disambiguation method based on BiLSTM and GraphSAGE in the embodiments of the present invention can select diversified and accurate disambiguation features, and determine the semantic category of ambiguous words in combination with the Softmax function, which has high accuracy.

[0167] The above is a detailed introduction to the embodiments of the present invention with reference to the accompanying drawings. The specific implementations herein are only used to help understand the method of the present invention. For those of ordinary skill in the art, various changes and modifications can be made in the specific implementations and application scopes according to the idea of the present invention, so the specification of the present invention should not be construed as a limitation to the present invention.

Claims

1. A word sense disambiguation method based on BiLSTM and GraphSAGE, characterized in that, The method includes the following steps: Step 1: Perform word segmentation, part-of-speech tagging, semantic class tagging, and traditional Chinese character tagging on all Chinese sentences contained in the SemEval-2007:Task#5 corpus. Convert words into word vectors using the Word2Vec tool and calculate the correlation between ambiguous words and other words. Select the word form, part-of-speech, semantic class, and traditional Chinese characters of the four word units with the highest correlation with ambiguous words in the sentence, as well as the sentences containing ambiguous words, as disambiguation features. Step 2: Use the Doc2Vec tool to vectorize the extracted sentences, and use the Word2Vec tool to vectorize the lexical form, part-of-speech tag, and semantic class. Use the processed training corpus from SemEval-2007:Task#5 as the training data for GraphSAGE, and use the processed test corpus from SemEval-2007:Task#5 as the test data for GraphSAGE. Use the BERT encoder to vectorize the lexical form, part-of-speech tag, semantic class, and traditional Chinese characters. Use the processed training and test corpora from SemEval-2007:Task#5 as the training and test data for BiLSTM, respectively. Step 3: Use the extracted sentence and the word form, part of speech, and semantic class of the four word units with the highest correlation to the ambiguous words as graph nodes to construct the disambiguation feature map of GraphSAGE; Step 4: During the training phase, the disambiguation feature map constructed using the training data is used to optimize the GraphSAGE model, and the training data is used to optimize the BiLSTM model. Step 5: The testing phase is a semantic classification process. The disambiguation feature map constructed from the test data is input into the optimized GraphSAGE model, and the test data is input into the optimized BiLSTM model. The outputs of the two models are concatenated, and the probability distribution of the ambiguous word under each semantic category is calculated. The semantic category with the highest probability is the semantic category of the ambiguous word.

2. The word sense disambiguation method based on BiLSTM and GraphSAGE according to claim 1, characterized in that, In step 1, Chinese sentences containing the ambiguous word 'w' are segmented, labeled with parts of speech, semantically categorized, and annotated with traditional Chinese characters to extract their disambiguation features. The specific steps are as follows: Step 1-1: Use a Chinese word segmentation tool to segment the Chinese sentence into words; Steps 1-2: Use Chinese part-of-speech tagging tools to tag the words with their parts of speech; Steps 1-3: Use Chinese semantic annotation tools to perform semantic category annotation on words; Steps 1-4: Use a traditional Chinese character annotation tool to annotate the vocabulary with traditional Chinese characters; Steps 1-5 calculate the degree of association between the ambiguous word w and the surrounding words, as shown below: Where n is the dimension of the word vector, and a = (a1, a2, ..., a... n Let ) be the Word2Vec word vector of the ambiguous word w, and b = (b1, b2, ..., bb2) n ) represents the Word2Vec word vectors of other words t in the context, and dist(w,t) represents the distance between words w and t. The word form, part of speech, semantic class, traditional Chinese characters, and sentences containing ambiguous words are selected as disambiguation features from the four word units with the highest correlation to the ambiguous word w.

3. The word sense disambiguation method based on BiLSTM and GraphSAGE according to claim 1, characterized in that, In step 2, the sentence is vectorized, and the word form, part of speech, semantic class, and traditional Chinese characters are also vectorized to obtain training and test data. The specific steps are as follows: Step 2-1: Use the Doc2Vec tool to vectorize the extracted sentence text T1 to obtain a 200-dimensional feature vector F. T1 As shown below: F T1 =Doc2Vec(T1) The extracted text T2, containing lexical, part-of-speech, and semantic categories, was vectorized using the Word2Vec tool to obtain a 2400-dimensional feature vector F. T2 As shown below: F T2 =Word2Vec(T2) Step 2-2 uses the BERT encoder to vectorize the extracted text T3 containing lexical forms, parts of speech, semantic classes, and traditional Chinese characters, resulting in a 12288-dimensional feature vector F. T3 As shown below: F T3 =BERT(T3) Step 2-3: Use the training corpus of SemEval-2007:Task#5 processed in Step 2-1 as the training data of the GraphSAGE model, denoted as training data 1; and use the test corpus of SemEval-2007:Task#5 processed in Step 2-1 as the test data of the GraphSAGE model, denoted as test data 1. Step 2-4: Use the training corpus of SemEval-2007:Task#5 processed in Step 2-2 as the training data of the BiLSTM model, denoted as training data 2. Use the test corpus of SemEval-2007:Task#5 processed in Step 2-2 as the test data of the BiLSTM model, denoted as test data 2.

4. The word sense disambiguation method based on BiLSTM and GraphSAGE according to claim 1, characterized in that, In step 3, the disambiguation feature map is constructed, and the specific steps are as follows: Step 3-1 takes the sentence containing the ambiguous word w and the word form, part of speech, and semantic class of the four word units with the highest correlation to w as nodes of the disambiguation feature map of GraphSAGE; Step 3-2 uses the feature vector F from step 2-1 T1 and F T2 Weighted embedding is performed on the nodes of the disambiguation feature map, as shown below: embedding(v d )=F T1 embedding(v c )=F T2 Among them, v d v is the feature vector node of the sentence. c These are feature vector nodes representing word form, part-of-speech, and semantic classes; embedding is the weighted embedding. Step 3-3: Establish edges between nodes based on the inclusion relationship between word form, part of speech, semantic class, and sentence.

5. The word sense disambiguation method based on BiLSTM and GraphSAGE according to claim 1, characterized in that, In step 4, the GraphSAGE and BiLSTM models are optimized. The specific steps are as follows: Step 4-1: Input the disambiguation feature map constructed from training data 1 into the initialized GraphSAGE model, and obtain the new feature vector t' of sentence node v through neighbor sampling and feature aggregation. v The aggregation formula is as follows: Where v is the sentence node, k is the k-th order neighbor sampling level, u is the neighbor node of node v, and N(v) represents the sampled neighbor set of node v. W is the feature vector of node v in the k-th level. mean is the weight coefficient matrix of the mean aggregator, MEAN represents the element-wise mean operation, and σ is the non-linear activation function; Step 4-2: Convert the feature vector F of training data 2 T3 The input is fed into an LSTM model to obtain the forward hidden state of sentence node v. and backward hidden state As shown below: Connecting the forward and backward hidden states yields the global feature h of sentence node v. t As shown below: Step 4-3 concatenates the outputs of the two model output layers to obtain x, as shown below: x=concat(t' v ,h t ) After the fully connected layer in step 4-4x, the Softmax function is used to calculate the semantic category s of the ambiguous word w. i Predicted probabilities for (i = 1, 2, ..., n): Among them, a i This represents the input data for the Softmax function; Steps 4-5 calculate the loss using the cross-entropy loss function, perform backpropagation based on the loss, and update the model parameters θ layer by layer. The update process is as follows: Where θ represents the parameter set, θ' represents the updated parameter set, and α is the learning rate; Steps 4-6 repeat steps 4-1 to 4-5 until the required number of training iterations is reached, resulting in the optimized GraphSAGE and BiLSTM.

6. The word sense disambiguation method based on BiLSTM and GraphSAGE according to claim 1, characterized in that, In step 5, semantic classification is performed on the ambiguous word w. The specific steps are as follows: Step 5-1: Input the disambiguation feature map constructed from test data 1 into the optimized GraphSAGE model, and obtain the new feature vector t' of sentence node v through neighbor sampling and feature aggregation. v The aggregation formula is shown below: Where v is the sentence node, k is the k-th order neighbor sampling level, u is the neighbor node of node v, and N(v) represents the sampled neighbor set of node v. W is the feature vector of node v in the k-th level. mean is the weight coefficient matrix of the mean aggregator, MEAN represents the element-wise mean operation, and σ is the non-linear activation function; Step 5-2: Convert the feature vector F of test data 2 T3 The input is fed into an LSTM model to obtain the forward hidden state of sentence node v. and backward hidden state As shown below: Connecting the forward and backward hidden states yields the global feature h of sentence node v. t As shown below: Step 5-3 concatenates the outputs of the two model output layers to obtain x, as shown below: x=concat(t' v ,h t ) After the fully connected layer in step 5-4x, the Softmax function is used to calculate the semantic category s of the ambiguous word w. i Predicted probabilities for (i = 1, 2, ..., n): Among them, a i This represents the input data for the Softmax function; Step 5-5: From P(s1|w), P(s2|w), ..., P(s n The semantic class with the highest probability in |w) is selected as the semantic category of the ambiguous word w: .

Images