Method and system for automatically annotating unknown words based on syntactic and semantic analysis

Abstract

The application relates to a new word automatic marking method and system based on syntax and semantic analysis, which comprises the following steps: S1, selecting a new word to be marked and a new word text sentence; S2, screening target entries according to the part of speech of the new word to be marked, if there are multiple target entries, turning to step S3, if there is no target entry, returning to "no corresponding part of speech entry" and turning to step S4, if there is only one target entry, determining the word meaning of the new word to be marked and turning to step S5; S3, calculating the similarity of the new word text sentence and each target entry and sorting in descending order to obtain a similarity list, if the difference between the maximum value and the second maximum value of the similarity list is not greater than K, prompting "there are multiple possible entries" and turning to step S4, otherwise, determining the word meaning of the new word to be marked according to the target entry corresponding to the maximum value of the similarity list and turning to step S5; S4, manually determining the word meaning of the new word to be marked; and S5, marking the new word to be marked according to the word meaning. The application provides a method for realizing automatic marking of English vocabulary.

Description

Technical Field

[0001] This invention relates to the field of text information processing, specifically to a method and system for automatic word tagging based on syntactic and semantic analysis. Background Technology

[0002] Vocabulary annotation is a crucial part of English language education, particularly important for developing students' reading skills and building their vocabulary. Annotators need to select words based on the target learners' existing English proficiency and vocabulary level, and then choose appropriate definitions based on the meaning of the text. This work heavily relies on the annotator's understanding of the English text and their grasp of the readers' learning progress; it is time-consuming and labor-intensive, yet has a significant impact on learners. Accurate annotation can promote English learners' progress. Reading is the primary way for students to improve their language skills, and unfamiliar words are both a major obstacle and a key area for improvement. Appropriate vocabulary annotation can effectively reduce the difficulty of learning for students and increase their vocabulary accumulation.

[0003] Currently, vocabulary annotation for student English reading is either done manually by English teachers and editors or automatically by computers using a set-up thesaurus. The accuracy of manual annotation heavily relies on the annotator's understanding of students' abilities at different levels, leading to individual differences and potential errors or omissions. Furthermore, manual annotation is labor-intensive and unsuitable for processing large volumes of English reading material. Computer-based vocabulary annotation often employs a one-size-fits-all approach, directly using basic or primary definitions without considering specific semantic usage or context. While efficient, this method lacks flexibility. Some names, place names, and occasionally encountered unfamiliar words that don't affect reading comprehension are still annotated, reducing the reader's experience and increasing distractions. In recent years, annotation methods based on the similarity between word vectors and word vectors in various definition sentences have emerged. While these methods consider the semantic information of word vectors, they fail to consider the specific usage context and actual scenarios, relying solely on uniform word vector features. This still easily results in one-dimensional annotations and discrepancies between the annotated meaning and actual usage. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method and system for automatic word tagging based on syntactic and semantic analysis.

[0005] The technical solution of this invention is: an automatic word tagging method based on syntactic and semantic analysis, comprising:

[0006] Step S1: Based on the target audience's English proficiency and vocabulary list, select the new words to be annotated, and define all sentences containing the new words to be annotated as new word text sentences;

[0007] Step S2: Filter target entries based on the part of speech of the new word to be tagged. If there are multiple target entries, proceed to step S3. If the target entry does not exist, return the prompt: No corresponding part of speech entry, and proceed to step S4. If there is only one target entry, determine the meaning of the new word to be tagged based on the target entry, and proceed to step S5.

[0008] Step S3: Calculate the sentence vector similarity CosSim and sentence co-occurrence similarity CooSim between the new word text sentence and the definition example sentences of each target word; perform a weighted sum of CosSim and CooSim to obtain the overall similarity Sim between the new word text sentence and each definition example sentence, and sort them; according to the sorted Sim, calculate the similarity AveSim between the new word text sentence and each target word and sort them in descending order to obtain AveSimList. If the difference between the maximum value and the second largest value of AveSimList is not greater than the threshold K, return the prompt: there are multiple possible words, and go to step S4; otherwise, determine the meaning of the new word to be labeled according to the word corresponding to the maximum value of AveSimList and go to step S5.

[0009] Step S4: The meaning of the new words to be annotated is determined manually based on different prompts;

[0010] Step S5: Based on the meaning of the new word to be annotated, complete the annotation of the new word to be annotated, including annotating its part of speech, phonetic symbols, English definition, and Chinese definition.

[0011] Compared with the prior art, the present invention has the following advantages:

[0012] This invention discloses an automatic vocabulary annotation method based on syntactic and semantic analysis. The automatic annotation of new words in reading texts allows for flexible selection of words for annotation based on the learner's learning stage and the specific word usage within the text. This method considers the semantic information of the word context and the network structure features of word co-occurrence. Furthermore, the selection of definitions uses the similarity calculation units of each definition clause, highlighting the application scenario of each clause. This invention achieves efficient and accurate vocabulary annotation, effectively improving the efficiency of vocabulary annotation in reading texts while reducing related manpower and financial investment. Attached Figure Description

[0013] Figure 1 This is a flowchart of an automatic word tagging method based on syntactic and semantic analysis in an embodiment of the present invention;

[0014] Figure 2 This is a schematic diagram of the syntactic component analysis results in an embodiment of the present invention;

[0015] Figure 3 This is a schematic diagram of the annotation results in an embodiment of the present invention;

[0016] Figure 4 This is a framework diagram of the automatic word tagging method based on syntactic and semantic analysis in an embodiment of the present invention;

[0017] Figure 5 This is a structural block diagram of an automatic word tagging system based on syntactic and semantic analysis in an embodiment of the present invention. Detailed Implementation

[0018] This invention provides an automatic vocabulary annotation method based on syntactic and semantic analysis, which realizes automated annotation of English vocabulary and saves annotators' time and effort.

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below through specific implementations and in conjunction with the accompanying drawings.

[0020] Example 1

[0021] like Figure 1 As shown in the figure, an automatic word tagging method based on syntactic and semantic analysis provided by an embodiment of the present invention includes the following steps:

[0022] Step S1: Based on the target audience's English proficiency and vocabulary list, select the new words to be annotated and define all sentences containing the new words as new word text sentences;

[0023] Step S2: Filter target entries based on the part of speech of the new words to be annotated. If there are multiple target entries, proceed to step S3. If no target entry exists, return the prompt: No corresponding part of speech entry, and proceed to step S4. If there is only one target entry, determine the meaning of the new words to be annotated based on the target entry, and proceed to step S5.

[0024] Step S3: Calculate the sentence vector similarity CosSim and sentence co-occurrence similarity CooSim between the new word text sentence and the definition example sentences of each target word; perform a weighted sum of CosSim and CooSim to obtain the overall similarity Sim between the new word text sentence and each definition example sentence, and sort them; according to the sorted Sim, calculate the similarity AveSim between the new word text sentence and each target word and sort them in descending order to obtain AveSimList. If the difference between the maximum value and the second largest value of AveSimList is not greater than the threshold K, return the prompt: there are multiple possible words, and go to step S4; otherwise, determine the meaning of the new word to be labeled according to the word corresponding to the maximum value of AveSimList and go to step S5.

[0025] Step S4: The meaning of the new words to be annotated is determined manually based on different prompts;

[0026] Step S5: Based on the meaning of the new words to be annotated, complete the annotation of the new words to be annotated, including annotating their part of speech, phonetic symbols, English definition, and Chinese definition.

[0027] In one embodiment, step S1 above, based on the target audience's English proficiency and vocabulary, selects new words to be annotated and defines all sentences containing these new words as new word text sentences, specifically including:

[0028] Step S11: Based on the target audience's English proficiency and vocabulary, identify the new words to be annotated in the current text that are not in the vocabulary list, and add them to the NewWordList.

[0029] The vocabulary list in this step can be determined based on the vocabulary list that the target audience has already learned in their grade level, or it can be determined based on the target audience's own vocabulary mastery (such as the vocabulary list they have already mastered in vocabulary learning software).

[0030] Step S12: Define all sentences containing "newword" as newword text sentences and add them to the newword text sentence list NewSentenceList.

[0031] For example, consider the sentence "She draws a painting with a pencil" which corresponds to the word "draw" to be annotated.

[0032] In one embodiment, step S2 above: Filter target terms based on the part of speech of the new word to be tagged. If multiple target terms exist, proceed to step S3. If no target term exists, return the message: No corresponding part of speech term, and proceed to step S4. If only one target term exists, determine the meaning of the new word to be tagged based on the target term, and proceed to step S5, specifically including:

[0033] Step S21: Perform syntactic component analysis on the new word text sentences in NewSentenceList to determine the component of the new word to be tagged in the new word text sentences, thereby determining the part of speech Pos of the new word to be tagged, and storing the part of speech Pos in WordPOS;

[0034] Using the Stanford CoreNLP syntactic analysis tool, syntactic component analysis was performed on the sentences containing new words in NewSentenceList to determine the sentence component information of the new words. The processing results were stored in NewParse. If the tagging result of the new word begins with NN, it is tagged as a noun; if it begins with VB, it is tagged as a verb; if it begins with JJ, it is tagged as an adjective; if it begins with RB, it is tagged as an adverb; other cases are tagged according to the meaning of Stanford CoreNLP syntactic tags to complete the syntactic tagging. After syntactic analysis, the syntactic POS of the new word newword is stored in WordPOS. Taking "She draws a picture with a pencil." as an example, the syntactic analysis result is as follows: Figure 2 As shown, the word "draw" is marked as a verb in this sentence.

[0035] Step S22: Based on the newword and its part of speech Pos, search for the target word in an English electronic dictionary containing example sentences. If multiple target words exist, proceed to step S3. If the target word does not exist, return the message "No corresponding part of speech word" and proceed to step S4. If there is only one target word, determine the meaning of the new word to be marked based on the target word and proceed to step S5.

[0036] The word "draw" has multiple target entries in the Collins Advanced Learner's Dictionary, each with a different verb form. Target entry A defines it as "When you draw, or when you draw something, you use an axe or pento produce a painting, pattern, or diagram." Target entry B defines it as "When a vehicle draws somewhere, it moves there smoothly and steadily." Target entry C defines it as "If you draw somewhere, you move it slowly." This indicates that the word "draw" has multiple meanings. The next step is to calculate the similarity between the example sentences corresponding to the definitions of target entries A, B, and C and the text sentence containing the new word, in order to find the definition of "draw" that best matches the text sentence "She draws a painting with an axe."

[0037] In one embodiment, S3 above involves: calculating the sentence vector similarity CosSim and sentence co-occurrence similarity CooSim between the new word text sentence and the definition example sentences of each target word; weighted summing of CosSim and CooSim to obtain the overall similarity Sim between the new word text sentence and each definition example sentence, and sorting them; calculating the similarity AveSim between the new word text sentence and each target word according to the sorted Sim, and sorting it in descending order to obtain AveSimList; if the difference between the maximum value and the second largest value of AveSimList is not greater than the threshold K, a prompt is returned: there are multiple possible words, and the process proceeds to step S4; otherwise, the meaning of the new word to be annotated is determined according to the word corresponding to the maximum value of AveSimList, and the process proceeds to step S5, which specifically includes:

[0038] Step S31: Calculate the similarity CosSim between the related sentence vectors of the new word text sentence and the target word entry;

[0039] Step S32: Calculate the sentence co-occurrence similarity (CooSim) between the new word text sentence and the target word entry;

[0040] Step S33: Use formula (2) to perform a weighted summation of sentence vector similarity CosSim(s,e) and sentence co-occurrence similarity CooSim(s,e) to obtain the overall similarity Sim(s,e) between the new word text sentence and each definition example sentence of the target word entry;

[0041] Sim(s,e)=aCosSim(s,e)+bCooSim(s,e)(1)

[0042] Where s is the sentence of the new word text, e is the example sentence of the definition in the entry, and a and b are hyperparameters with values ​​in the range of 0-1;

[0043] Step S34: Sort all examples in descending order of their overall similarity to the sentence Sim(s,e), and store the sorting results in SimList;

[0044] For example, the vocabulary text sentence for "draw" has three target entries A, B, and C. Assume that target entry A includes three example sentences A1, A2, and A3, with overall similarity values ​​of Sim. A1 Sim A2 Sim A3 The target term B includes one example sentence B1, with an overall similarity of Sim. B1 The target term C includes two example sentences, C1 and C2, with an overall similarity of Sim. C1 Sim C2After the above calculations and sorting steps, the resulting SimList is Sim. A3 Sim A1 Sim C1 Sim A2 Sim C2 Sim B1 ;

[0045] Step S35: Calculate the similarity between the sentence and the target term entry, AveSim(s,entry):

[0046] For the target term "entry", select the first h definition examples from SimList and sum them using DCG formula (2), that is:

[0047]

[0048] Where r is the rank of the selected example sentence in SimList;

[0049] The similarity between sentence and the target term entry, AveSim(s,entry), is calculated using formula (3):

[0050]

[0051] For example, calculate the DCG(s,entry) of the target word A in the new word text sentence of draw. A ), AveSim(s,entry A ); DCG(s,entry) of target term B B ), AveSim(s,entry B ); DCG(s,entry) of target term C C ), AveSim(s,entry C ).

[0052] Step S36: Sort the similarity between the sentence and each word entry, AveSim(s,entry), in descending order and store it in AveSimList. If the difference between the maximum and the second largest value in AveSimList is not greater than the threshold K, return the prompt: there are multiple possible words, and go to step S4. Otherwise, determine the meaning of the new word to be labeled according to the word corresponding to the maximum value in AveSimList and go to step S5.

[0053] AveSimList in descending order is AveSim(s, entry) A ), AveSim(s,entry C ), AveSim(s,entry B), and AveSim(s,entry A ) and AveSim(s,entry C If the difference is less than K, it indicates that the meanings of target terms A and C are similar, and further manual determination of the meanings of the new words to be annotated is required. If AveSim(s,entry) A ) and AveSim(s,entry C If the difference is not less than K, then the meaning of "draw" in target word A can be used as the meaning of the new word to be tagged in the new word text sentence. In this embodiment of the invention, the value of K ranges from 0 to 1.

[0054] In step S3, the similarity between the new word text sentence and the target term is calculated based on the similarity of the associated sentence vectors and the co-occurrence similarity. A weighted average of these two similarities is used, combining semantic similarity and contextual relevance, to more accurately measure the relevance between the new word text sentence and the target term. Simultaneously, the appropriate annotation path is determined based on the magnitude of the similarity difference between the target terms.

[0055] In one embodiment, step S31 above: calculating the similarity CosSim between the related sentence vectors of the new word text sentence and the target word entry, specifically includes:

[0056] Step S311: Based on NewSentenceList, use the target sentence vector calculation module to calculate the sentence vector Nsvec for each new word text sentence;

[0057] Step S312: Based on the target term, construct the example sentence list ExampleSentenceList, and use the target sentence vector calculation module to calculate the sentence vector Dsvec for each example sentence;

[0058] Step S313: Calculate the similarity CosSim between Nsvec and Dsvec using cosine similarity, i.e., the similarity of the related sentence vectors:

[0059]

[0060] Where |.| is the length of the vector.

[0061] This step calculates the sentence vectors of the new word text sentence and the definition example sentence of the target term, and then calculates the similarity of the associated sentence vectors between the sentence vectors of the new word text sentence and the definition example sentence. This method measures the similarity between sentences, is concise, and preserves the semantic similarity between content words.

[0062] In one embodiment, the target sentence vector calculation module in S311 above specifically includes:

[0063] Step S3111: Input the target sentence, delete stop words and function words in the target sentence, keep only content words, and use the BERT model to generate word vectors vec for each content word in the target sentence;

[0064] Step S3112: For a target sentence containing "newword", perform a weighted summation of the word vectors (vec) for each content word in the target sentence except "newword":

[0065]

[0066] Where, weight n Let be the weight of the nth content word in the target sentence excluding newword, a be the smoothing parameter, and p(n) be the expected frequency of the nth content word in the current text.

[0067] Step S3113: Calculate the sentence vector svec of the target sentence:

[0068]

[0069] Where |t| is the length of the target sentence excluding the newword, and weight n For the nth word vector vec n Corresponding weights.

[0070] This invention constructs a target sentence vector calculation module to calculate the sentence vector of the target sentence. This module obtains a sentence vector excluding new word vectors by retaining content words and using a weighted average of the word vectors. Its advantages are: first, retaining content words reduces the influence of irrelevant words on the sentence vector; second, weighting reduces the influence of high-frequency but semantically low-importance words on the sentence vector; and third, using word vectors other than new word vectors to calculate the sentence vector allows for better calculation of sentence similarity, avoiding the problem that directly using BERT model word vectors cannot effectively express semantic relevance between words.

[0071] In one embodiment, step S32 above: calculating the sentence co-occurrence similarity CooSim between the new word text sentence and the target word entry, specifically includes:

[0072] Step S321: Input the new word text sentence and all the definition example sentences of the target word, delete stop words and non-content words in the new word text sentence and definition example sentences, and keep only the content words;

[0073] Step S322: Input the new word text sentence and the definition example sentence after retaining the content words into the word co-occurrence similarity calculation module to obtain the word co-occurrence similarity WordSim for each content word in the two;

[0074] Step S323: Assume s is the new word text sentence, e is the definition example sentence, r is the content word in the new word text sentence, and p is the content word in the definition example sentence;

[0075] Let i = 0 in s and j = 0 in e. Let WordSim(s) be the co-occurrence similarity between content words in the new word text sentence and the definition example sentence. i ,e j The maximum value is set to max[i] = 0, the maximum co-occurrence similarity is set to sum = 0;

[0076] At the current positions i,j, if WordSim(s) i ,e j If ) is greater than max[i], then max[i] = WordSim(s) i ,e j ), j++; otherwise, j++;

[0077] When j reaches the position of the last word p, i++, j = 0;

[0078] When i reaches the position of the last word r, max[i] records the maximum co-occurrence similarity between content words in the example sentences corresponding to each content word in the new word text sentence;

[0079] Let the current position in s be k = 0;

[0080] At the current position k, sum = sum + max[k], k++;

[0081] When k reaches the position of the last word r, sum records the maximum co-occurrence similarity between the new word text sentence and the example sentence of the definition, i.e., sum(s,e);

[0082] Calculate the sentence co-occurrence similarity (CooSim(s,e)) between the text sentences containing new words and the example sentences containing definitions:

[0083]

[0084] Where |s| is the sentence length after retaining content words.

[0085] This step calculates the sentence co-occurrence similarity between the new word text sentence and the definition example sentence by summing the maximum values ​​of the content words in the new word text sentence and the content words in the definition example sentence. This simple calculation preserves the maximum co-occurrence similarity information between sentences.

[0086] In one embodiment, the word co-occurrence similarity calculation module in S322 above specifically includes:

[0087] Step S3221: Input the definition examples of all entries in the selected English electronic dictionary and use them as corpus C for text preprocessing, including: sentence segmentation, word segmentation, deletion of stop words or non-content words, and part-of-speech restoration;

[0088] Step S3222: If a content word appears in corpus C, increment its frequency by 1, and count the frequency of each content word in corpus C;

[0089] Step S3223: Define the sentence length as the co-occurrence window size. Two content words appearing in the same co-occurrence window are called co-occurring words. Increment the co-occurrence frequency by 1 and count the co-occurrence frequency between content words in corpus C.

[0090] For example, in the example sentence "Drawaroughdesignforalogo.", the words draw and design co-occur.

[0091] Step S3224: Assume f(w) is the frequency of the content word w in the corpus C, f(w i ,w j ) is a content word w i With content words w j The co-occurrence frequency is used to measure the co-occurrence distance between content words, dis(w). i ,w j ),Right now:

[0092]

[0093] Step S3226: Based on the co-occurrence distance dis(w) i ,w j WordSim(w) calculates the co-occurrence similarity between content words. i ,w j ),Right now:

[0094]

[0095] This invention constructs a word co-occurrence similarity calculation module, which calculates the co-occurrence distance between content words in the corpus based on information such as word frequency and co-occurrence word frequency in the corpus. This is used to characterize the proximity of the co-occurrence association between words, and the co-occurrence similarity characterizes the degree of similarity between co-occurring words that retains the contextual association of the sentence in which they are located.

[0096] In one embodiment, step S4 above, where a person manually determines the meaning of the new word to be annotated based on different prompts, specifically includes:

[0097] Step S41: When the prompt is: No corresponding part of speech entry, the meaning of the new word to be annotated is determined manually;

[0098] Step S42: When the prompt indicates that there are multiple possible entries, that is, multiple entries with similarity AveSim(s,entry), select the definition example sentence MaxSentence with the highest overall similarity Sim(s,e) to the new word text sentence from each possible entry. Sort the entries from high to low according to the overall similarity Sim(s,e) of each MaxSentence sentence and store them in MaxList. Then, according to the order in MaxList, based on the MaxSentence example sentences in the similar entries, manually determine the meaning of the new word to be labeled.

[0099] For example, the MaxList of the vocabulary word "draw" is Sim. A3 Sim C1 Sim B1 The corresponding example sentences are: A3: "He starts a painting by quickly drawing simplified shapes." C1: "She drew away and didn't smile." B1: "Claire had seen the taxi drawing away." The word "draw" in the manually determined text sentences is closest in meaning to entry A, which means "When you draw, or when you draw something, you use a pencil or pento produce a diagram, pattern, or diagram."

[0100] In one embodiment, step S5 above: according to the meaning of the new word to be annotated, complete the annotation of the new word to be annotated, including annotating its part of speech, phonetic symbols, English definition and Chinese definition.

[0101] Based on the selected target entry in the dictionary, obtain all content within that entry, such as part of speech, phonetic transcription, and definition. Then, select the necessary annotations (part of speech, phonetic transcription, definition, etc.) to mark unfamiliar words. For example... Figure 3 The image shows the annotation results for the new word "draw".

[0102] If Chinese definitions are required, a dictionary with Chinese definitions should be selected, or automatic translation software should be used to translate the dictionary entries, and then the Chinese translation results should be added to the annotations.

[0103] This invention discloses an automatic vocabulary annotation method based on syntactic and semantic analysis. The automatic annotation of new words in reading texts allows for flexible selection of words for annotation based on the learner's learning stage and the specific word usage within the text. This method considers the semantic information of the word context and the network structure features of word co-occurrence. Furthermore, the selection of definitions uses the similarity calculation units of each definition clause, highlighting the application scenario of each clause. This invention achieves efficient and accurate vocabulary annotation, effectively improving the efficiency of vocabulary annotation in reading texts while reducing related manpower and financial investment.

[0104] Example 2

[0105] like Figure 5 As shown, this embodiment of the invention provides an automatic word tagging system based on syntactic and semantic analysis, comprising the following modules:

[0106] The module 61 for identifying new words to be annotated is used to select new words to be annotated based on the English proficiency and vocabulary of the target audience, and to define all sentences containing the new words to be annotated as new word text sentences;

[0107] The target word filtering module 62 is used to filter target words based on the part of speech of the new word to be tagged. If there are multiple target words, the process is transferred to the word meaning determination module. If the target word does not exist, a prompt is returned: No corresponding part of speech word, and the process is transferred to the prompt module. If there is only one target word, the word meaning of the new word to be tagged is determined based on the target word, and the process is transferred to the tagging module.

[0108] The module 63 for determining the meaning of new words to be annotated is used to calculate the similarity of the associated sentence vectors CosSim and the sentence co-occurrence similarity CooSim between the new word text sentence and the definition example sentences of each target word; the weighted sum of CosSim and CooSim is used to obtain the overall similarity Sim between the new word text sentence and each definition example sentence and sorted; according to the sorted Sim, the similarity AveSim between the new word text sentence and each target word is calculated and sorted in descending order to obtain AveSimList. If the difference between the maximum value and the second largest value of AveSimList is not greater than the threshold K, a prompt is returned: there are multiple possible words, and the process is transferred to the prompt module; otherwise, the meaning of the new word to be annotated is determined according to the word corresponding to the maximum value of AveSimList and the process is transferred to the annotation module.

[0109] The prompt module 64 is used for manual determination of the meaning of the new words to be marked based on different prompts;

[0110] The annotation module 65 is used to annotate the new words according to their meanings, including annotating their parts of speech, phonetic symbols, English definitions, and Chinese definitions.

[0111] The above embodiments are provided merely for the purpose of describing the present invention and are not intended to limit the scope of the invention. The scope of the invention is defined by the appended claims. Various equivalent substitutions and modifications made without departing from the spirit and principles of the invention should be covered within the scope of the invention.

Claims

1. A method for automatic word tagging based on syntactic and semantic analysis, characterized in that, include: Step S1: Based on the target audience's English proficiency and vocabulary list, select the new words to be annotated, and define all sentences containing the new words to be annotated as new word text sentences; Step S2: Filter target entries based on the part of speech of the new word to be tagged. If there are multiple target entries, proceed to step S3. If the target entry does not exist, return the prompt: No corresponding part of speech entry, and proceed to step S4. If there is only one target entry, determine the meaning of the new word to be tagged based on the target entry, and proceed to step S5. Step S3: Calculate the sentence vector similarity CosSim and sentence co-occurrence similarity CooSim between the new word text sentence and the definition example sentences of each target word; perform a weighted sum of CosSim and CooSim to obtain the overall similarity Sim between the new word text sentence and each definition example sentence, and sort them; according to the sorted Sim, calculate the similarity AveSim between the new word text sentence and each target word and sort them in descending order to obtain AveSimList. If the difference between the maximum value and the second largest value of AveSimList is not greater than the threshold K, return the prompt: there are multiple possible words, and go to step S4; otherwise, determine the meaning of the new word to be labeled according to the word corresponding to the maximum value of AveSimList and go to step S5. Step S4: The meaning of the new words to be annotated is determined manually based on different prompts; Step S5: Based on the meaning of the new word to be annotated, complete the annotation of the new word to be annotated, including annotating its part of speech, phonetic symbols, English definition, and Chinese definition.

2. The automatic word tagging method based on syntactic and semantic analysis according to claim 1, characterized in that, Step S1: Based on the target audience's English proficiency and vocabulary, select new words to be annotated, and define all sentences containing the new words as new word text sentences, specifically including: Step S11: Based on the target audience's English proficiency and vocabulary list, identify the new words to be annotated in the current text that are not in the vocabulary list, and add them to the list of new words to be annotated, NewWordList; Step S12: Define all sentences containing "newword" as new word text sentences and add them to the new word text sentence list NewSentenceList.

3. The automatic word tagging method based on syntactic and semantic analysis according to claim 2, characterized in that, Step S2: Filter target terms based on the part-of-speech of the word to be tagged. If multiple target terms exist, proceed to step S3. If no target term exists, return the message: No corresponding part-of-speech term, and proceed to step S4. If only one target term exists, determine the meaning of the word to be tagged based on the target term, and proceed to step S5, which specifically includes: Step S21: Perform syntactic component analysis on the new word text sentences in NewSentenceList to determine the component of the new word to be annotated in the new word text sentences, thereby determining the part-of-speech POS of the new word to be annotated, and storing the part-of-speech POS in WordPOS; Step S22: Based on the newword and its part-of-speech (POS), search for the target word in an English electronic dictionary containing example sentences. If multiple target words exist, proceed to step S3. If the target word does not exist, return the message "No corresponding part-of-speech word" and proceed to step S4. If there is only one target word, determine the meaning of the new word to be annotated based on the target word and proceed to step S5.

4. The automatic word tagging method based on syntactic and semantic analysis according to claim 3, characterized in that, Step S3: Calculate the sentence vector similarity CosSim and sentence co-occurrence similarity CooSim between the new word text sentence and the definition example sentences of each target word; perform a weighted sum of CosSim and CooSim to obtain the overall similarity Sim between the new word text sentence and each definition example sentence, and sort them; according to the sorted Sim, calculate the similarity AveSim between the new word text sentence and each target word and sort them in descending order to obtain AveSimList. If the difference between the maximum value and the second largest value of AveSimList is not greater than the threshold K, return a prompt: there are multiple possible words, and proceed to step S4; otherwise, determine the meaning of the new word to be labeled according to the word corresponding to the maximum value of AveSimList and proceed to step S5, which specifically includes: Step S31: Calculate the similarity CosSim between the related sentence vectors of the new word text sentence and the target word entry; Step S32: Calculate the sentence co-occurrence similarity (CooSim) between the new word text sentence and the target word entry; Step S33: Use formula (1) to perform a weighted summation of sentence vector similarity CosSim(s,e) and sentence co-occurrence similarity CooSim(s,e) to obtain the overall similarity Sim(s,e) between the new word text sentence and each definition example sentence of the target word entry; Sim(s,e)=aCosSim(s,e)+bCooSim(s,e) (1) Where s is the sentence of the new word text, e is the example sentence of the definition in the entry, and a and b are hyperparameters with values ​​in the range of 0-1; Step S34: Sort all examples in descending order according to their overall similarity to the sentence Sim(s,e), and store the sorting results in SimList; Step S35: Calculate the similarity between sentence and the target term entry, AveSim(s,entry): For the target term entry, select the first h example sentences from SimList and sum them using DCG formula (2), that is: Where r is the rank of the selected example sentence in SimList; The similarity between sentence and the target term entry, AveSim(s,entry), is calculated according to formula (3): Step S36: Sort the similarity AveSim(s,entry) between the sentence and each word in descending order and store it in AveSimList. If the difference between the maximum value and the second largest value in AveSimList is not greater than the threshold K, return a prompt: there are multiple possible words and go to step S4. Otherwise, determine the meaning of the new word to be labeled according to the word corresponding to the maximum value of AveSimList and go to step S5.

5. The automatic word tagging method based on syntactic and semantic analysis according to claim 4, characterized in that, Step S31: Calculate the similarity CosSim between the related sentence vectors of the new word text sentence and the target word entry, specifically including: Step S311: Based on NewSentenceList, use the target sentence vector calculation module to calculate the sentence vector Nsvec for each new word text sentence; Step S312: Based on the target term, construct a list of example sentences ExampleSentenceList, and use the target sentence vector calculation module to calculate the sentence vector Dsvec for each example sentence; Step S313: Calculate the similarity CosSim between Nsvec and Dsvec using cosine similarity, i.e., the similarity of the related sentence vectors: Where |.| is the length of the vector.

6. The automatic word tagging method based on syntactic and semantic analysis according to claim 5, characterized in that, The target sentence vector calculation module mentioned in step S311 specifically includes: Step S3111: Input the target sentence, delete stop words and function words in the target sentence, keep only content words, and use the BERT model to generate word vectors vec for each content word in the target sentence; Step S3112: For the target sentence containing "newword", perform a weighted summation of the word vectors (vec) of each content word in the target sentence except "newword": Where, weight n Let be the weight of the nth content word in the target sentence excluding newword, a be the smoothing parameter, and p(n) be the expected frequency of the nth content word in the current text. Step S3113: Calculate the sentence vector svec of the target sentence: Where |t| is the length of the target sentence excluding the newword, and weight n For the nth word vector vec n Corresponding weights.

7. The automatic word tagging method based on syntactic and semantic analysis according to claim 4, characterized in that, Step S32: Calculating the sentence co-occurrence similarity (CooSim) between the new word text sentence and the target word entry, specifically including: Step S321: Input the new word text sentence and all the definition example sentences of the target word, delete stop words and non-content words in the new word text sentence and the definition example sentences, and keep only the content words; Step S322: Input the new word text sentence after retaining the content words and the definition example sentence into the word co-occurrence similarity calculation module to obtain the word co-occurrence similarity WordSim of each content word in the two; Step S323: Assume s is the new word text sentence, e is the definition example sentence, r is the content word in the new word text sentence, and p is the content word in the definition example sentence; Let i = 0 in s and j = 0 in e. The co-occurrence similarity between content words in the new word text sentence and the definition example sentence is WordSim(s). i ,e j The maximum value is set to max[i] = 0, the maximum co-occurrence similarity is set to sum = 0; At the current positions i,j, if WordSim(s) i ,e j If ) is greater than max[i], then max[i] = WordSim(s) i ,e j ), j++; otherwise, j++; When j reaches the position of the last word p, i++, j = 0; When i reaches the position of the last word r, max[i] records the maximum co-occurrence similarity between content words in the definition example sentences corresponding to each content word in the new word text sentence; Let the current position in s be k = 0; At the current position k, sum = sum + max[k], k++; When k reaches the position of the last word r, sum records the sum of the maximum co-occurrence similarity between the new word text sentence and the definition example sentence, i.e. sum(s,e); Calculate the sentence co-occurrence similarity CooSim(s,e) between the new word text sentence and the definition example sentence: Where |s| is the sentence length after retaining content words.

8. The automatic word tagging method based on syntactic and semantic analysis according to claim 7, characterized in that, The word co-occurrence similarity calculation module mentioned in step S322 specifically includes: Step S3221: Input the definition examples of all entries in the selected English electronic dictionary, use them as corpus C, and perform text preprocessing, including: sentence segmentation, word segmentation, deletion of stop words or non-content words, and part-of-speech restoration; Step S3222: If a content word appears in corpus C, increment its frequency by 1, and count the frequency of each content word in corpus C; Step S3223: Define the sentence length as the co-occurrence window size. Two content words appearing in the same co-occurrence window are called co-occurring words. Increment the co-occurrence frequency by 1 and count the co-occurrence frequency between content words in corpus C. Step S3224: Assume f(w) is the frequency of the content word w in the corpus C, f(w i ,w j ) is a content word w i With content words w j The co-occurrence frequency is used to measure the co-occurrence distance between content words, dis(w). i ,w j ),Right now: Step S3226: Based on the co-occurrence distance dis(w) i ,w j WordSim(w) calculates the co-occurrence similarity between content words. i ,w j ),Right now:

9. The automatic word tagging method based on syntactic and semantic analysis according to claim 8, characterized in that, Step S4: The meaning of the new word to be annotated is determined manually based on different prompts, specifically including: Step S41: When the prompt is: no corresponding part of speech entry, the meaning of the new word to be annotated is determined manually; Step S42: When the prompt indicates that there are multiple possible entries, i.e., multiple entries with similarity AveSim(s,entry), select the definition example sentence MaxSentence with the highest overall similarity Sim(s,e) to the new word text sentence from each of the possible entries. Sort the entries from high to low according to the overall similarity Sim(s,e) of each MaxSentence sentence and store them in MaxList. Then, according to the order in MaxList, based on the MaxSentence example sentences in the similar entries, manually determine the meaning of the new word to be labeled.

10. An automatic word tagging system based on syntactic and semantic analysis, characterized in that, Includes the following modules: The module for identifying new words to be annotated is used to select new words to be annotated based on the English proficiency and vocabulary of the target audience, and to define all sentences containing the new words to be annotated as new word text sentences; The target term filtering module is used to filter target terms based on the part of speech of the new word to be tagged. If there are multiple target terms, the process is redirected to the word meaning determination module. If the target term does not exist, a prompt is returned: No corresponding part of speech term, and the process is redirected to the prompt module. If there is only one target term, the word meaning of the new word to be tagged is determined based on the target term, and the process is redirected to the tagging module. The module for determining the meaning of new words to be annotated is used to calculate the similarity of the associated sentence vectors CosSim and the sentence co-occurrence similarity CooSim between the new word text sentence and the definition example sentences of each target word; the CosSim and CooSim are weighted and summed to obtain the overall similarity Sim between the new word text sentence and each definition example sentence, and then sorted; according to the sorted Sim, the similarity AveSim between the new word text sentence and each target word is calculated and sorted in descending order to obtain AveSimList. If the difference between the maximum value and the second largest value of AveSimList is not greater than the threshold K, a prompt is returned: there are multiple possible words, and the process is transferred to the prompt module; otherwise, the meaning of the new word to be annotated is determined according to the word corresponding to the maximum value of AveSimList and the process is transferred to the annotation module. The prompting module is used for manual determination of the meaning of the new words to be marked based on different prompts; The annotation module is used to annotate the new words according to their meanings, including annotating their parts of speech, phonetic symbols, English definitions, and Chinese definitions.

Images