Contextual relationship-based and scenario-selectable english multiple-choice question generation method

Abstract

The application discloses a context relation-based and situational optional English single-choice question generation method, which comprises the following steps: extracting data from a data set, which is used for judging difficulty and vocabulary in the future; screening sentences from a corpus according to the vocabulary and the difficulty and the preset grammar; and generating appropriate options for the screened sentences and question blanks of specific grammar points which meet the vocabulary and the difficulty. The application can generate several English single-choice questions which meet a specific English situation through a given corpus of the specific English situation, and can be applied to the test question generation in the specific English teaching process, the question generation in the learning process of a robot, the question generation of a question and answer robot and the like, so that the repetitive work in the above situations can be reduced.

Description

Technical Field

[0001] This invention relates to a technology in the field of natural language processing, specifically a method for generating English multiple-choice questions based on context and with selectable scenarios. Background Technology

[0002] Multiple-choice questions in English have wide applications, such as English teaching, robotic learning, and question-answering robots. These applications all require generating appropriate questions based on specific scenarios, and then providing a suitable evaluation standard for the generated questions. The process of generating English multiple-choice questions is usually tedious and repetitive manual labor, leaving room for optimization through automated generation using other technologies. Summary of the Invention

[0003] To address the aforementioned shortcomings of existing technologies, this invention proposes a context-based and scenario-selectable English multiple-choice question generation method. This method can generate several English multiple-choice questions that fit a given specific English scenario corpus. It can be applied to scenarios such as test question generation in specific English teaching processes, question generation in robot learning processes, and question generation in question-answering robots, thereby reducing repetitive work in the above scenarios.

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a method for generating English multiple-choice questions based on context and with selectable scenarios. The method extracts the data base used to determine the difficulty level and vocabulary size from the dataset, then selects sentences from the corpus that meet the vocabulary size and difficulty level and are based on preset grammar. Finally, it generates appropriate alternatives for the selected sentences that meet the vocabulary size and difficulty level and their corresponding question blanks.

[0006] The difficulty level is determined by the following criteria: sentence length (Len), lowest frequency word ranking (Pos), total word meaning (Mea), component syntactic tree depth (Syn), dependency relation length (Dep), and perplexity (Per).

[0007] The filtering is achieved by calculating the Euclidean distance to the difficulty coefficient of a specific scenario.

[0008] The aforementioned options are determined by identifying a set of words closely related to the missing word based on context or dependency relationships. Word2vec is then used to find a set of words similar to the missing word from the corpus. Finally, a set of words closely related to the missing word and closely related to the missing word from the corpus is used as the single-choice option set.

[0009] The alternative options are preferably further modified by substituting each word in the single-choice alternative option set into the sentence of the original question and searching for similar usages in the corpus. If a similar usage is found, it is removed from the alternative option set, thereby avoiding multiple answer selections.

[0010] The aforementioned alternatives are preferably selected based on the difficulty evaluation dimension of the alternatives, and appropriate words are chosen as alternatives according to different needs.

[0011] This invention relates to a system for implementing the above method, comprising: a data preprocessing module, a question sentence acquisition module, and an alternative option generation module, wherein: the data preprocessing module performs statistical analysis on two dimensions, vocabulary size and difficulty, based on a given scenario; the question sentence acquisition module selects suitable question sentences based on the vocabulary size and difficulty dimensions obtained from data preprocessing and the required question; and the alternative option generation module generates alternative options using relevant natural language processing technologies while providing evaluation dimensions for the alternative options, allowing users to select appropriate alternative options according to their specific needs.

[0012] Technical effect

[0013] Compared to existing technologies, this invention defines appropriate difficulty classifications for specific corpus scenarios and, based on these classifications, extracts a large number of sentences of similar difficulty from a large corpus for subsequent question generation. This invention generates multiple-choice questions based on the contextual relationships between words in a sentence. The generated options are semantically similar to the original blanks, creating a deceptive effect without completely replacing them. Furthermore, it generates a large number of options and provides difficulty scoring criteria for each option, allowing for the generation of multiple questions for the same sentence based on varying difficulty levels. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the technical solution of an embodiment of the present invention;

[0015] Figure 2 This is a schematic diagram of the operation flow of an embodiment of the present invention;

[0016] Figure 3 This is a schematic diagram illustrating the principle of the alternative generation model in an embodiment of the present invention. Detailed Implementation

[0017] like Figure 1 As shown in this embodiment, a method for generating English multiple-choice questions based on context and with controllable difficulty is provided. This includes: generating multiple-choice questions based on prepositions and generating multiple-choice questions based on dependencies (excluding prepositions). The following uses the Shanghai Oxford Edition primary school English textbook (grades 1-5) as the selection scenario, and the specific steps include:

[0018] Step 1) Dataset preprocessing: Extracting the data foundation from the dataset for subsequent judgment of sentence difficulty and vocabulary size, specifically including:

[0019] 1.1) For elementary school textbooks, compile the vocabulary lists from grades one to five as a vocabulary set W1.

[0020] 1.2) Since the vocabulary list does not include all the words that fifth graders need to know, an expanded vocabulary set W2 is obtained from the Brown, BNC, and COCA corpora based on word frequency. For example, if the required vocabulary for fifth graders is 1500 words, the top 1500 words by frequency are counted for each of the three corpora, and the words that appear at least twice are taken out as the vocabulary set W2.

[0021] 1.3) The vocabulary set W for the fifth grade of elementary school is W1∪W2.

[0022] 1.4) The average value of the six difficulty dimensions of sentences in the fifth-grade textbook is used to judge the difficulty of the sentences, and the feature vector Y is obtained as Y = [Y1, Y2, Y3, Y4, Y5, Y6].

[0023] The six difficulty dimensions include: sentence length (Len), which is the count of all symbols after sentence segmentation; lowest frequency word ranking (Pos), which is the ranking of the lowest frequency word in the sentence within the corpus D; total semantic sum (Mea), which is the total semantic sum of each word in the sentence; constituent parse tree depth (Syn), which is the depth of the constituent parse tree corresponding to the sentence; dependency relation length and sum (Dep), which is the total length of all dependency chains in the dependency structure of the sentence; and perplexity (Per), which is the perplexity of the sentence and is correlated with the inverse of the probability model of the corresponding language model. Generally, the lower the probability of the language model of a sentence, the higher the perplexity, and the less frequently the sentence appears in normal language use, the more difficult the sentence is. Perplexity can be obtained through recurrent neural networks or some newer techniques such as BERT.

[0024] Step 2) Obtain a candidate set of problem sentences: For the large number of sentences in corpus D, filter out sentences that match the vocabulary size and difficulty level and are based on the preset grammar, specifically including:

[0025] 2.1) Calculate the sentence length L and the number of words in the segmented sentence in the required vocabulary W, and assume that... Sentences with threshold ≤ 1 are those that meet the required vocabulary size, while threshold is a custom threshold.

[0026] 2.2) Based on the average of the six difficulty dimensions in sentences from the fifth-grade textbook, the difficulty dimension of a sentence is calculated: A feature vector X = [X1, X2, X3, X4, X5, X6] is written for the difficulty dimension of a specific sentence. The feature vectors X and Y of all sentences are then standardized using z-score, specifically: Where: x is the individual observation value, μ is the mean of the population data, and σ is the standard deviation of the population data. The result has a mean of 0 and a standard deviation of 1.

[0027] 2.3) Calculate the Euclidean distance between feature vectors as a criterion for judging the similarity of sentence difficulty, specifically as follows: Sort all sentences in ascending order of Distance, and select the top sentences to represent the difficulty level. Here, X is the sentence difficulty coefficient, Y is the difficulty coefficient of the specific scenario, and the value of Top is customized as needed.

[0028] 2.4) Based on the specific grammar point, use regular expressions to filter out sentences S and the blank word that match the specific grammar from the sentences selected in the previous steps.

[0029] Step 3) Generation of Question Options: For a selected sentence S that meets the vocabulary and difficulty level requirements and specifies a particular grammar point, and the missing word "blank" in the question, generate appropriate options, including:

[0030] 3.1) Train the skip-gram model w2v in the word2vec model using corpus D. The trained w2v model can find several words in the training corpus that are most similar to a given word.

[0031] 3.2) Cases involving prepositional grammar points. The characteristics of a preposition and its most relevant words are often those in its context. Using w2v to find and blank the words before and after `pre` and `next`, the words with the same part of speech but different word prototypes are `pre_sim` and `next_sim`.

[0032] 3.3) Due to the inherent characteristics of single-choice questions, the answer choices are generally words with the same part of speech; that is, when "blank" is a preposition, the answer choice is also a preposition. This method statistically searches the corpus D for the set B of prepositions "select" that appear most frequently in the context of "pre_sim" and "next_sim".

[0033] 3.4) Cases involving grammatical points other than prepositions. The word most relevant to the blank may not be the preceding or following word. Perform dependency parsing on the sentence to find the set of words that blank depends on, `dep`, and the set of words that depend on blank, `deped` (where `dep` and `deped` may be empty sets).

[0034] 3.5) Use w2v to find the word set dep_sim and word set deped_sim that are similar to dep and deped.

[0035] 3.6) Similar to the above, in cases other than prepositions, the answer choices are mostly words with the same part of speech. This method statistically searches the corpus for a set B of select words with the same part of speech that have the most frequent occurrences and dependencies on dep_sim and deped_sim.

[0036] 3.7) The generated alternative set B may contain incorrect alternatives, that is, alternatives that still hold true when the original sentence is returned. Therefore, it is necessary to resubmit the alternative and the context of blank (if blank is a preposition) or the set of dependent words of blank (if blank is not a preposition) into D for retrieval. If there is a case that satisfies the context or dependency relationship, the alternative is removed from the alternative set B.

[0037] 3.8) The scoring criteria for calculating the options include: Similarity (sim): The degree of similarity between the option and blank under w2v. A larger Sim value indicates greater similarity and a more difficult option; Edit Distance (Edi): The distance between the option and blank after Edi steps of insertion, deletion, and replacement. A smaller Edi value indicates easier conversion and a more difficult option; Frequency (Fre): The number of times the option appears in the corpus D when searching for context or dependency relationships. A larger Fre value indicates more frequent occurrence and a more difficult option; Word Frequency Rank (Pos): The word frequency rank of the option in the corpus D. A larger Pos value indicates a more uncommon word and a more difficult option; Sentence Perplexity (Per): The perplexity of the option after substituting it into the original sentence. A smaller Per value indicates a more normal language fit and a more difficult option; Lexicon (In): Whether the option is in the required vocabulary W. 1 if in, 0 if not. Options not in the vocabulary are more difficult.

[0038] 3.9) For the scoring criteria, z-score standardization is applied. Taking the inverse of Edi, In, and Per results in all indicators being equal; the higher the value, the more difficult the option. All scores are summed; higher scores indicate a higher overall difficulty of the option. For the In scoring criterion, options must strictly be within the vocabulary list; options with an In value of 0 are not included in the final selection. The final suitable option is selected based on the specific needs using the option scoring criteria. For example, in the most difficult case, the highest-scoring option is selected; in the simplest case, the lower-scoring option is selected.

[0039] Through specific practical experiments, in the specific context of the "Shanghai Oxford Edition Primary School English Textbook", the resulting vocabulary list is as follows:

[0040]

[0041] Difficulty dimension is

[0042]

[0043]

[0044] Sentences that match the vocabulary and difficulty level of a fifth-grade student and are found by using "of" as the question word.

[0045]

[0046] With "Erm and thenjustgenerallylanguage is far more than just a means of

[0047] "communication." is the question sentence, with "of" as the question blank. The specific experimental data that can be obtained is:

[0048] pre pre_sim next next_sim means suppose communication interaction

[0049] The following options were obtained, where "by" represents the option that was removed from the example (the excluded sentence is "But what is best by communication?"). The higher the score, the more difficult the option is.

[0050]

[0051]

[0052] Using "The meaning of words changes every fifty years." as the question sentence, and "changes" as the blank, the specific experimental data that can be obtained is:

[0053] dep pre_sim deped deped_sim meaning definition years month

[0054] The following options are provided, with higher scores indicating greater difficulty for each option.

[0055] Sim Edi Fre Pos Per in Score is 0.22 6 3564 2 158.92 1 0.32 have 0.34 3 1178 8 250.03 1 -3.08 end 0.20 5 461 647 288.63 1 0.80 see 0.39 5 334 51 400.16 1 0.82 work 0.41 6 269 129 300.75 1 1.49 begin 0.37 5 266 214 352.47 1 0.90 know 0.29 5 197 52 441.01 1 -0.03 make 0.33 4 184 46 417.44 1 -1.02 use 0.51 5 180 92 441.96 1 2.74 come 0.33 4 139 66 234.52 1 -2.93

[0056] The above method selects sentences for appropriate scenarios based on vocabulary size and difficulty. It obtains the vocabulary size and difficulty dimension of a given scenario through data preprocessing, thereby obtaining question sentences that better meet the needs of the scenario. It generates alternatives based on context or dependency relationships and provides an evaluation of the alternatives. It can generate confusing alternatives that are closely related to the context and select appropriate alternatives according to needs.

[0057] In summary, this method can generate several questions that fit a specific context. It also generates multiple options and provides objective difficulty assessment criteria for these options, allowing for the selection of appropriate options to generate suitable questions for the same statement based on different needs.

[0058] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A context relationship-based and situational optional English single-choice question generation method, characterized in that, Specifically comprising: Step 1) data set preprocessing, extracting data from the data set for subsequent judgment of sentence difficulty and vocabulary, specifically comprising: 1.1) for primary school textbooks, statistics of words from first to fifth grade are taken as the vocabulary set W1; 1.2) since the word list does not contain all the words that need to be recognized by the fifth grade, the expanded vocabulary set W2 is obtained from the Brown, BNC and COCA corpora according to the word frequency; such as the required vocabulary of the fifth grade is 1500, the word frequency of the top 1500 words in the three corpora is counted, and at least the words appearing twice are taken as the vocabulary set W2; 1.3) the vocabulary set W of the fifth grade is W1∪W2; 1.4) the average value of the six difficulty dimension information of the sentences in the fifth grade textbook is calculated for subsequent judgment of the difficulty of the sentence, and the characteristic vector Y is Y=[Y1, Y2, Y3, Y4, Y5, Y6]; Step 2) obtain the question sentence candidate set: for a large number of sentences in the corpus D, filter out the sentences that meet the vocabulary and difficulty for the preset grammar, specifically comprising: 2.1) Calculate the sentence length L and the number of words num in the sentence which are in the required vocabulary W, and consider >threshold, threshold≦1 as a sentence which meets the required vocabulary, and threshold is a self-defined threshold value; 2.2) Calculate the difficulty dimension of the sentence according to the average value of the six difficulty dimensions in the sentence in the fifth-grade textbook: the difficulty dimension of a specific sentence is written as a feature vector X = [X1, X2, X3, X4, X5, X6], and the feature vectors X and Y of all sentences are z-score standardized, specifically: z = x - μ / σ wherein: x is the observation value of the individual, μ is the mean of the population data, σ is the standard deviation of the population data, the mean of the result is 0, and the standard deviation is 1; 2.3) Calculate the Euclidean distance between the feature vectors as the judgment standard of sentence difficulty degree similarity, specifically: Distance(X, Y) = Sort all sentences according to Distance in ascending order, and take the top Top sentences as the sentences that meet the difficulty degree, wherein: X is the sentence difficulty coefficient, Y is the difficulty coefficient of a specific scenario; The value of Top is defined by the user according to the situation. 2.4) according to the specific grammar point, the regular expression is used to filter out the sentences S and the problem blank from the sentences filtered out in the previous step that meet the specific grammar; Step 3) generation of question options: for a sentence S and a problem blank that meet the vocabulary and difficulty of a specific grammar point, generate appropriate options, specifically comprising: 3.1) use the corpus D to train the skip-gram model w2v in the word2vec model; using the trained w2v model can find out several words that are most similar to a word in the training corpus; 3.2) the case of preposition grammar point; the characteristics of preposition itself are most related to the context words; using w2v to find the most similar words pre_sim and next_sim to pre and next of blank, which have the same part of speech and different word prototypes; 3.3) due to the characteristics of single selection itself, the options are words with the same part of speech, that is, when blank is a preposition, the answer options are also prepositions; this method finds the set B of select prepositions that appear most frequently in the context of pre_sim and next_sim from the corpus D; 3.4) the case of grammar points other than prepositions; the word most related to the blank may not be the pre and next words; perform dependency grammar analysis on the sentence, find the blank dependent word set dep and the blank dependent word set deped; 3.5) use w2v to find the word set dep_sim and the word set deped_sim similar to dep and deped; 3.6) The same word with the same part of speech is the most common word in the corpus with the same dependency relationship as dep_sim and deped_sim. The method searches for the most common word with the same part of speech in the corpus with the same dependency relationship as dep_sim and deped_sim. 3.7) The generated alternative set B may have incorrect alternatives, that is, alternatives that still hold true when substituted back into the original sentence. Therefore, the alternative and the context of the blank, that is, the blank is a preposition or the dependency word set of the blank, is re-substituted into D to retrieve if there is a context or dependency relationship that meets the requirements. If there is a context or dependency relationship that meets the requirements, the alternative is removed from the alternative set B. 3.8) Calculate the scoring criteria for the alternatives, including: similarity sim: the similarity of the alternative and the blank in w2v; the greater the value of sim, the more similar the alternative; edit distance Edi: the alternative can be converted into blank after Edi steps of insertion, deletion, and replacement; the smaller the Edi, the easier the conversion, and the more difficult the alternative; frequency Fre: the number of times the alternative appears when searching for the context or dependency relationship in the corpus D; the greater the Fre, the more common the alternative; word frequency ranking Pos: the word frequency ranking of the alternative in the corpus D; the greater the Pos, the more rare the word, and the more difficult the alternative; sentence perplexity Per: the perplexity of the alternative when substituted into the original sentence; the smaller the Per, the more consistent with normal language, and the more difficult the alternative; whether in the vocabulary In: whether the alternative is in the required vocabulary W; 1 if it is, and 0 if it is not; alternatives not in the vocabulary are difficult; 3.9) Standardize all scoring criteria with z-score, and take the opposite number of Edi, In, and Per, so that all indicators represent that the greater the alternative, the more difficult it is; add up all the scores, and the higher the score, the higher the overall difficulty of the alternative; for the scoring criteria In, which requires that the alternative be in the vocabulary, alternatives with In = 0 are not included in the final candidate; select the appropriate alternative according to the specific requirements of the alternative scoring criteria; for example, the most difficult case selects the highest score, and the simple case selects the lower score.

2. The method according to claim 1, wherein, The six difficulty dimension information includes: sentence length Len, which is the count of all tokens after sentence segmentation; the lowest frequency word ranking Pos, which is the ranking of the lowest frequency word in the corpus D; the total word meaning Mea, which is the total word meaning of each word in the sentence; the constituent syntax analysis tree depth Syn, which is the depth of the constituent syntax analysis tree corresponding to the sentence; the dependency relationship length and Dep, which is the total length of all dependency chains in the dependency structure of the sentence; and the perplexity Per, which is the perplexity corresponding to the sentence, related to the inverse of the language model probability model corresponding to the sentence.

3. A context relation and situational optional English single-choice question generation system for implementing the method of claim 1 or 2, characterized in that, including: The data preprocessing module, the question sentence acquisition module and the candidate option generation module, wherein: the data preprocessing module carries out statistics on the two dimensions of vocabulary and difficulty according to a given scenario, the question sentence acquisition module screens appropriate question sentences according to the dimensions of vocabulary and difficulty obtained by data preprocessing and a required question, and the candidate option generation module uses the related technology of natural language processing to generate candidate options and give the evaluation dimensions of the candidate options, so that the user selects appropriate candidate options according to specific requirements.

4. The system of claim 3, wherein, The difficulty degree is determined by the following criteria: sentence length Len, minimum frequency word ranking Pos, word meaning sum Mea, component syntax analysis tree depth Syn, dependency relationship length and Dep, and perplexity Per.

5. The system of claim 3, wherein, The screening is realized by calculating the Euclidean distance of the difficulty coefficient of a specific scenario.

6. The system of claim 3, wherein, The candidate options are found out according to the context relationship or the dependency relationship, a word set closely related to the blank word is found out, a word set similar to the word set is found out from a corpus by using word2vec, and a word set similar to the blank word and closely related to the similar word set is found out from the corpus as the single-choice candidate option set.

7. The system of claim 3, wherein, The candidate options are further substituted into the original question sentence, and it is found out from the corpus whether there is a similar usage, if there is, the similar usage is deleted from the candidate option set, so as to avoid multiple-choice answers.

8. The system of claim 3, wherein, The candidate options are selected from the candidate options according to different needs by the difficulty evaluation dimension of the candidate options, and appropriate words are selected as the candidate options.

Images