A multiple-feature fusion-based selection question retrieval method and retrieval system

Abstract

The application discloses a multiple-feature fusion-based selection question retrieval method and a retrieval system, wherein the retrieval method comprises the following steps: calculating fusion features of a to-be-retrieved question; calculating the similarity between the fusion features of the to-be-retrieved question and the fusion features of questions in a question bank; and selecting a question with the maximum similarity or a question with a similarity greater than a similarity threshold as a retrieval result. The method extracts the features of the stem, options and user answers of the selection question through natural language analysis, a knowledge graph, text analysis and the like, and fuses the extracted multiple features, so that the finally extracted features can reflect more information of the question, and the question can be positioned more accurately, and an accurate retrieval result can be obtained.

Description

Technical Field

[0001] This invention belongs to the field of question retrieval technology, specifically relating to a multiple-choice question retrieval method and system based on multi-feature fusion. Background Technology

[0002] In the field of online education, question banks are typically established for each subject. Maintaining and supplementing these question banks involves scenarios such as checking if a question already exists in the bank, identifying similar questions, or recommending similar questions to users for a given question. These applications require the ability to quickly retrieve or locate similar questions to support subsequent processing.

[0003] Currently, question retrieval can be roughly categorized as follows:

[0004] The first type involves manually maintaining the question bank beforehand, tagging each question with a relevant knowledge point, and then retrieving questions by matching input keywords. This method requires a significant amount of manual work, is inefficient, and is highly dependent on the individual skill level of the operator.

[0005] The second type uses natural language processing methods from artificial intelligence to process the question text, extract text features, and retrieve similar questions based on the similarity between these features. This method relies solely on the similarity between text features to determine question similarity, which is too crude. If questions describe the same knowledge point using different wording, it may lead to incorrect judgments.

[0006] Another approach is to construct the question stem into an original syntax tree and then perform similarity comparisons based on the syntax tree. This approach also has significant room for optimization. Summary of the Invention

[0007] Purpose of the invention: To address the problems existing in the prior art, the present invention provides a multiple-choice question retrieval method based on multi-feature fusion, which can obtain more accurate multiple-choice question retrieval results.

[0008] Technical solution: This invention discloses a multiple-choice question retrieval method based on multi-feature fusion, including the following steps:

[0009] Calculate the fusion features of the item to be retrieved;

[0010] Calculate the similarity between the fusion features of the question to be retrieved and the fusion features of the questions in the question bank;

[0011] The search results are selected from the questions with the highest similarity or those with a similarity greater than the similarity threshold.

[0012] Furthermore, calculating the fusion features of the item to be retrieved specifically includes the following steps:

[0013] S1. Identify the characteristics of the question stem, specifically including:

[0014] (11) The original question stem is obtained by removing stop words and punctuation marks from the question stem;

[0015] (12) Extract sentence features from the original question stem;

[0016] (13) Extract the knowledge point features from the original question stem;

[0017] (14) Extract the part-of-speech features and syntactic dependency features of the original question stem;

[0018] (15) Add the sentence features, knowledge point features, part-of-speech features and syntactic dependency features of the original question stem to obtain the question stem features;

[0019] S2. Obtain the characteristics of the answer choices, specifically including:

[0020] (21) Remove stop words and punctuation marks from each option in the question to obtain the corresponding original options;

[0021] (22) Extract sentence features, knowledge point features, part-of-speech features and syntactic dependency features of each original option;

[0022] (23) Extract the relationship characteristics between each original option and the question stem;

[0023] (24) By combining the sentence features, knowledge point features, part-of-speech features, syntactic dependency features, and relationship features with the question stem of each original option, the option features of the question are obtained;

[0024] S3. Extract user answer features, specifically including:

[0025] (31) Obtain the average answering time and average answering accuracy of the questions;

[0026] (32) Obtain common incorrect answers and correct answers, and extract the relationship features between common incorrect answers and correct answers and the question stem;

[0027] (33) Combine the average answering time, average answering accuracy, relationship between common wrong answers and question stems, and relationship between correct answers and question stems into user answering characteristics;

[0028] S4. Combine the stem features, option features, and user answer features of the question to be retrieved to form the fusion features of the question to be retrieved.

[0029] Furthermore, the extraction of sentence features from the original question stem employs one of the following two methods:

[0030] Method 1: Use the word2vec model to obtain word vectors from the original question stem, and use the word vectors as sentence features of the original question stem;

[0031] Method 2: Use the BERT model to encode the original question stem, and use the resulting encoding as the sentence feature of the original question stem.

[0032] Furthermore, step (13) extracts the knowledge point features of the original question stem using the following steps:

[0033] a1. Extract the knowledge points from the original question stem: Compare the original question stem with the knowledge point database established by experts. If the knowledge points in the knowledge point database appear in the original question stem, extract them to form the first set of knowledge points in the original question stem.

[0034] a2. Search for knowledge points in the first knowledge point set within the triplet set of the knowledge graph to form the second knowledge point set; the distance between the knowledge points in the second knowledge point set and the knowledge points in the first knowledge point set is less than the first distance threshold.

[0035] The knowledge graph is represented by a set of triples, each triple including a first entity, a second entity, and the relationship between the first and second entities; the first and second entities are both knowledge points in the knowledge point base, and the relationship between the first and second entities is set by experts.

[0036] a3. Based on the search results of step a2, obtain the knowledge graph triplets involved in the knowledge points in the second knowledge point set, and convert them into vector representations as the knowledge point features of the original question stem.

[0037] Furthermore, in step a2, the distance between the knowledge points in the second knowledge point set and the knowledge points in the first knowledge point set is Euclidean distance, text editing distance, or Manhattan distance.

[0038] Furthermore, in step (14), the original extracted part-of-speech and syntactic dependency analysis results are first obtained using spacy, and then converted into vectors using the BERT model, which serve as the part-of-speech features and syntactic dependency features of the original stem.

[0039] Furthermore, step (23) extracts the relationship features between the original options and the question stem, specifically including:

[0040] b1. Extract the knowledge points from the original options;

[0041] b2. Find the triplet where the first entity and the second entity are the knowledge points of the original question stem and the knowledge points of the original options, obtain the relationship between the knowledge points of the original question stem and the knowledge points of the original options, and convert it into a vector representation as the relationship feature between the original options and the question stem.

[0042] Furthermore, the similarity is calculated as follows:

[0043] Multiple distance algorithms are used to calculate the distance similarity between the fusion features of the question to be retrieved and the fusion features of the questions in the question bank;

[0044] The harmonic average of multiple distance similarities is calculated and used as the similarity between the fusion feature of the question to be retrieved and the fusion feature of the questions in the question bank.

[0045] On the other hand, the present invention also discloses a retrieval system for implementing the above-mentioned multiple-choice question retrieval method based on multi-feature fusion, comprising:

[0046] The question's fusion feature calculation module is used to calculate the fusion features of the question.

[0047] The similarity calculation module is used to calculate the similarity between the fusion features of the question to be retrieved and the fusion features of the questions in the question bank;

[0048] The search result acquisition module is used to select the question with the highest similarity or the question with a similarity greater than the similarity threshold as the search result.

[0049] Furthermore, the fusion feature calculation module for the question includes:

[0050] The question stem feature extraction module is used to extract the question stem features from the questions;

[0051] The question option feature acquisition module is used to extract the features of the question options;

[0052] The user answer feature acquisition module is used to extract user answer features for questions;

[0053] The feature fusion module is used to combine the features of the question stem, the options, and the user's answer to form the fusion features of the question to be retrieved.

[0054] Beneficial effects: Compared with the prior art, the multiple-choice question retrieval method and retrieval system disclosed in this invention extracts the features of the question stem, options and user answers of multiple-choice questions through natural language analysis, knowledge graphs and text analysis, and fuses the extracted features so that the final extracted features can reflect more information about the question, thereby achieving more accurate question positioning and obtaining accurate retrieval results. Attached Figure Description

[0055] Figure 1 This is a flowchart of the multiple-choice question retrieval method based on multi-feature fusion disclosed in this invention;

[0056] Figure 2 A flowchart for calculating the fusion features of the problem;

[0057] Figure 3 This is a schematic diagram of the composition of the multiple-choice question retrieval system based on multi-feature fusion disclosed in this invention. Detailed Implementation

[0058] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0059] This invention discloses a multiple-choice question retrieval method based on multi-feature fusion, such as... Figure 1 As shown, it includes the following steps:

[0060] Calculate the fusion features of the item to be retrieved;

[0061] Calculate the similarity between the fusion features of the question to be retrieved and the fusion features of the questions in the question bank;

[0062] The search results are selected from the questions with the highest similarity or those with a similarity greater than the similarity threshold.

[0063] In this invention, the stem, options, and user answers of multiple-choice questions are thoroughly analyzed, and the features of each part are extracted to form a fused feature. Specifically, for example... Figure 2 As shown, calculating the fusion features of the question to be retrieved includes the following steps:

[0064] S1. Identify the characteristics of the question stem, specifically including:

[0065] (11) Remove stop words and punctuation marks from the question stem to obtain the original question stem; then use the three methods in (12)-(14) to analyze the original question stem and extract its features;

[0066] (12) Extract sentence features from the original question stem;

[0067] To process the complete original question stem and obtain sentence features, the following two methods can be used:

[0068] Method 1: Use the word2vec model to obtain word vectors from the original question stem, and use the word vectors as sentence features of the original question stem;

[0069] Method 2: Use the BERT model to encode the original question stem, and use the resulting encoding as the sentence feature of the original question stem.

[0070] The sentence features reflect the overall textual features of the question stem.

[0071] (13) Extract the knowledge point features from the original question stem, specifically:

[0072] a1. Extract the knowledge points from the original question stem: Compare the original question stem with the knowledge point database established by experts. If the knowledge points in the knowledge point database appear in the original question stem, extract them to form the first set of knowledge points in the original question stem.

[0073] The original knowledge points in the first set of knowledge points are those knowledge points that are explicitly mentioned in the question stem;

[0074] a2. Search for knowledge points in the first knowledge point set within the triplet set of the knowledge graph to form the second knowledge point set; the distance between the knowledge points in the second knowledge point set and the knowledge points in the first knowledge point set is less than the first distance threshold.

[0075] The knowledge graph is represented by a set of triples, each triple including a first entity, a second entity, and the relationship between the first and second entities; the first and second entities are both knowledge points in the knowledge point base, and the relationship between the first and second entities is set by experts.

[0076] The search for knowledge points in the first knowledge point set within the triplet set of the knowledge graph is performed by calculating distance. Specifically, for each knowledge point in the first knowledge point set, a search is conducted within the triplet set for knowledge points whose distance to it is less than a preset threshold. These found knowledge points form the second knowledge point set. In this embodiment, Euclidean distance, text editing distance, or Manhattan distance are used to calculate the distance between knowledge points. The second knowledge point set is an extension of the first knowledge point set, thereby revealing knowledge points that are not explicitly stated in the question stem, i.e., knowledge points that may be present in the question stem.

[0077] a3. Based on the search results of step a2, obtain the knowledge graph triplets involved in the knowledge points in the second knowledge point set, and convert them into vector representations as the knowledge point features of the original question stem.

[0078] Step a2 yields all possible knowledge points in the question stem, and step a3 yields the relevant knowledge points in the question stem, as well as the relationship between the possible knowledge points and the relevant knowledge points. These contents are represented by triples, and when converted into vectors, they become the knowledge point features of the original question stem.

[0079] (14) Extract the part-of-speech features and syntactic dependency features of the original question stem;

[0080] First, spacy is used to obtain the original extracted part-of-speech and syntactic dependency analysis results, and then the BERT model is used to convert them into vectors, which serve as the part-of-speech features and syntactic dependency features of the original question stem.

[0081] (15) Add the sentence features, knowledge point features, part-of-speech features and syntactic dependency features of the original question stem to obtain the question stem features, that is, add the multiple features extracted in steps (12)-(14) together as the question stem features.

[0082] S2. Obtain the characteristics of the answer choices, specifically including:

[0083] (21) Remove stop words and punctuation marks from each option in the question to obtain the corresponding original options;

[0084] (22) Extract sentence features, knowledge point features, part-of-speech features and syntactic dependency features of each original option;

[0085] The steps (21) and (22) are the same as the steps (11)-(14) for processing the question stem, and will not be repeated here.

[0086] (23) Extract the relationship characteristics between each original option and the question stem;

[0087] For a given option in a multiple-choice question, extracting its relationship features with the question stem specifically includes:

[0088] b1. Extract the knowledge points from the original options;

[0089] Similar to step a1, extract the knowledge points from the original options and compare them with the knowledge point database established by experts to obtain the relevant information.

[0090] b2. Find the triplet of the knowledge points in the original question stem and the knowledge points in the original options, which is the first entity and the second entity. Obtain the relationship between the knowledge points in the original question stem and the knowledge points in the original options, and convert it into a vector representation as the relationship feature between the original options and the question stem. In this way, the association between the options and the question stem is established.

[0091] (24) By combining the sentence features, knowledge point features, part-of-speech features, syntactic dependency features, and relationship features with the question stem of each original option, the option features of the question are obtained;

[0092] S3. Extract user answer features, specifically including:

[0093] (31) Obtain the average answering time and average answering accuracy of the questions;

[0094] (32) Obtain common incorrect answers and correct answers, and extract the relationship features between common incorrect answers and correct answers and the question stem;

[0095] (33) Combine the average answering time, average answering accuracy, relationship between common wrong answers and question stems, and relationship between correct answers and question stems into user answering characteristics;

[0096] S4. Combine the stem features, option features, and user answer features of the question to be retrieved to form the fusion features of the question to be retrieved.

[0097] After the above steps S1-S4, the fusion features of the question to be retrieved are obtained. Each question in the question bank is processed according to the above steps to obtain the fusion features of the questions in the question bank; then, the similarity between the fusion features of the question to be retrieved and the fusion features of the questions in the question bank is calculated. Similarity can be expressed as distance similarity between features or cosine similarity between vectors. In this embodiment, the similarity calculation method is as follows:

[0098] Multiple distance algorithms are used to calculate the distance similarity between the fusion features of the question to be retrieved and the fusion features of the questions in the question bank;

[0099] The harmonic average of multiple distance similarities is calculated and used as the similarity between the fusion feature of the question to be retrieved and the fusion feature of the questions in the question bank.

[0100] In this embodiment, Euclidean distance similarity, text edit distance similarity, and Manhattan distance similarity are used as multiple distance similarity measures. Specifically, the Euclidean distance similarity s between the fused features of the question to be retrieved and the fused features of the questions in the question bank is calculated. a Text edit distance similarity s b Similarity to Manhattan distance s c Its harmonic mean is: The resulting 's' represents the similarity between the fusion features of the question to be retrieved and the fusion features of questions in the question bank. The question with the highest similarity to the question to be retrieved, or the question with a similarity threshold, is selected as the search result.

[0101] A retrieval system that implements the above-mentioned multiple-choice question retrieval method based on multi-feature fusion, such as... Figure 3 As shown, it includes:

[0102] The question's fusion feature calculation module 1 is used to calculate the fusion features of the question.

[0103] Similarity calculation module 2 is used to calculate the similarity between the fusion features of the question to be retrieved and the fusion features of the questions in the question bank;

[0104] The search result acquisition module 3 is used to select the question with the highest similarity or the question with similarity greater than the similarity threshold as the search result.

[0105] The fusion feature calculation module 1 for the question specifically includes:

[0106] The question stem feature acquisition module 11 is used to extract question stem features from the questions.

[0107] The question option feature acquisition module 12 is used to extract the features of the question options;

[0108] User answer feature acquisition module 13 is used to extract user answer features of questions;

[0109] The feature fusion module 14 is used to combine the stem features, option features, and user answer features to form the fusion features of the question to be retrieved.

Claims

1. A multiple-choice question retrieval method based on multi-feature fusion, characterized in that, Including the following steps: Calculate the fusion features of the item to be retrieved; The specific steps include: S1. Identify the characteristics of the question stem, specifically including: (11) The original question stem is obtained by removing stop words and punctuation marks from the question stem; (12) Extract sentence features from the original question stem; use one of the following two methods: Method 1: Use the word2vec model to obtain word vectors from the original question stem, and use the word vectors as sentence features of the original question stem; Method 2: Use the BERT model to encode the original question stem, and use the resulting encoding as the sentence feature of the original question stem; (13) Extract the knowledge point features from the original question stem; including: a1. Extract the knowledge points from the original question stem: Compare the original question stem with the knowledge point database established by experts. If the knowledge points in the knowledge point database appear in the original question stem, extract them to form the first set of knowledge points in the original question stem. a2. Search for knowledge points in the first knowledge point set within the triplet set of the knowledge graph to form the second knowledge point set; the distance between the knowledge points in the second knowledge point set and the knowledge points in the first knowledge point set is less than the first distance threshold. The knowledge graph is represented by a set of triples, each triple including a first entity, a second entity, and the relationship between the first and second entities; the first and second entities are both knowledge points in the knowledge point base, and the relationship between the first and second entities is set by experts. a3. Based on the search results of step a2, obtain the knowledge graph triplets involved in the knowledge points in the second knowledge point set, and convert them into vector representations as the knowledge point features of the original question stem; (14) Extract the part-of-speech features and syntactic dependency features of the original question stem; specifically, first use spacy to obtain the original extracted part-of-speech and syntactic dependency analysis results, and then use the BERT model to convert them into vectors as the part-of-speech features and syntactic dependency features of the original question stem; (15) Add the sentence features, knowledge point features, part-of-speech features and syntactic dependency features of the original question stem to obtain the question stem features; S2. Obtain the characteristics of the answer choices, specifically including: (21) Remove stop words and punctuation marks from each option in the question to obtain the corresponding original options; (22) Extract sentence features, knowledge point features, part-of-speech features and syntactic dependency features from each original option; (23) Extract the relationship features between each original option and the question stem; specifically including: b1. Extract the knowledge points from the original options; b2. Find the triplet where the first entity and the second entity are the knowledge points of the original question stem and the knowledge points of the original options, obtain the relationship between the knowledge points of the original question stem and the knowledge points of the original options, and convert it into a vector representation as the relationship feature between the original options and the question stem; (24) By combining the sentence features, knowledge point features, part-of-speech features, syntactic dependency features, and relationship features with the question stem of each original option, the option features of the question are obtained; S3. Extract user answer features, specifically including: (31) Obtain the average answering time and average answering accuracy of the questions; (32) Obtain common incorrect answers and correct answers, and extract the relationship features between common incorrect answers and correct answers and the question stem; (33) Combine the average answering time, average answering accuracy, relationship characteristics between common wrong answers and question stems, and relationship characteristics between correct answers and question stems to form user answering characteristics; S4. Combine the stem features, option features, and user answer features of the question to be retrieved to form the fusion features of the question to be retrieved. Calculate the similarity between the fusion features of the question to be retrieved and the fusion features of the questions in the question bank; The search results are selected from the questions with the highest similarity or those with a similarity greater than the similarity threshold.

2. The multiple-choice question retrieval method based on multi-feature fusion according to claim 1, characterized in that, In step a2, the distance between the knowledge points in the second knowledge point set and the knowledge points in the first knowledge point set is calculated using Euclidean distance, text editing distance, or Manhattan distance.

3. The multiple-choice question retrieval method based on multi-feature fusion according to claim 1, characterized in that, The similarity is calculated as follows: Multiple distance algorithms are used to calculate the distance similarity between the fusion features of the question to be retrieved and the fusion features of the questions in the question bank; The harmonic average of multiple distance similarities is calculated and used as the similarity between the fusion feature of the question to be retrieved and the fusion feature of the questions in the question bank.

4. A multiple-choice question retrieval system based on multi-feature fusion, characterized in that, include: The problem's fusion feature calculation module is used to calculate the fusion features of the problem; it includes: The question stem feature extraction module is used to extract the features of the question stem; specifically, it includes: (11) The original question stem is obtained by removing stop words and punctuation marks from the question stem; (12) Extract sentence features from the original question stem; use one of the following two methods: Method 1: Use the word2vec model to obtain word vectors from the original question stem, and use the word vectors as sentence features of the original question stem; Method 2: Use the BERT model to encode the original question stem, and use the resulting encoding as the sentence feature of the original question stem; (13) Extract the knowledge point features from the original question stem; including: a1. Extract the knowledge points from the original question stem: Compare the original question stem with the knowledge point database established by experts. If the knowledge points in the knowledge point database appear in the original question stem, extract them to form the first set of knowledge points in the original question stem. a2. Search for knowledge points in the first knowledge point set within the triplet set of the knowledge graph to form the second knowledge point set; the distance between the knowledge points in the second knowledge point set and the knowledge points in the first knowledge point set is less than the first distance threshold. The knowledge graph is represented by a set of triples, each triple including a first entity, a second entity, and the relationship between the first and second entities; the first and second entities are both knowledge points in the knowledge point base, and the relationship between the first and second entities is set by experts. a3. Based on the search results of step a2, obtain the knowledge graph triplets involved in the knowledge points in the second knowledge point set, and convert them into vector representations as the knowledge point features of the original question stem; (14) Extract the part-of-speech features and syntactic dependency features of the original question stem; specifically, first use spacy to obtain the original extracted part-of-speech and syntactic dependency analysis results, and then use the BERT model to convert them into vectors as the part-of-speech features and syntactic dependency features of the original question stem; (15) Add the sentence features, knowledge point features, part-of-speech features and syntactic dependency features of the original question stem to obtain the question stem features; The question option feature acquisition module is used to extract the features of the question options; specifically, it includes: (21) Remove stop words and punctuation marks from each option in the question to obtain the corresponding original options; (22) Extract sentence features, knowledge point features, part-of-speech features and syntactic dependency features from each original option; (23) Extract the relationship features between each original option and the question stem; specifically including: b1. Extract the knowledge points from the original options; b2. Find the triplet where the first entity and the second entity are the knowledge points of the original question stem and the knowledge points of the original options, obtain the relationship between the knowledge points of the original question stem and the knowledge points of the original options, and convert it into a vector representation as the relationship feature between the original options and the question stem; (24) By combining the sentence features, knowledge point features, part-of-speech features, syntactic dependency features, and relationship features with the question stem of each original option, the option features of the question are obtained; The user answer feature acquisition module is used to extract user answer features for questions; specifically, it includes: (31) Obtain the average answering time and average answering accuracy of the questions; (32) Obtain common incorrect answers and correct answers, and extract the relationship features between common incorrect answers and correct answers and the question stem; (33) Combine the average answering time, average answering accuracy, relationship characteristics between common wrong answers and question stems, and relationship characteristics between correct answers and question stems to form user answering characteristics; The feature fusion module is used to combine the features of the question stem, the options, and the user's answer to form the fusion features of the question to be retrieved. The similarity calculation module is used to calculate the similarity between the fusion features of the question to be retrieved and the fusion features of the questions in the question bank; The search result acquisition module is used to select the question with the highest similarity or the question with a similarity greater than the similarity threshold as the search result.

Images