Simplified question generation method and system based on vector model and vector similarity retrieval

Abstract

The invention discloses a simplified question generation method and system based on a vector model and vector similarity retrieval, and the method comprises the steps: firstly extracting multi-modal test question data uploaded by a user, and obtaining a query test question text set; calculating a vector value of each query test question text in the query test question text set by utilizing a vector model generation method, and obtaining a query test question vector; performing multi-modal retrieval on each query test question text in the query test question text set in the to-be-retrieved test question database by adopting a vector similarity retrieval method based on the query test question vectors, and obtaining a query test question comprehensive similarity score; according to the method, the functions of comprehensively identifying repeated test questions and test point similar test questions and simplifying the question set by adopting accurate semantic representation and similarity retrieval are effectively realized, so that the identification accuracy of the simplified questions is greatly improved, the labor cost of manually screening redundant test questions is reduced, meanwhile, the user practice efficiency is also improved, and the user experience is improved. And the test question bank resource consumption is reduced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and system for generating concise questions based on vector models and vector similarity retrieval. Background Technology

[0002] Against the backdrop of rapid development in educational informatization and online education, test questions, as the core carrier of knowledge assessment and skills training, have seen a continuous expansion in their reserves. Currently, test question banks built by educational institutions, enterprises, and individuals mainly come from two channels: first, user-uploaded test question resources, covering various types of questions accumulated in daily teaching and internal assessment scenarios; and second, publicly available test question resources obtained from public educational platforms and examination websites through web scraping technology. These two channels have jointly driven the rapid expansion of test question banks, but have also brought significant issues related to test question quality and management.

[0003] Currently, existing question banks generally suffer from high redundancy, leading to a large number of duplicate questions and questions with similar test points mixed together. This not only makes it easy for duplicate questions to exist in different forms within the same question bank due to subtle differences in question stem wording, option order adjustments, or different sources, but also makes it easy for similar questions with different question stem scenarios and question formats to test highly consistent core knowledge points and abilities. This redundancy problem causes learners to repeatedly encounter duplicate or similar questions during practice, resulting in wasted training time and failure to achieve comprehensive coverage and efficient consolidation of knowledge points, seriously affecting learners' learning outcomes. At the same time, redundant questions also consume a lot of server storage resources, requiring more manpower and time to manually screen questions during question bank updates, classifications, and maintenance, reducing the overall efficiency of question bank management. Therefore, it is necessary to design a simplified question generation method and system based on vector models and vector similarity retrieval. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and to better and more effectively address the problem of high redundancy in existing test question banks. This redundancy leads to a large number of duplicate questions and questions with similar test points mixed together. Not only do duplicate questions easily exist in different forms within the same test bank due to subtle differences in question stem wording, option order adjustments, or different sources, but similar questions with different question scenarios and question formats but highly consistent core knowledge points and ability requirements are also common. This redundancy problem causes learners to repeatedly encounter duplicate or similar questions during practice, resulting in wasted training time and hindering comprehensive coverage and efficient consolidation of knowledge points. This seriously affects learners' learning outcomes, and redundant questions also consume a large amount of server storage resources. This requires more manpower and time to manually screen questions during the updating, classification, and maintenance of the question bank, reducing the overall efficiency of question bank management. To address this issue, a simplified question generation method and system based on vector models and vector similarity retrieval is provided. This method achieves the function of comprehensively identifying and simplifying the question set by using accurate semantic representation and similarity retrieval. It not only significantly improves the accuracy of simplified question identification but also reduces the manpower cost of manually screening redundant questions, while also improving user practice efficiency and reducing the consumption of question bank resources.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A simplified question generation method based on vector models and vector similarity retrieval includes the following steps: Step A: Extract the multimodal test question data uploaded by the user and obtain the set of query test question texts; Step B: Calculate the vector value of each query question text in the query question text set using the vector model generation method and obtain the query question vector; Step C: Based on the query question vector, the vector similarity retrieval method is used to perform multimodal retrieval on each query question text in the query question text set in the database to be retrieved and obtain the comprehensive similarity score of the query questions. Then, the query question texts are sorted in descending order according to the comprehensive similarity score of the query questions, and the query question texts with the highest ranking are selected to form a query question retrieval result list. Step D: Based on the database of questions to be retrieved, set a standard threshold for the similarity of simplified questions, and then compare the standard threshold for the similarity of simplified questions with the comprehensive similarity score of the query question text in the query question retrieval results list to obtain the query question judgment result; Step E involves storing the query question text set into and updating the question database to be retrieved, and then assembling the simplified questions from the question judgment results into a simplified question set and outputting it.

[0006] The aforementioned simplified question generation method based on vector model and vector similarity retrieval, in step A, extracts multimodal test question data uploaded by users and obtains a set of query question texts, wherein the multimodal test question data uploaded by users includes Word, Excel, PDF, TXT, WPS, WPT, PPT, JPEG, PNG, CSV, ET and JSON formats.

[0007] The aforementioned simplified question generation method based on vector models and vector similarity retrieval, step B, involves calculating the vector value of each query question text in the query question text set using the vector model generation method to obtain the query question vector. The specific steps are as follows. Step B1: Use the BERT tokenizer to segment the query text and generate a token sequence containing CLS and SEP tags; Step B2 involves truncating or padding the token sequence using a set maximum sequence length (maxlen) to generate a fixed-length token ID sequence. The maximum sequence length (maxlen) can be adjusted as needed for different lengths of multiple-choice questions, fill-in-the-blank questions, short-answer questions, and comprehensive questions. Step B3: Input the token ID sequence into the BERT segmenter, wherein the BERT segmenter specifically maps the token ID sequence into an initial vector, and then obtains the hidden state matrix through a self-attention mechanism and a feedforward neural network; Step B4: Select a feature extraction strategy based on the text length to extract the core vector of the hidden state matrix. The feature extraction strategy uses word embedding model and bag-of-words model to dynamically adapt to test content of different lengths. Step B5: Perform L2 normalization on the core vector of the hidden state matrix to obtain the query question vector with a modulus of 1, as shown in formula (1). (1) in, To query the question vector, This is the core vector of the hidden state matrix.

[0008] The aforementioned method for generating concise questions based on vector models and vector similarity retrieval, specifically the truncation or padding process in step B2, is as follows: Step B21: If the length of the query question text is greater than the maximum sequence length maxlen, then truncate the query question text from the end of the query question text. Step B22: If the length of the query question text is less than the maximum sequence length maxlen, then use PAD padding markers to pad the end of the query question text to maxlen and form a token ID sequence.

[0009] The aforementioned method for generating concise questions based on vector models and vector similarity retrieval, specifically the text length selection feature extraction strategy in step B4, involves the following steps: Step B41: If the length of the query question text is no more than 512 tokens, then extract the row vector corresponding to the CLS token; Step B42: If the length of the query question text is greater than 512 tokens, then extract the average value of the row vectors corresponding to all CLS tokens.

[0010] The aforementioned simplified question generation method based on vector models and vector similarity retrieval, in step C, involves using vector similarity retrieval to perform multimodal retrieval on each query question text in the query question text set within the database of questions to be retrieved, based on the query question vector, and obtaining a comprehensive similarity score for each query question. Then, the query question texts are sorted in descending order according to the comprehensive similarity score, and the top-ranked query question texts are selected to form a query question retrieval result list. The specific steps are as follows. Step C1: For each question text to be retrieved in the question database... Using the BGE-M3 model Encode and obtain the first encoded vector ; Step C2, query the test question text. Using the BGE-M3 model Encode and obtain the second encoded vector ; Step C3, based on the first encoding vector Second encoding vector Perform multimodal retrieval and similarity score calculation to obtain the overall similarity score of the query questions; Step C4: Sort the query question texts in descending order based on the comprehensive similarity score of the query questions, and select the top-ranked query question texts to form a query question retrieval result list. The proportion of the top-ranked texts can be adjusted as needed.

[0011] The aforementioned simplified question generation method based on vector models and vector similarity retrieval, specifically the multimodal retrieval and similarity score calculation steps in step C3, are as follows: Step C31: Perform dense retrieval and obtain an initial candidate text set, wherein the dense retrieval specifically involves calculating dense vector similarity, as shown in formula (2). (2) in, For dense vector similarity, To retrieve the test question text Dense vectors, For the text of the test questions to be retrieved Dense vectors; Step C32 involves performing sparse retrieval based on the initial candidate text set. Specifically, sparse retrieval involves calculating sparse vector similarity, as shown in formula (3). (3) in, For sparse vector similarity, To retrieve the test question text sparse vectors, For the text of the test questions to be retrieved sparse vectors, Let be the value in the i-th dimension; Step C33: The initial candidate text set is reordered to obtain a rearranged candidate text set, and then multi-vector retrieval is performed based on the rearranged candidate text set. Specifically, the multi-vector retrieval involves calculating multi-vector similarity, as shown in formula (4). (4) in, For multi-vector similarity, To retrieve the test question text Multivectors, For the text of the test questions to be retrieved Multivectors, In the first Values ​​in each dimension; Step C34 involves weighted fusion of dense vector similarity, sparse vector similarity, and multi-vector similarity to obtain the comprehensive similarity score of the query questions, as shown in formula (5). (5) in, To query the overall similarity score of test questions, For intensive retrieval weights, For sparse retrieval weights, Weights for multi-vector retrieval.

[0012] The aforementioned method for generating concise questions based on vector models and vector similarity retrieval includes step D, which involves setting a standard threshold for the similarity of concise questions based on the database of questions to be retrieved. Then, the standard threshold is compared sequentially with the comprehensive similarity score of the query question text in the query question retrieval results list to obtain the query question determination result. This query question determination result includes both concise and non-concise questions. The specific steps are as follows. Step D1: If the similarity standard threshold of the simplified question is greater than the comprehensive similarity score of the query question text, then the query question text is determined to be a simplified question. Step D2: If the similarity standard threshold of the simplified question is less than the overall similarity score of the query question text, then the query question text is determined to be a non-simplified question.

[0013] A simplified question generation system based on vector models and vector similarity retrieval includes a question extraction module, a question vector calculation module, a question similarity calculation module, a question judgment module, and a question output module. The question extraction module extracts multimodal question data uploaded by the user and obtains a set of query question texts. The question vector calculation module calculates the vector value of each query question text in the query question text set using a vector model generation method to obtain a query question vector. The question similarity calculation module uses vector similarity retrieval to perform a similarity retrieval on each query question text in the query question text set within the question database. The system performs multimodal retrieval and obtains the comprehensive similarity score of the query questions. Then, it sorts the query question texts in descending order based on the comprehensive similarity score and selects the top-ranked query question texts to form a query question retrieval result list. The question determination module sets a simplified question similarity standard threshold based on the database of questions to be retrieved, and then compares the simplified question similarity standard threshold with the comprehensive similarity score of the query question texts in the query question retrieval result list to obtain the query question determination result. The question output module stores the query question text set into and updates the database of questions to be retrieved, and then assembles the simplified questions from the question determination result into a simplified question set and outputs it.

[0014] The beneficial effects of this invention are as follows: This invention provides a simplified question generation method and system based on vector models and vector similarity retrieval. First, it extracts multimodal test question data uploaded by the user to obtain a set of query question texts. Then, it uses a vector model generation method to calculate the vector value of each query question text in the query question text set to obtain a query question vector. Next, based on the query question vector, it uses a vector similarity retrieval method to perform multimodal retrieval on each query question text in the query question text set in the database to be retrieved, obtaining a comprehensive similarity score for the query questions. Then, it sorts the query question texts in descending order according to the comprehensive similarity score and selects the top-ranked query question texts to form a query question retrieval result list. Subsequently, it sets a simplified question similarity standard threshold based on the database to be retrieved, and then compares the simplified question similarity standard threshold with the comprehensive similarity score of the query question texts in the query question retrieval result list to obtain the query question determination result. Finally, it retrieves the query question text... This set stores and updates the database of questions to be retrieved, and then assembles the simplified questions from the question judgment results into a simplified question set and outputs it. This effectively realizes that the simplified question generation method and system can comprehensively identify and simplify the question set by using accurate semantic representation and similarity retrieval. Moreover, by combining the BERT word segmenter with the BGE-M3 model, it can perform deep semantic representation of the questions and break through the limitations of traditional keyword matching, thereby accurately identifying redundant questions with different stems but the same test points. At the same time, by combining vector database indexing and multimodal retrieval, the similarity retrieval time of the million-level question database can be shortened to the second level. By setting similarity thresholds, the number of returned results and retrieval weights, it can be dynamically adjusted according to different subjects and different exam types to meet the customized simplification needs of different users such as educational institutions, enterprises and individuals. This not only greatly improves the recognition accuracy of simplified questions, but also reduces the labor cost of manually screening redundant questions, while also improving user practice efficiency and reducing the consumption of question database resources. Attached Figure Description

[0015] Figure 1 This is an overall flowchart of a simplified question generation method based on vector model and vector similarity retrieval according to the present invention; Figure 2 This is a schematic diagram illustrating the principle of a simplified question generation system based on vector models and vector similarity retrieval, according to the present invention. Figure 3 This is a flowchart of the vector model generation method of the present invention. Figure 4 This is a flowchart of the multimodal retrieval process of the present invention. Detailed Implementation

[0016] The present invention will now be further described with reference to the accompanying drawings.

[0017] like Figure 1 As shown, the present invention provides a method for generating concise questions based on vector models and vector similarity retrieval, comprising the following steps: Step A: Extract the multimodal test data uploaded by the user and obtain a set of query test texts. The multimodal test data uploaded by the user can be in the formats of Word, Excel, PDF, TXT, WPS, WPT, PPT, JPEG, PNG, CSV, ET and JSON.

[0018] The file size should not exceed 100MB and the number of pages should not exceed 200.

[0019] like Figure 3 As shown, step B involves using a vector model generation method to calculate the vector value of each query question text in the query question text set and obtaining the query question vector. The specific steps are as follows: Step B1: Use the BERT tokenizer to segment the query text and generate a token sequence containing CLS and SEP tags; Step B2 involves truncating or padding the token sequence using a set maximum sequence length (maxlen) to generate a fixed-length token ID sequence. The maximum sequence length (maxlen) can be adjusted as needed for different question types, such as multiple-choice, fill-in-the-blank, short-answer, and comprehensive questions. The specific steps are as follows. Step B21: If the length of the query question text is greater than the maximum sequence length maxlen, then truncate the query question text from the end of the query question text. Step B22: If the length of the query question text is less than the maximum sequence length maxlen, then use PAD padding markers to pad the end of the query question text to maxlen and form a token ID sequence.

[0020] The maxlen setting is 8192 tokens.

[0021] Step B3: Input the token ID sequence into the BERT segmenter, wherein the BERT segmenter specifically maps the token ID sequence into an initial vector, and then obtains the hidden state matrix through a self-attention mechanism and a feedforward neural network; Step B4: Based on the text length, select a feature extraction strategy to extract the core vector of the hidden state matrix. The feature extraction strategy uses word embedding model and bag-of-words model to dynamically adapt to test content of different lengths. The specific steps are as follows: Step B41: If the length of the query question text is no more than 512 tokens, then extract the row vector corresponding to the CLS token; Step B42: If the length of the query question text is greater than 512 tokens, then extract the average value of the row vectors corresponding to all CLS tokens.

[0022] Step B5: Perform L2 normalization on the core vector of the hidden state matrix to obtain the query question vector with a modulus of 1, as shown in formula (1). (1) in, To query the question vector, This is the core vector of the hidden state matrix.

[0023] like Figure 4 As shown, in step C, based on the query question vector, the vector similarity retrieval method is used to perform multimodal retrieval on each query question text in the query question text set in the database to be retrieved, and obtain the comprehensive similarity score of the query questions. Then, the query question texts are sorted in descending order according to the comprehensive similarity score of the query questions, and the query question texts with the highest ranking are selected to form the query question retrieval result list. The specific steps are as follows. Step C1: For each question text to be retrieved in the question database... Using the BGE-M3 model Encode and obtain the first encoded vector ; Step C2, query the test question text. Using the BGE-M3 model Encode and obtain the second encoded vector ; Step C3, based on the first encoding vector Second encoding vector The specific steps for performing multimodal retrieval, calculating similarity scores, and obtaining the overall similarity score of the query questions are as follows. Step C31: Perform dense retrieval and obtain an initial candidate text set, wherein the dense retrieval specifically involves calculating dense vector similarity, as shown in formula (2). (2) in, For dense vector similarity, To retrieve the test question text Dense vectors, For the text of the test questions to be retrieved Dense vectors; Step C32 involves performing sparse retrieval based on the initial candidate text set. Specifically, sparse retrieval involves calculating sparse vector similarity, as shown in formula (3). (3) in, For sparse vector similarity, To retrieve the test question text sparse vectors, For the text of the test questions to be retrieved sparse vectors, Let be the value in the i-th dimension; Step C33: The initial candidate text set is reordered to obtain a rearranged candidate text set, and then multi-vector retrieval is performed based on the rearranged candidate text set. Specifically, the multi-vector retrieval involves calculating multi-vector similarity, as shown in formula (4). (4) in, For multi-vector similarity, To retrieve the test question text Multivectors, For the text of the test questions to be retrieved Multivectors, In the first Values ​​in each dimension; Step C34 involves weighted fusion of dense vector similarity, sparse vector similarity, and multi-vector similarity to obtain the comprehensive similarity score of the query questions, as shown in formula (5). (5) in, To query the overall similarity score of test questions, For intensive retrieval weights, For sparse retrieval weights, Weights for multi-vector retrieval.

[0024] The range of values ​​for the overall similarity score of the query questions is [0,1]. Step C4: Sort the query question texts in descending order based on the comprehensive similarity score of the query questions, and select the top-ranked query question texts to form a query question retrieval result list. The proportion of the top-ranked texts can be adjusted as needed.

[0025] Step D involves setting a simplified question similarity standard threshold based on the database of questions to be retrieved. Then, the simplified question similarity standard threshold is compared sequentially with the overall similarity score of the query question text in the query question retrieval results list to obtain the query question determination result. This result includes both simplified and non-simplified questions. The specific steps are as follows. The similarity threshold for simplified questions is 0.9. Step D1: If the similarity standard threshold of the simplified question is greater than the comprehensive similarity score of the query question text, then the query question text is determined to be a simplified question. Step D2: If the similarity standard threshold of the simplified question is less than the overall similarity score of the query question text, then the query question text is determined to be a non-simplified question.

[0026] Step E involves storing the query question text set into and updating the question database to be retrieved, and then assembling the simplified questions from the question judgment results into a simplified question set and outputting it.

[0027] like Figure 2 As shown, a simplified question generation system based on vector models and vector similarity retrieval includes a question extraction module, a question vector calculation module, a question similarity calculation module, a question judgment module, and a question output module. The question extraction module extracts multimodal question data uploaded by the user and obtains a set of query question texts. The question vector calculation module calculates the vector value of each query question text in the query question text set using a vector model generation method to obtain a query question vector. The question similarity calculation module uses vector similarity retrieval to search the database for each query question text in the query question text set based on the query question vector. The system performs multimodal retrieval and obtains the comprehensive similarity score of the query questions. Then, it sorts the query question texts in descending order based on the comprehensive similarity score and selects the top-ranked query question texts to form a query question retrieval result list. The question determination module sets a simplified question similarity standard threshold based on the database of questions to be retrieved, and then compares the simplified question similarity standard threshold with the comprehensive similarity score of the query question texts in the query question retrieval result list to obtain the query question determination result. The question output module stores the query question text set into and updates the database of questions to be retrieved, and then assembles the simplified questions from the question determination result into a simplified question set and outputs it.

[0028] To better illustrate the effects of the present invention, a comparative embodiment of generating simplified questions using the method of the present invention and existing methods is presented below, as shown in Table 1.

[0029] Table 1. Comparison of the simplification principles of using the method of the present invention and existing methods for generating simplified questions;

[0030] This embodiment's experimental record: Python was used to process the test questions, with a test question bank of 1353 items. The final threshold was set to be less than 0.6, or the number of simplified questions was less than 35% of the original question bank's total data. 1. After text deduplication and using regularization to remove interfering characters, 1164 items remained. 2. For the 1164 deduplicated questions, the initial vector similarity retrieval threshold was set to 0.96. The question set was traversed to generate question vector values, which were then inserted into the vector database. Subsequent questions, after generating vector values, needed to undergo vector similarity retrieval with the previous question set. If the similarity exceeded the specified threshold, the question was discarded and not inserted into the vector database. 3. The vector similarity retrieval was then iterated. After each round of question set processing, the threshold was reduced by 0.02 to simplify the questions.

[0031] 4. Ultimately, because the number of streamlined questions was less than 35% of the original number, the streamlined question bank contained 381 questions. The test question bank log is as follows: Database ID: test_1, Start Streamlining Question Bank Database ID: test_1, similarity: 0.96, simplification time: 40.682995319366455; The database ID is test_1, the similarity is 0.96, and the number of remaining questions is 1072. Database ID: test_1, similarity: 0.94, simplification time: 37.839807987213135; The database ID is test_1, the similarity is 0.94, and the number of remaining questions is 1000. Database ID: test_1, similarity: 0.92, simplification time: 33.78770685195923; The database ID is test_1, the similarity is 0.92, and the number of remaining questions is 929. Database ID: test_1, similarity: 0.90, simplification time: 31.109954833984375; The database ID is test_1, the similarity is 0.90, and the number of remaining questions is 865. Database ID: test_1, Similarity: 0.88, Simplification time: 29.686406135559082 The database ID is test_1, the similarity is 0.88, and the number of remaining questions is 820. Database ID: test_1, Similarity: 0.86, Simplification time: 27.643317222595215 The database ID is test_1, the similarity is 0.86, and the number of remaining questions is 757. Database ID: test_1, Similarity: 0.84, Simplification time: 25.35301423072815 The database ID is test_1, the similarity is 0.84, and the number of remaining questions is 700. Database ID: test_1, Similarity: 0.82, Simplification time: 23.299750089645386 The database ID is test_1, the similarity is 0.82, and the number of remaining questions is 608. Database ID: test_1, Similarity: 0.80, Simplification time: 19.963558673858643 seconds The database ID is test_1, the similarity is 0.80, and the number of remaining questions is 514. Database ID: test_1, Similarity: 0.78, Simplification time: 16.942593812942505 The database ID is test_1, the similarity is 0.78, and the number of remaining questions is 441. Database ID: test_1, Similarity: 0.76, Simplification time: 15.808372020721436 The database ID is test_1, the similarity is 0.76, and the number of remaining questions is 381. paper_id:test_1 Simplified test questions successfully.

[0032] In summary, the present invention provides a simplified question generation method and system based on vector models and vector similarity retrieval. First, it extracts multimodal test question data uploaded by the user to obtain a set of query question texts. Then, it uses a vector model generation method to calculate the vector value of each query question text in the set and obtains a query question vector. Next, based on the query question vector, it uses a vector similarity retrieval method to perform multimodal retrieval on each query question text in the set of query question texts in the database to be retrieved and obtains a comprehensive similarity score for the query questions. Then, it sorts the query question texts in descending order according to the comprehensive similarity score and selects the top-ranked query question texts to form a query question retrieval result list. Subsequently, it sets a simplified question similarity standard threshold based on the database to be retrieved, and then compares the simplified question similarity standard threshold with the comprehensive similarity scores of the query question texts in the query question retrieval result list to obtain a query question determination result. Finally, it returns the query question text set to the database. The system combines and updates the database of questions to be retrieved, then assembles the simplified questions from the question judgment results into a simplified question set and outputs it. This effectively realizes that the simplified question generation method and system can comprehensively identify and simplify the question set by using accurate semantic representation and similarity retrieval. Furthermore, by combining the BERT word segmenter with the BGE-M3 model, it can perform deep semantic representation of the questions and break through the limitations of traditional keyword matching, thereby accurately identifying redundant questions with different stems but the same test points. At the same time, by combining vector database indexing and multimodal retrieval, the similarity retrieval time of the million-level question database can be shortened to the second level. By setting similarity thresholds, the number of returned results, and retrieval weights, it can be dynamically adjusted according to different subjects and different exam types to meet the customized simplification needs of different users such as educational institutions, enterprises, and individuals. This not only greatly improves the recognition accuracy of simplified questions, but also reduces the labor cost of manually screening redundant questions, while also improving user practice efficiency and reducing the consumption of question database resources.

[0033] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating a concise question based on a vector model and vector similarity search, characterized in that: The method comprises the following steps, Step A, extracting the multi-modal test data uploaded by the user and obtaining a query test text set; Step B, calculating the vector value of each query test text in the query test text set using a vector model generation method and obtaining a query test vector; Step C, based on the query test vector, using vector similarity retrieval method to perform multi-modal retrieval on each query test text in the query test text set in the to-be-retrieved test database and obtaining a query test comprehensive similarity score, then sorting the query test texts in descending order according to the query test comprehensive similarity score and selecting the query test texts with high ranking to form a query test retrieval result list, the specific steps are as follows, Step C1, for each to-be-searched test question text in the to-be-searched test question database Using the BGE-M3 model Encoding and obtaining a first encoding vector ; Step C2, query the test text using the BGE-M3 model encoding and obtaining a second encoding vector ; Step C3, performing multimodal retrieval and similarity score calculation with the first encoding vector and the second encoding vector and obtaining a query item comprehensive similarity score; Step C4, sorting the query test texts in descending order according to the query test comprehensive similarity score and selecting the query test texts with high ranking to form a query test retrieval result list, wherein the proportion of the query test texts with high ranking can be adjusted as needed; Step D, setting a simplified test similarity standard threshold based on the to-be-retrieved test database, then comparing the simplified test similarity standard threshold with the query test comprehensive similarity scores of the query test texts in the query test retrieval result list in sequence and obtaining a query test determination result; Step E, storing and updating the query test text set in the to-be-retrieved test database, and then forming a simplified test set from the simplified tests in the test determination result and outputting the simplified test set. 2.The simplified question generation method based on vector model and vector similarity search according to claim 1, characterized in that: Step A, extracting the multi-modal test data uploaded by the user and obtaining a query test text set, wherein the format of the multi-modal test data uploaded by the user includes Word, Excel, PDF, TXT, WPS, WPT, PPT, JPEG, PNG, CSV, ET and JSON formats. 3.The simplified question generation method based on vector model and vector similarity search according to claim 2, characterized in that: Step B, calculating the vector value of each query test text in the query test text set using a vector model generation method and obtaining a query test vector, the specific steps are as follows, Step B1, using a BERT tokenizer to tokenize the query test text and generating a token sequence containing CLS and SEP markers; Step B2, truncating or padding the token sequence using a set maximum sequence length maxlen and generating a fixed-length token ID sequence, wherein the maximum sequence length maxlen can be adjusted as needed according to the different lengths of multiple-choice questions, fill-in-the-blank questions, short answer questions and comprehensive questions; Step B3, inputting the token ID sequence into the BERT tokenizer, wherein the BERT tokenizer specifically maps the token ID sequence into an initial vector, and then obtains a hidden state matrix through a self-attention mechanism and a feedforward neural network; Step B4, extracting the core vector of the hidden state matrix according to the text length selection feature extraction strategy, wherein the feature extraction strategy uses a word embedding model and a bag-of-words model to dynamically adapt to test content with different lengths; Step B5, performing L2 normalization on the core vector of the hidden state matrix to obtain a query test vector with a modulus of 1, as shown in formula (1), （1） wherein, is a query item vector, is a hidden state matrix core vector.

4. The method of claim 3, wherein the method is characterized by: The specific steps of the truncation or padding processing in step B2 are as follows, Step B21, if the query test text length is greater than the maximum sequence length maxlen, the query test text is truncated from the tail of the query test text; Step B22, if the query test text length is less than the maximum sequence length maxlen, the PAD fill mark is used to complete the query test text to maxlen at the tail of the query test text and form a token ID sequence.

5. The method of claim 3, wherein the method is characterized by: Step B4, the specific extraction steps of the text length selection feature extraction strategy are as follows, Step B41, if the query test text length is not greater than 512 tokens, the row vector corresponding to the CLS mark is extracted; Step B42, if the query test text length is greater than 512 tokens, the average value of the row vectors corresponding to all CLS marks is extracted.

6. The method of claim 1, wherein the method is characterized by: The specific steps of the multi-modal retrieval and similarity score calculation in step C3 are as follows, Step C31, performing dense retrieval and obtaining an initial candidate text set, wherein the dense retrieval is specifically calculating dense vector similarity, as shown in formula (2), （2） wherein, is a dense vector similarity, is a dense vector of the query test text is a dense vector of the test text to be retrieved to be retrieved​ Step C32, performing sparse retrieval based on the initial candidate text set, wherein the sparse retrieval is specifically calculating sparse vector similarity, as shown in formula (3), （3） wherein, is a sparse vector similarity, is a sparse vector of the query test text , is a sparse vector of the test text to be retrieved , is a value in the i-th dimension; Step C33, reordering the initial candidate text set to obtain a rearranged candidate text set, and performing multi-vector retrieval based on the rearranged candidate text set, wherein the multi-vector retrieval is specifically calculating multi-vector similarity, as shown in formula (4), （4） in, For multi-vector similarity, To retrieve the test question text Multivectors, For the text of the test questions to be retrieved Multivectors, In the first Values ​​in each dimension; Step C34, weighting and fusing the dense vector similarity, the sparse vector similarity and the multi-vector similarity to obtain a query test comprehensive similarity score, as shown in formula (5), （5） wherein, is a query item comprehensive similarity score, is a dense retrieval weight, is a sparse retrieval weight, is a multi-vector retrieval weight.

7. The method of claim 6, wherein the method is characterized by: Step D, setting a simplified test similarity standard threshold based on the test database to be retrieved, and then comparing the simplified test similarity standard threshold with the query test comprehensive similarity scores of the query test texts in the query test retrieval result list in turn to obtain query test determination results, wherein the query test determination results include simplified tests and non-simplified tests, and the specific steps are as follows, Step D1, if the simplified test similarity standard threshold is greater than the query test comprehensive similarity score of the query test text, the query test text is determined as a simplified test; Step D2, if the simplified test similarity standard threshold is less than the query test comprehensive similarity score of the query test text, the query test text is determined as a non-simplified test.

8. A reduced item generation system based on vector model and vector similarity search, the reduced item generation system based on the reduced item generation method according to any one of claims 1-7, characterized in that: The test item extraction module is used to extract the multi-modal test item data uploaded by the user and obtain a query test text set; The test item vector calculation module is used to calculate the vector value of each query test text in the query test text set by using a vector model generation method and obtain a query test vector; The test item similarity calculation module is used to calculate the similarity between the query test text and the test item in the test database to be retrieved by using a multi-modal retrieval method and obtain a query test comprehensive similarity score; The test item determination module is used to compare the query test comprehensive similarity score with a simplified test similarity standard threshold to obtain a query test determination result, wherein the query test determination result includes a simplified test and a non-simplified test; The test item output module is used to output the query test determination result. The test question similarity calculation module is configured to perform multi-modal retrieval on each query test question text in the query test question text set in the to-be-retrieved test question database based on the query test question vector by using a vector similarity retrieval method and obtain a query test question comprehensive similarity score, and sort the query test question texts in descending order according to the query test question comprehensive similarity score and select the query test question texts at the top of the sorting to form a query test question retrieval result list; The test question determination module is configured to set a simplified test question similarity standard threshold value based on the to-be-retrieved test question database, and then compare the simplified test question similarity standard threshold value with the query test question comprehensive similarity scores of the query test question texts in the query test question retrieval result list in sequence and obtain a query test question determination result; The test question output module is configured to store and update the query test question text set in the to-be-retrieved test question database, and then form a simplified test question set by using the simplified test questions in the query test question determination result and output the simplified test question set.

Images