Reading understanding test question generation method based on large language model multi-agent

Abstract

The invention discloses a reading understanding test question generation method based on a large language model multi-agent, and the process of generating a reading understanding test question can be divided into a plurality of stages: firstly, generating question stem contents including reading materials and questions through an RAG method, and after the question stem contents are generated, generating the reading materials and the questions through the RAG method; a plurality of groups of corresponding options are generated through an option generation agent according to the content of a question stem, the generated options are preferentially selected through two judgment models in a debate mode to find the optimal option, then the optimal option is revised through a debate record obtained by a revision agent in the debate, and the optimal option is revised. The question answer of the to-be-answered question composed of the question stem content and the revised options is made through the question doing agent, and if it is determined that the to-be-answered question is unreasonable according to the question answer made by the question doing agent, the to-be-answered question is further revised, so that the generation quality of the reading understanding question is improved.

Description

Technical Field

[0001] The present application relates to the field of text generation technology, and in particular to a method for generating reading comprehension test questions based on a large language model and multiple agents. Background Art

[0002] With the vigorous development of artificial intelligence and deep learning technologies, more and more intelligent solutions have emerged for the generation of reading comprehension test questions (reading comprehension multiple-choice questions).

[0003] Existing technology uses Retrieval-Augmented Generation (RAG) to generate questions. This involves retrieving relevant material from a vector database and then combining it with a large language model to generate a complete reading comprehension question. Specifically, an embedding model can be used to vectorize text, retrieve relevant paragraphs as context for generation, and then a pre-trained large language model can be used to generate a complete reading comprehension question.

[0004] Although this method increases the diversity of questions, it still has certain problems: The above method has the problem of coarse retrieval granularity in the RAG process, which leads to interference between the retrieval results. After retrieving the relevant materials for generating reading comprehension test questions, a single model is used to directly generate complete reading comprehension test questions. The generated reading comprehension test questions may have logical loopholes, resulting in low quality of the generated reading comprehension test questions.

[0005] With respect to the problem that the generated reading comprehension test questions in the above-mentioned prior art may have logical loopholes, thus resulting in the generated reading comprehension test questions being of low or even poor quality, no effective solution has been proposed so far. Summary of the Invention

[0006] The embodiments of the present disclosure provide a method, device and storage medium for generating reading comprehension test questions based on a large language model and multiple agents, so as to at least solve the technical problem in the prior art that the generated reading comprehension test questions may have logical loopholes, thereby resulting in low or even poor quality of the generated reading comprehension test questions.

[0007] According to one aspect of an embodiment of the present disclosure, a method for generating reading comprehension test questions based on a large language model and multiple agents is provided, comprising: obtaining keywords input by a user corresponding to the reading comprehension test questions to be generated; generating stem content corresponding to the reading comprehension test questions according to the keywords through a stem generation model based on RAG, wherein the stem content includes reading materials and questions; generating an agent through multiple options, and generating multiple groups of options corresponding to the questions based on the stem content, wherein the multiple groups of options are different; debating which group of options among the multiple groups of options is the best through two preset evaluation models, obtaining a debate record and an optimal option selected by the two evaluation models; revising the optimal option based on the debate record through a revision agent, obtaining a revised option; answering the stem content and the revised option through a question-solving agent, obtaining an answer result; and generating the reading comprehension test questions based on the stem content, the revised options and the answer result.

[0008] According to another aspect of an embodiment of the present disclosure, a storage medium is further provided, the storage medium including a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0009] According to another aspect of the embodiments of the present disclosure, a reading comprehension test question generation device based on a large language model and multiple agents is also provided, including: an acquisition module for acquiring keywords input by a user corresponding to the reading comprehension test question to be generated; a stem generation module for generating stem content corresponding to the reading comprehension test question based on the keywords through a stem generation model based on RAG, wherein the stem content includes reading materials and questions; an option generation module for generating an agent through multiple options, and generating multiple groups of options corresponding to the questions based on the stem content, wherein the multiple groups of options are different; a debate module for debating which group of options is the best among the multiple groups of options through two preset evaluation models, and obtaining a debate record and an optimal option selected by the two evaluation models; a revision module for revising the optimal option based on the debate record through a revision agent to obtain a revised option; an answering module for answering the stem content and the revised options through a question-answering agent to obtain an answer result; and a test question generation module for generating the reading comprehension test question based on the stem content, the revised options and the answer result.

[0010] According to another aspect of the embodiments of the present disclosure, a reading comprehension test question generation device based on a large language model and multiple agents is also provided, including: a processor; and a memory, connected to the processor, for providing the processor with instructions for processing the following processing steps: obtaining keywords corresponding to the reading comprehension test question to be generated input by the user; generating stem content corresponding to the reading comprehension test question through a stem generation model based on RAG based on the keywords, the stem content including reading materials and questions; generating an agent through multiple options, and generating multiple groups of options corresponding to the questions based on the stem content, wherein the multiple groups of options are different; debating which group of options is the best among the multiple groups of options through two preset evaluation models, obtaining a debate record and an optimal option selected by the two evaluation models; revising the optimal option based on the debate record through a revision agent to obtain a revised option; answering the stem content and the revised option through a question-solving agent to obtain an answer result; and generating the reading comprehension test question based on the stem content, the revised options and the answer result.

[0011] In the disclosed embodiment, after the question stem content is generated by the RAG method, the generation of options and test answers is divided into multiple stages (divergence stage, convergence stage, revision stage, practice stage and backtracking stage), and is completed by multiple intelligent agents in collaboration. The generation of options and test answers is based on the reference to the question stem content, and multiple intelligent agents with different functions collaboratively evaluate and revise the options and test answers, so that the final generated reading comprehension test questions are reasonable. Therefore, in this method, through the multi-agent collaboration mechanism, combined with the collaborative optimization of options and test answers by the option generation agent, the evaluation model (also called the "evaluation agent"), the revision agent and the question-solving agent, the diversity and logical rigor of the generation of reading comprehension test questions are improved from multiple angles, thereby solving the problem described in the background technology that "the generated reading comprehension test questions in the existing technology may have logical loopholes, which leads to the low or even poor quality of the generated reading comprehension test questions." BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of this application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings: Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to embodiment 1 of the present disclosure; Figure 2 1 is a flowchart of a method for generating reading comprehension test questions based on a large language model and multiple agents according to the first aspect of embodiment 1 of the present disclosure; Figure 3 A schematic diagram of a process for generating reading comprehension test questions through multi-stage multi-agent collaboration provided in Example 1 of the present disclosure; Figure 4 This is a flow chart of generating question content using the RAG method provided in Example 1 of the present disclosure; Figure 5 This is a schematic diagram of an example of a generated reading comprehension test question provided in Example 1 of the present disclosure; Figure 6 is a schematic diagram of a reading comprehension test question generation device based on a large language model and multiple agents according to the first aspect of embodiment 2 of the present disclosure; and Figure 7 It is a schematic diagram of a reading comprehension test question generation device based on a large language model and multiple agents according to the first aspect of Example 3 of the present disclosure. DETAILED DESCRIPTION

[0013] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.

[0014] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0015] Example 1 According to this embodiment, an embodiment of a method for generating reading comprehension test questions based on a large language model and multiple agents is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0016] The method embodiment provided in this embodiment can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Figure 1 The hardware structure block diagram of a computing device for implementing a method for generating reading comprehension test questions based on a large language model and multiple agents is shown. Figure 1 As shown, a computing device may include one or more processors (the processor may include, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA) or other processing device), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, transmission device, and input / output interface are connected to the processor via a bus. In addition, it may also include: a display, a keyboard, and a cursor control device connected to the input / output interface. Those skilled in the art will understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0017] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be fully or partially integrated into any of the other components of the computing device. As discussed in the embodiments of the present disclosure, the data processing circuitry functions as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0018] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for generating reading comprehension test questions based on a large language model and multiple agents in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the method for generating reading comprehension test questions based on a large language model and multiple agents of the above-mentioned application program is realized. The memory may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0019] The transmission device is used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by a communications provider of the computing device. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0020] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computing device.

[0021] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing device described above.

[0022] In the above operating environment, according to the first aspect of this embodiment, a method for generating reading comprehension questions based on a large language model and multiple agents is provided. The method can be Figure 1 The computing device implementation shown. Figure 2 A schematic diagram showing the process of the method is shown in FIG. Figure 2 As shown, the method includes: S202: Obtain keywords input by the user corresponding to the reading comprehension test questions to be generated; S204: Generate stem content corresponding to the reading comprehension test questions using a RAG-based stem generation model based on the keywords, the stem content including reading materials and questions; S206: Generate an agent using multiple options, and generate multiple groups of options corresponding to the question based on the question stem, wherein the multiple groups of options are different; S208: Using two preset evaluation models, debate is conducted on which of the multiple groups of options is optimal, obtaining a debate record and the optimal option selected by the two evaluation models; S210: The revision agent revises the optimal option based on the debate record to obtain a revised option; S212: The question-solving agent answers the question stem and the revised options to obtain an answer result; and S214: Generate reading comprehension questions based on the question content, revised options and answer results.

[0023] Figure 3 A flowchart of a multi-stage multi-agent collaborative generation of reading comprehension test questions provided in Example 1 of the present disclosure.

[0024] Specifically, see Figure 3 As shown, when generating a reading comprehension question, the user inputs keywords corresponding to the reading comprehension question into the computing device. The user may input keywords such as "history of aerospace development". The computing device can then obtain the keywords input by the user corresponding to the reading comprehension question to be generated (S202). The computing device then generates the stem content corresponding to the reading comprehension question based on the keyword "space development history" using a RAG-based stem generation model (S204). Specifically, the user can input the keyword corresponding to the reading comprehension question to be generated through their terminal. The terminal transmits the keyword to the computing device (the specific form of the terminal is not limited here). After receiving the keyword, the computing device uses RAG to generate the stem content corresponding to the reading comprehension question based on the keyword, including the reading material and the question (stem generation stage).

[0025] Continue to refer Figure 3 As shown, after the computing device generates the question content, it can generate an agent using multiple options. Based on the question content, it generates multiple sets of options corresponding to the question, where each set of options is different (S206). Then, using two preset evaluation models, a debate is conducted on which of the multiple sets of options is optimal, resulting in a debate record and the optimal option selected by the two evaluation models (S208). The optimal option is then revised by a revision agent based on the debate record to obtain a revised option (S210). The question-solving agent answers the question content and the revised options to obtain an answer result (S212). Finally, based on the question content, the revised options, and the answer result, a reading comprehension test question is generated (S214).

[0026] As described in the background technology, the existing methods for generating reading comprehension test questions have the following technical problems: the retrieval granularity in the RAG process is too coarse, resulting in interference between the retrieval results; after retrieving the relevant materials for generating reading comprehension test questions, a single model is used to directly generate complete reading comprehension test questions, and the generated reading comprehension test questions may have logical loopholes, resulting in low quality of the generated reading comprehension test questions.

[0027] In view of this, in the technical solution of the present application, the stem content (i.e., reading materials and questions) of the reading comprehension test questions is first generated only by the RAG method, and then the options and test answers are generated based on the stem content. Since the present invention only generates the stem content rather than the complete test questions (including stem content, options and test answers) by the RAG method, it solves the coarse-grained retrieval problem existing in the prior art to a certain extent. Moreover, in the process of generating options and test answers, the generation of options and test answers is divided into multiple stages (divergence stage, convergence stage, revision stage, practice stage and backtracking stage), and is completed by the collaboration of multiple intelligent agents. The generation of options and test answers is based on the reference to the stem content, and multiple intelligent agents with different functions collaboratively evaluate and revise the options and test answers, so that the reading comprehension test questions finally generated are reasonable. Therefore, in this method, through the multi-agent collaboration mechanism, combined with the option generation agent, the evaluation model (also called the "evaluation agent"), the revision agent and the question-solving agent to collaboratively optimize the options and test answers, the diversity and logical rigor of the generated reading comprehension test questions are improved from multiple angles, thereby solving the problem described in the background technology that "the generated reading comprehension test questions in the existing technology may have logical loopholes, which leads to the low or even poor quality of the generated reading comprehension test questions."

[0028] Optionally, based on the keyword, the operation of generating the stem content corresponding to the reading comprehension test question through the RAG-based stem generation model specifically includes: searching in a preset vector database based on the keyword vector corresponding to the keyword, obtaining corresponding search results, filling the search results into a preset prompt word template to obtain reconstructed prompt words, wherein the prompt word template includes a real reading comprehension test question as a demonstration; and inputting the reconstructed prompt words into the stem generation model to generate the above-mentioned stem content.

[0029] Among them, the computing device can first construct a corpus specifically for generating the stem content of reading comprehension test questions, and then construct the above-mentioned vector database through the corpus, and then use the RAG method to generate the stem content through the constructed vector database.

[0030] To ensure that the reading materials in the question stems have a realistic basis, rather than being fabricated by the model, and to enhance the scalability and timeliness of the reading materials, we first construct a corpus based on existing data crawling and processing methods. The specific steps are as follows: A large amount of relevant corpus is obtained through existing paper materials, web crawling, and other channels; text cleaning: using a hashing algorithm to remove duplicate text data; removing irrelevant characters; unifying the text format; dividing long texts into several semantically continuous paragraph blocks based on paragraph characteristics, ensuring that each paragraph block is not too lengthy; and annotating each paragraph with topic, time, and source to form the metadata of the corpus. Then, we can construct a vector database from this corpus using the standard RAG method, using the embedding model for vectorization and the FAISS vector library for persistence.

[0031] Then, when the user has a need to generate a reading comprehension question (i.e., when receiving keywords input by the user), the computing device can use the RAG method through the vector database to generate the question content.

[0032] in, Figure 4 This is a flow chart of generating the content of the title by the RAG method provided by Example 1 of the present disclosure. Figure 4 As can be seen in the figure, after receiving the keyword input by the user, the computing device uses the embedding model to determine the embedding vector corresponding to the keyword. Based on the determined embedding vector, the computing device then searches the vector database for relevant paragraph blocks, which serve as search material (search results). The computing device then applies this search material to a pre-set prompt word template to obtain a reconstructed prompt word. Finally, the computing device inputs the prompt word into the question stem generation model to generate the question stem content, which includes the reading material and the proposed question.

[0033] For example, a user can enter a keyword such as "space development history." The computing device can use the embedding model to determine the embedding vector corresponding to the keyword "space development history." The computing device then performs a search using the RAG method, obtaining a search result such as "China's space development originated from..." During this search process, the metadata annotated with the text also facilitates fine-grained retrieval, accurately retrieving text blocks categorized as "space." Next, this method reconstructs the prompt word. A prompt word template might be: "Please use this passage {search material} to integrate and generate a reading comprehension test material and provide an appropriate question based on the material. The material requires: ..., several demonstrations." The demonstrations can be selected from real reading comprehension questions. This few-shot prompting method helps the question stem generation model better understand the requirements. The reconstructed prompt word is then passed to the question stem generation model to generate the final question stem, which includes the reading material and the question.

[0034] It should be noted that the existing technologies described in the background still suffer from the problem of coarse-grained retrieval. Specifically, traditional RAG question generation methods fail to design retrieval strategies tailored to the characteristics of reading comprehension questions. Instead, they often use entire retrieved documents as references, which may contain a lot of irrelevant or interfering information. Furthermore, this method fails to consider the impact of retrieved documents on the reading material and options in the reading comprehension test. Instead, it simply generates a complete reading comprehension test question based on these documents at a single time step, resulting in a high probability of abnormal model output. Furthermore, the documents lack annotation of features such as topic and time.

[0035] Furthermore, a whole section of a document in a vector library in the prior art often represents a complete set of test papers, reading comprehension questions, and other documents. The stem content and options are generated simultaneously, leading to a coarse-grained retrieval problem: much of the retrieved content is actually unhelpful for question generation, and may even interfere with each other and hinder question generation.

[0036] In this specification, however, only the stem content (i.e., reading material and questions) for the reading comprehension test is generated using RAG. Because the metadata in the corpus used to generate the stem content is finely annotated, the vector database constructed based on the corpus is guaranteed to have the same fine granularity. This allows for fine-grained searches within the vector database based on embedded vectors, preventing search results from containing interfering content. Consequently, the cue word input material generation model reconstructed based on the search results can generate high-quality stem content, thus, to a certain extent, resolving the coarse-grained search problem existing in the prior art.

[0037] The aforementioned corpus can be constructed using the following approach: 100,000 reading comprehension-related texts are obtained from public education platforms (e.g., the Education Resources Public Service Platform) and authoritative publications (e.g., "Five-Year College Entrance Examination Three-Year Simulation"). The SimHash algorithm is used to deduplicate text, with a similarity threshold of 0.85, removing duplicate paragraphs. Regular expressions are used to remove HTML tags, extra spaces, and special characters, and the text is encoded in UTF-8. The spaCy tool is used to segment long text into paragraphs, ensuring that each paragraph block does not exceed 500 characters in length. The DeepSeek-R1-671B model's API is used to categorize paragraph blocks by subject (e.g., "Science and Technology," "History," and "Literature"). Regular expressions are used to extract timestamps (e.g., "2023") and source information to generate metadata. The resulting corpus contains nearly 500,000 annotated texts, with 95% subject coverage and support for dynamic updates.

[0038] The computing device then uses the Sentence-BERT embedding model to encode the paragraph chunks into 768-dimensional vectors, which are then stored in the FAISS vector database and associated with metadata (including topic, time, and source). When a user enters a query, it is first vectorized using Sentence-BERT, followed by a search for the top-1 similar paragraph chunks. The retrieved paragraph chunks are then fed into a local question stem generation model (here, the Deepseek-R1-32B model deployed locally on Ollama) to generate integrated reading material and questions.

[0039] It should be noted that the model used to instantiate the option generation agent and the question-solving agent can be obtained by fine-tuning the pre-trained basic model (i.e., the pre-trained large language model).

[0040] Optionally, before generating multiple sets of options corresponding to the questions based on the question stem content through multiple option generation agents, the method also includes: instantiating the option generation model based on the pre-trained basic model to obtain multiple option generation agents, wherein the option generation model is obtained by the following operations: obtaining a standard question in a standard question set, the standard question is obtained by converting a real reading comprehension test question into structured data containing a reading material field, a question field, an option field, and a test answer field. Based on the question stem content of the standard question through a preset large language model, corresponding option generation logic information is generated, the option generation logic information is used to represent the thinking process of generating corresponding options according to the question stem content in the standard question, and an option generation training set is constructed according to the question stem content of the standard question in the standard question set, the corresponding options, and the corresponding option generation logic information; and based on the option generation training set, the pre-trained basic model is fine-tuned to obtain the option generation model.

[0041] Optionally, before the question-solving intelligent agent answers the question content and the revised options and obtains the answer result, the method also includes: instantiating the question-solving model based on the pre-trained basic model to obtain the question-solving intelligent agent, wherein the question-solving model is obtained by the following operations: obtaining a standard question in a standard question set, and based on the question content of the standard question, generating question-solving idea information and test answers selected from the options corresponding to the standard question through a preset large language model; when the selected test answer matches the test answer corresponding to the standard question, constructing a question-solving training set according to the reading materials, questions, options, test answers and question-solving idea information corresponding to the standard question; and fine-tuning the pre-trained basic model based on the question-solving training set to obtain the question-solving model.

[0042] Among them, the methods of obtaining the option generation model by fine-tuning the pre-trained basic model and instantiating the option generation agent, and obtaining the question-solving model by fine-tuning the pre-trained basic model and instantiating the question-solving agent are similar and are explained below.

[0043] This phase aims to build the training data needed to fine-tune the models (i.e., the option generation model and the test-taking model mentioned above). First, the computing device can obtain a large number of reading comprehension questions through various channels and format them into files containing fields for reading material, questions, options, and correct options (i.e., the test answer fields mentioned above). Next, a standard question set containing thousands of high-quality, representative standard questions can be constructed (e.g., through manual screening).

[0044] For the option generation training set, the input is the reading material and questions from the standard questions set. What needs to be annotated are the four generated options and the thinking behind the generation process (i.e., the option generation logic mentioned above). Therefore, for a training example in the option generation training set mentioned above, the sample data includes the content of a standard question, and the annotation information includes the options and the option generation logic for this standard question.

[0045] For the problem-solving training set, the input is a complete standard problem from the standard problem set (excluding the answer). What needs to be annotated is the correct answer to this standard problem and the thinking process during the problem-solving process. Therefore, for the above options, a training sample is generated in the training set. The sample data includes a standard problem without the answer, and the annotation information includes the answer to this standard problem and the thinking process during the problem-solving process.

[0046] Since manual labeling requires a lot of costs and the quality is difficult to guarantee, automated labeling is used to generate option training sets and question training sets, that is, using model API interfaces with thinking capabilities, such as Deepseek-R1 and GPT4-O1.

[0047] For the option generation training set, the options that need to be annotated are actually the options already in the standard question set (which can be four options), but the generation logic and process of the four options require model annotation. Therefore, this method proposes a reverse thinking approach. Given the reading material, questions, and options, the large language model is asked to work backwards to understand how the four options were generated. This provides information on the option generation logic, and the options are annotated before they are selected (i.e., the options use the options from the standard question itself).

[0048] Similarly, for the problem-solving training set, the part that requires annotation with the help of the large language model is the thought process involved in solving the problem. A free-answer-and-screening approach is proposed here. Without any additional prompts, the large language model first freely generates the thought process and selected answer for this standard problem. It then determines whether the model's selected answer is the actual answer to the problem in the standard problem set. If so, the complete thought process of this free-answer attempt (i.e., the thought process information mentioned above) is retained as an annotation. If not, the model re-answers the problem without memorization until the problem is solved correctly or the maximum number of attempts is reached, at which point the problem is considered invalid.

[0049] Among them, 10,000 reading comprehension questions can be extracted from the reading comprehension test bank and formatted into a JSON file containing "reading material, questions, options, correct options (answers to the test questions)". 5,000 high-quality questions are manually screened and retained as the basic data set. The API interface of the DeepSeek-R1-671B model is called, and according to the designed question template, the thinking chain data for option generation and question solving are constructed respectively (option generation training set and question solving training set). The entire training set adopts the alpaca format, that is, a JSON file containing three fields: system, instruction, and output. The content of the output field is annotated by DeepSeek-R1-671B.

[0050] After obtaining the Thinking Chain dataset (using the above options to generate a training set and a problem-solving training set), you can select a pre-trained base model (for example, using Qwen2.5-32B-Instruct as the pre-trained base model) and perform supervised fine-tuning of the model based on the existing Low-Rank Adaptation (LoRA) method. The goal of supervised fine-tuning is to gradually make the model's output logic approach the output of the dataset given the input portion.

[0051] The model weight update uses the standard LoRA method, and the forward propagation formula is as follows. When the model parameters are updated, the original weights are frozen. , only update the low-rank matrix with fewer parameters and , thereby achieving the goal of reducing training costs.

[0052]

[0053] The training objective function uses the standard cross-entropy loss function. The training process follows the standard next-token-predict method, predicting the token at position i at a time, with a predicted probability p for the true value at that position. The true value of the token at position i is then added to the context, and the model predicts the token at position i+1. This process is repeated until the prediction is complete, and the total loss is obtained for gradient descent.

[0054]

[0055] The LoRA adapter (rank r=8) can be loaded. Using the Transformers framework, the learning rate is set to 1e-4, the batch size is set to 4, and training is performed for 2 epochs. The loss function is cross-entropy loss, and the AdamW optimizer is used. Training takes approximately 4 hours (on an 8×4090D GPU). The LoRA weights (matrices B and A) are merged with the pre-trained base model to obtain the fine-tuned option generation model and question-solving model.

[0056] The following is a detailed description of the five stages of collaborative generation of options and test answers mentioned above. Figure 3 ,from Figure 3 As can be seen from the figure, the process of generating a reading comprehension test question can be divided into multiple stages. First, the computing device generates the stem content including the reading material and the questions. After the stem content is generated, the computing device generates multiple groups of corresponding options according to the stem content through the option generation agent (in Figure 3 The divergence stage is in the middle stage), and the generated options are selected through the judgment model in a debate to find the best option (in Figure 3 The optimal option is then revised by revising the debate record obtained by the agent in the debate (in Figure 3 The answer to the question to be answered is determined by the question-solving agent (in the Figure 4 It is called the practice stage in the literature). If there are any unreasonable questions to be answered, the questions to be answered will be further revised (in Figure 3 The following will introduce each stage in detail starting from the divergent stage.

[0057] Optionally, multiple option generation agents are used to generate multiple groups of options corresponding to the questions based on the question stem content, specifically including: inputting the question stem content into multiple option generation agents, and each option generation agent outputs a corresponding group of options and corresponding test answers.

[0058] Specifically, refer to Figure 3As shown in the figure, in the divergence phase: First, the fine-tuned option generation model is used as the option generation agent. To enhance the diversity of reasoning paths, multiple option generation agents are instantiated. Each option generation agent uses a sampling reasoning strategy rather than a greedy reasoning strategy, and sets high reasoning parameters (such as temperature, top-p, and top-k) to ensure that the outputs of each option generation agent are unique.

[0059] After generating the question stem, the option generation agent receives this content and reconstructs it into an input paradigm for fine-tuning the option generation model. After in-depth reflection, the option generation agent outputs its thought process and final options for the question. During this process, the inputs and actions of multiple option generation agents are identical, but the outputs are different reasoning paths and diverse options.

[0060] Then, enter Figure 3 The goal of this stage is to converge the divergent sets of options into the optimal set of options. In this method, a debate is used to debate the optimal set of options.

[0061] Optionally, a debate is conducted through two preset evaluation models as to which group of options is the best among multiple groups of options, and a debate record and the best options selected by the two evaluation models are obtained. The operation specifically includes: an initialization stage, in which the two evaluation models are divided into an affirmative evaluation model and a negative evaluation model, and the opinion information of the affirmative evaluation model and the negative evaluation model are determined, wherein the opinion information of the affirmative evaluation model includes the best options selected by the affirmative evaluation model from multiple groups of options and the corresponding reasons, and the opinion information of the negative evaluation model includes the best options selected by the negative evaluation model from multiple groups of options and the corresponding reasons, wherein the best option selected by the negative evaluation model is inconsistent with the best option selected by the affirmative evaluation model. In the debate stage, in each round of debate in the debate stage, the affirmative evaluation model is used to determine the speech information and corresponding confidence level of the affirmative evaluation model based on the speech information of the negative evaluation model in the previous round. The negative evaluation model is also used to determine the speech information and corresponding confidence level of the negative evaluation model based on the speech information of the affirmative evaluation model in the previous round. The speech information is used to indicate whether the corresponding evaluation model agrees with the other party's point of view and whether it insists on its own point of view. After a round of debate, the speech information of the evaluation models of both sides is stored to obtain a debate record of the round of debate; and after each round of debate, the confidence levels corresponding to the affirmative evaluation model and the negative evaluation model are determined. If the confidence levels of the two sides are different, the debate ends, and the optimal option selected by the evaluation model with a higher confidence level is used as the optimal option selected by the two evaluation models. If the confidence levels of the two sides are consistent, the next round of debate begins.

[0062] Specifically, we first instantiate two judging models (judging agents), one representing the affirmative side (i.e., the affirmative judging model mentioned above) and the other side (i.e., the negative judging model mentioned above). Their task is to select the optimal set of options (i.e., the optimal options mentioned above) from the multiple sets of options mentioned above and to debate their views with the other side. For ease of description, we will use "affirmative" to represent the affirmative judging model, and "negative" to represent the negative judging model.

[0063] First, the opinions are initialized (initialization phase). The affirmative party chooses a set of options that it thinks is better without role restrictions and gives corresponding reasons. The negative party is induced by prompt words to play the role that opposes the affirmative party's opinion, so that it gives a different opinion, that is, a different set of optimal options and corresponding reasons. As a result, the computing device will obtain two completely different opinion information.

[0064] Next, the debate officially begins. The affirmative side receives the previous round's speech from the negative side and is asked whether they agree with the negative side's point of view and whether they maintain their own position. Similarly, the negative side receives the previous round's speech and is asked whether they agree with the affirmative side's point of view and whether they maintain their own position. This allows the two sides to obtain information about their respective speeches. If the two sides fail to reach an agreement in this round, the next round of debate begins. After each round of debate, the speech information of both sides is stored by role, for example, in the format of "Affirmative Speech Record: First Round Speech, Second Round Speech..., Negative Speech Record:..."

[0065] To help conclude debates, the concept of confidence analysis is introduced. Each evaluation model is required to conduct a self-confidence assessment after each round of debate, thereby determining the confidence levels of both parties. For example, confidence levels can be categorized into four levels: very confident, moderately confident, uncertain, and incorrect. At the end of a round, if one side's confidence level is greater than the other, the other side is convinced, and the debate ends. The set of options considered by the more confident side (the one with the higher confidence level) is then output as the optimal option. Otherwise, if the confidence levels of both sides match, indicating that there is still disagreement, the debate continues. To prevent the debate from going on endlessly, a maximum number of rounds can be set, k. If disagreement persists after the maximum number of rounds, the optimal option selected by the affirmative side is automatically considered the optimal option selected by both evaluation models.

[0066] Then, enter Figure 3 In the revision phase, during this phase, the computing device revises the above optimal option based on the debate record obtained in the above debate.

[0067] Optionally, the operation of revising the optimal option based on the debate record by the revision agent to obtain the revised option specifically includes: revising the optimal option based on the objections to the optimal option in the debate record by the revision agent to obtain the revised option.

[0068] In other words, during the revision phase, the optimal option is adjusted by absorbing objections to it during the debate. The revision agent can incorporate reasonable objections to the optimal option as revision suggestions, thereby revising the optimal option selected during the debate. For example, in a debate, the affirmative side's viewpoint ultimately prevails, but this viewpoint is not necessarily "completely correct," nor are the opposing side's arguments necessarily "worthless." Therefore, the revision agent can extract useful criticism from the opposing side's arguments and revise the optimal option selected by the affirmative side, resulting in a revised option. This type of revision is consistent with human intuition.

[0069] After the computing device determines the revised option, it will enter Figure 4 The practice phase and the backtracking phase are described below. Since the conclusions drawn in the practice phase will be used to determine whether to enter the backtracking phase, the two phases will be introduced together below.

[0070] Optionally, the operation of answering the question content and the revised options by the question-solving intelligent agent to obtain the answer results specifically includes: combining the question content and the revised options into questions to be answered, and having the question-solving intelligent agent answer the questions to be answered n times to obtain corresponding n answer results, wherein the process of the question-solving intelligent agent answering n times is independent and does not share context.

[0071] Optionally, a reading comprehension test question is generated based on the content of the question stem, the revised options and the answer results, specifically including: counting the answers to the test questions that appear most frequently among the n answer results, judging whether the answers to the test questions that appear most frequently are consistent with the answers to the test questions corresponding to the revised options; if they are consistent, a reading comprehension test question is generated based on the question to be answered and the answers to the test questions that appear most frequently; and if they are inconsistent, the question to be answered is revised by a revising intelligent agent to obtain a revised question, the revised question is answered again by a question-solving intelligent agent to obtain the result of the revised answer, and a reading comprehension test question is generated based on the result of the revised answer and the revised question.

[0072] Practice Phase: In this phase, the fine-tuned problem-solving model is instantiated into a problem-solving agent. Because the problem-solving agent is trained using a large amount of data from real reading comprehension tests, it can be used as a highly skilled test-taker in real-world scenarios. The question stem (reading material and options) and the revised options from the previous phase are used as the question (i.e., the aforementioned unanswered question) and reconstructed as the input for fine-tuning the problem-solving model. After deep reflection, the problem-solving agent then provides its own answer based on what it believes to be the correct option. To prevent randomness, the problem-solving agent is asked to solve the same problem (i.e., the aforementioned unanswered question) n times, each independent and without sharing context. The most frequently appearing option (i.e., the most frequently appearing answer) is then counted across all n attempts. For example, if the problem-solving agent chooses option A three times, option B once, and option D once in five attempts, then option A is chosen as its answer. At this time, determine whether the answer of the test-solving agent (i.e., the answer to the test question that appears most frequently mentioned above) is consistent with the correct option determined when the question was set (i.e., the answer to the test question corresponding to the revised option mentioned above). If they are consistent, it means that high-level candidates can answer this question correctly most of the time, and this question can be used in practice. After the question-setting task of each agent is completed, the question to be answered and the answer to the test question that appears most frequently mentioned above can be used as the generated reading comprehension test question.

[0073] If there is a discrepancy, it means that even a highly skilled student will not be able to answer this question correctly most of the time. This question may be off-topic, strange, or even contain a logical error. This question can be considered unreasonable. Therefore, it is necessary to enter the backtracking stage and re-examine this question.

[0074] Among them, the correct option identified when the question is asked (i.e., the answer to the test question corresponding to the revised option mentioned above) may refer to the answer to the test question given when the option generation agent generates the optimal option corresponding to the revised option, or it may refer to the answer to the test question given when the revision agent generates the revised option.

[0075] Backtracking Phase: To correct unreasonable questions encountered during the practice phase, the computing device backtracks to the revision phase. While the revision agent remains unchanged, its input can be changed to the output of the problem-solving agent, the complete problem to be answered (which can include reading material, questions, and options), and the problem-setting record (which can include the revision record of the revision agent and the option-generating agent's option-generating record). The scenario faced by the revision agent at this point is similar to a confrontation between a high-level test-taker and an average-level problem-setting expert. Both parties provide their own problem-solving and problem-setting records. The revision agent first synthesizes this information and then further revises the problem to be answered. During this phase, the revision agent can not only modify the options, but also the reading material and questions to address the problem where unreasonable questions are caused by the question stem rather than the options, thereby obtaining a revised problem.

[0076] Then, the question-solving agent can continue to answer the revised questions. If the answer to the question that appears most frequently in the n answers given by the question-solving agent this time is the same as the answer to the question corresponding to the revised question, the revised question and the corresponding answer can be used as the generated reading comprehension question.

[0077] It's important to note that while the fine-tuned models (i.e., both the option generation model and the question-solving model mentioned above) maintain excellent performance on downstream tasks (i.e., tasks related to reading comprehension test generation), they decline in general tasks and instruction-following abilities. Therefore, across the five phases, only the option generation agent and the question-solving agent were instantiated using the fine-tuned models. The other agents were instantiated using the pre-trained large language model, which performs mediocrely on downstream tasks but better on general tasks. This design balances the strengths of both the fine-tuned and pre-trained large language models while avoiding their respective weaknesses, achieving a complementary multi-agent effect.

[0078] Furthermore, to ensure that each agent in a multi-agent system understands its responsibilities, system prompts must be designed for the option generation agent, the judging agent, the revision agent, and the test-solving agent. For example, the system prompt for the generation agent could be: "You are an expert in generating reading comprehension questions, {Several test-solving demonstrations}"; the system prompt for the test-solving agent could be: "You are an expert in solving reading comprehension questions, {Several test-solving demonstrations}." These demonstrations can be carefully designed, and the few-shot approach has proven effective in this regard.

[0079] Next, the specific implementation method of the above-mentioned five-stage multi-agent collaboration is introduced, in which the specific behavior of each agent can be induced by prompt words.

[0080] The option generation agent and the problem-solving agent are instantiated from the fine-tuned model. Because the fine-tuned model is in safetensor format using the Transformers training framework, it needs to be converted to the gguf format required by Ollama using the llama.cpp tool. 4-bit quantization is then performed to complete the Ollama model creation. Langgraph's ChatOpenAI and create_react_agent methods are then used to instantiate the fine-tuned model from the local Ollama as an agent. Due to resource constraints, only two option generation agents can be instantiated, with inference parameters set to temperature=0.8, top-p=0.9, top-k=40; and temperature=0.7, top-p=0.8, top-k=30, respectively. For the judgment and revision agents, the Deepseek-R1-32B pre-trained model deployed locally on Ollama can be used and also instantiated using Langgraph. Finally, multiple agents can be arranged using Langgraph's graph structure, following a five-stage collaborative structure.

[0081] Debate records can be obtained by maintaining message queues. Message queues during the debate phase can be divided into two types: role message queues and debate message queues. Each judging model corresponds to a role message queue and debate message queue. The role message queue only maintains all speeches made by either side during the debate. For example, the affirmative side's role message queue records all speeches made in the first, second, and so on rounds. The debate message queue maintains a record of debate messages for a particular side, constructed by interleaving role message queues. Specifically, a debate record for a side includes not only its own speeches but also the opponent's rebuttals and speeches in each round. For example, the affirmative side's debate message queue might have a structure like: task description instructions - affirmative side's initial viewpoint - opposition speech - affirmative speech - opposition speech... Similarly, the opposition side's debate message queue might have a structure like: task description instructions - opposition initial viewpoint - affirmative speech - opposition speech - affirmative speech... Existing debate messages are treated as context and serve as memory for the agent's debate. Therefore, a complete debate record can be obtained from the messages in the role message queue and debate message queue.

[0082] Among them, since the objections to the optimal option in the debate stage will be needed in the subsequent revision stage, it is possible to determine which party selected the optimal option in the initialization stage, and determine the evaluation model corresponding to this party, and then determine the role message queue of the evaluation model opposite to the evaluation model, and determine the objections to the optimal option from the role message queue.

[0083] Specifically, post-debate revisions can first identify the winning viewpoint, using the affirmative side's perspective as an example. Next, the entire message queue of the opposing side's character can be fully accessed. The revision agent can then extract critical suggestions against the affirmative side's viewpoint (i.e., the aforementioned objections to the optimal option), while retaining irrelevant information such as the option set selected by the opposing side. Finally, the revision agent can combine these critical suggestions to revise the affirmative side's "optimal" option set to a "better" one, resulting in the revised options.

[0084] The input for the revision phase during the backtracking phase is relatively complex. To prevent the revision agent from having difficulty understanding the entire input, the input is primarily divided into three parts: the question record, the complete question, and the answer record. The complete question is input as a JSON dictionary, with keys including the question content, options (which can include A, B, C, and D), and the answer. The answer record, which includes the question-solving agent's thought process and the final option chosen, is also input in a JSON format that is easier for the model to understand. The question record needs to be streamlined and compressed. The revision agent first generates a summary of the original question record (which may include the output of the option generation agent) and the revised record. It then reconstructs the latest question record based on the question-generating logic of the revised summary. Finally, these three parts of input are embedded in a prompt template and supplemented with standard instructions to produce the revised question.

[0085] Figure 5 This is a schematic diagram of an example of a generated reading comprehension test question provided in Example 1 of the present disclosure.

[0086] A complete example of generating a reading comprehension question is available at Figure 5Based on the retrieved question stem (including the reading material and the question), the two option-generating agents generated two distinct sets of options. The first group identified the core message of the material as "resisting etiquette," while the second group identified it as "practicing one's natural instincts." The debate phase, conducted by the evaluation model, saw the pros and cons each choose a different option as the "optimal option" and refute the opposing side's arguments. Because the cons' confidence level was higher than the pros' in the second round, the "optimal" option converged on the second set. Entering the revision phase, the revision agent leveraged the pros' criticisms (objections) during the debate, incorporating "resistance" into option A and modifying the original option D. However, since the problem-solving agent preferred option A (choosing it three times out of five attempts), this was inconsistent with the original thinking, prompting the retrospective revision phase. The revision agent directly selected the key sentence from the source material as the correct option, while also using the previous option, slightly modified, as a distractor. The final version of the problem was given to the problem-solving agent, which correctly answered all five questions in its first attempt, demonstrating the problem's effectiveness. Furthermore, the three distractors each had their own unique characteristics, highlighting the problem's disruptive nature. The multi-agent collaborative process progressed step by step, with the optimization of each stage's problem version fully demonstrating the indispensability of each stage and each agent.

[0087] In addition, reference Figure 1 As shown, according to a second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0088] Therefore, according to this embodiment, the diversity and logical rigor of reading comprehension test question generation can be improved, the coarse-grained retrieval problem existing in the prior art can be solved, and training samples for fine-tuning the pre-trained basic model can be generated through automatic annotation, thereby reducing the cost of generating training samples.

[0089] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0090] Through the description of the above embodiments, those skilled in the art will clearly understand that the methods according to the above embodiments can be implemented using software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk) and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0091] Example 2 Figure 6 The apparatus 600 for generating reading comprehension questions based on a large language model and multiple agents according to the first aspect of this embodiment is shown. The apparatus 600 corresponds to the method according to the first aspect of embodiment 1. Figure 6 As shown, the device 600 includes: an acquisition module 610, which is used to acquire keywords input by the user corresponding to the reading comprehension test questions to be generated; a question stem generation module 620, which is used to generate question stem content corresponding to the reading comprehension test questions according to the keywords through a question stem generation model based on RAG, wherein the question stem content includes reading materials and questions; an option generation module 630, which is used to generate an intelligent agent through multiple options, and generate multiple groups of options corresponding to the questions based on the question stem content, wherein the multiple groups of options are different; a debate module 640, which is used to debate which group of options is the best among the multiple groups of options through two preset evaluation models, and obtain a debate record and the best option selected by the two evaluation models; a revision module 650, which is used to revise the best option based on the debate record through a revision intelligent agent, and obtain a revised option; an answer module 660, which is used to answer the question stem content and the revised options through a question-solving intelligent agent, and obtain an answer result; and a test question generation module 670, which is used to generate a reading comprehension test question based on the question stem content, the revised options and the answer result.

[0092] Optionally, the question stem generation module 620 is specifically used to search in a preset vector database based on the keyword vector corresponding to the keyword to obtain corresponding search results; fill the search results into a preset prompt word template to obtain reconstructed prompt words, wherein the prompt word template contains real reading comprehension test questions as a demonstration; and input the reconstructed prompt words into the question stem generation model to generate the question stem content.

[0093] Optionally, before generating multiple groups of options corresponding to the questions based on the question stem content through multiple option generation agents, the device 600 also includes a training module 680 for instantiating the option generation model based on the pre-trained basic model to obtain multiple option generation agents, wherein the option generation model is obtained by the following operations: obtaining standard questions in the standard question set, the standard questions are obtained by converting real reading comprehension test questions into structured data containing a reading material field, a question field, an option field and a test answer field; generating corresponding option generation logic information based on the question stem content of the standard question through a preset large language model, the option generation logic information is used to represent the thinking process of generating corresponding options according to the question stem content in the standard question; constructing an option generation training set based on the question stem content, corresponding options and corresponding option generation logic information of the standard questions in the standard question set; and fine-tuning the pre-trained basic model based on the option generation training set to obtain an option generation model.

[0094] Optionally, before the question-solving intelligent agent answers the question content and the revised options and obtains the answer result, the training module 680 is also used to instantiate the question-solving model based on the pre-trained basic model to obtain the question-solving intelligent agent, wherein the question-solving model is obtained by the following operations: obtaining standard questions in the standard question set, the standard questions are obtained by converting real reading comprehension test questions into structured data containing reading material fields, question fields, option fields and test answer fields; based on the question content of the standard questions, generating question-solving idea information and test answer selected from the options corresponding to the standard questions through a preset large language model; when the selected test answer matches the test answer corresponding to the standard question, constructing a question-solving training set according to the reading materials, questions, options, test answers and question-solving idea information corresponding to the standard question; and fine-tuning the pre-trained basic model according to the question-solving training set to obtain the question-solving model.

[0095] Optionally, the debate module 640 is specifically used for an initialization phase, in which the two evaluation models are divided into an affirmative evaluation model and an adverse evaluation model, and the viewpoint information of the affirmative evaluation model and the adverse evaluation model are determined, wherein the viewpoint information of the affirmative evaluation model includes the optimal option selected by the affirmative evaluation model from multiple groups of options and the corresponding reasons, and the viewpoint information of the adverse evaluation model includes the optimal option selected by the adverse evaluation model from multiple groups of options and the corresponding reasons, wherein the optimal option selected by the adverse evaluation model is inconsistent with the optimal option selected by the affirmative evaluation model; in the debate phase, in each round of debate in the debate phase, the affirmative evaluation model is determined based on the speech information of the adverse evaluation model in the previous round. The speech information and the corresponding confidence level, and the speech information and corresponding confidence level of the opposing evaluation model are determined based on the speech information of the affirmative evaluation model in the previous round through the opposing evaluation model, wherein the speech information is used to indicate whether the corresponding evaluation model agrees with the other party's point of view and whether it insists on its own point of view. After a round of debate, the speech information of the evaluation models of both sides is stored to obtain a debate record of a round of debate; and after each round of debate, the confidence levels corresponding to the affirmative evaluation model and the opposing evaluation model are determined. If the confidence levels of the two sides are different, the debate ends, and the optimal option selected by the evaluation model with a higher confidence level is used as the optimal option selected by the two evaluation models. If the confidence levels of the two sides are consistent, the next round of debate begins.

[0096] Optionally, the revision module 650 is specifically configured to revise the optimal option based on the opposing opinions against the optimal option in the debate record through a revision agent to obtain a revised option.

[0097] Optionally, the option generation module 630 is specifically used to input the question stem content into multiple option generation agents, each option generation agent outputs a corresponding set of options and corresponding test answers, and the answering module 660 is specifically used to combine the question stem content and the revised options into a question to be answered, and the question-solving agent answers the question to be answered n times to obtain corresponding n answer results, wherein the process of the question-solving agent answering n times is independent and does not share context, and the test generation module 670 is specifically used to count the test answers that appear most frequently in the n answer results, and determine whether the test answers that appear most frequently are consistent with the test answers corresponding to the revised options. If they are consistent, the reading comprehension test question is generated based on the question to be answered and the test answers that appear most frequently; and if they are inconsistent, the question to be answered is revised by the revision agent to obtain a revised question, the revised question is answered again by the question-solving agent to obtain the result of the revised answer, and the reading comprehension test question is generated based on the result of the revised answer and the revised question.

[0098] Therefore, according to this embodiment, the diversity and logical rigor of reading comprehension test question generation can be improved, the coarse-grained retrieval problem existing in the prior art can be solved, and training samples for fine-tuning the pre-trained basic model can be generated through automatic annotation, thereby reducing the cost of generating training samples.

[0099] Example 3 Figure 7 The apparatus 700 for generating reading comprehension questions based on a large language model and multiple agents according to the first aspect of this embodiment is shown. The apparatus 700 corresponds to the method according to the first aspect of embodiment 1. Figure 7 As shown, the device 700 includes: a processor 710; and a memory 720, which is connected to the processor 710 and is used to provide instructions for the processor 710 to process the following processing steps: obtaining keywords input by the user corresponding to the reading comprehension test questions to be generated; generating stem content corresponding to the reading comprehension test questions according to the keywords through a stem generation model based on RAG, wherein the stem content includes reading materials and questions; generating an intelligent agent through multiple options, and generating multiple groups of options corresponding to the questions based on the stem content, wherein the multiple groups of options are different; through two preset evaluation models, debate is conducted on which group of options among the multiple groups of options is the best, and a debate record and the best option selected by the two evaluation models are obtained; revising the best option based on the debate record through a revising intelligent agent to obtain a revised option; answering the stem content and the revised option through a question-solving intelligent agent to obtain an answer result; and generating a reading comprehension test question based on the stem content, the revised options and the answer result.

[0100] Optionally, based on keywords, the operation of generating stem content corresponding to the reading comprehension test questions through the RAG-based stem generation model includes: searching in a preset vector database based on the keyword vector corresponding to the keyword to obtain corresponding search results; filling the search results into a preset prompt word template to obtain reconstructed prompt words, wherein the prompt word template includes real reading comprehension test questions as demonstrations; and inputting the reconstructed prompt words into the stem generation model to generate stem content.

[0101] Optionally, before generating multiple groups of options corresponding to the questions based on the content of the question stem through multiple option generation agents, the memory 720 is also used to provide instructions for the processor 710 to process the following processing steps: instantiating the option generation model based on the pre-trained basic model to obtain multiple option generation agents, wherein the option generation model is obtained by the following operations: obtaining standard questions in the standard question set, the standard questions are obtained by converting real reading comprehension test questions into structured data containing a reading material field, a question field, an option field and a test answer field; generating corresponding option generation logic information based on the content of the question stem of the standard question through a preset large language model, the option generation logic information is used to represent the thinking process of generating corresponding options according to the content of the question stem in the standard question; constructing an option generation training set based on the content of the question stem of the standard question in the standard question set, the corresponding options and the corresponding option generation logic information; and fine-tuning the pre-trained basic model based on the option generation training set to obtain the option generation model.

[0102] Optionally, before the question-solving intelligent agent answers the question content and the revised options and obtains the answer result, the memory 720 is also used to provide the processor 710 with instructions for processing the following processing steps: instantiating the question-solving model based on the pre-trained basic model to obtain the question-solving intelligent agent, wherein the question-solving model is obtained by the following operations: obtaining a standard question in the standard question set, the standard question is obtained by converting a real reading comprehension test question into structured data containing a reading material field, a question field, an option field and a test answer field; based on the question content of the standard question, generating question-solving idea information and test answers selected from the options corresponding to the standard question through a preset large language model; when the selected test answer matches the test answer corresponding to the standard question, constructing a question-solving training set according to the reading material, question, option, test answer and question-solving idea information corresponding to the standard question; and fine-tuning the pre-trained basic model according to the question-solving training set to obtain a question-solving model.

[0103] Optionally, a debate is conducted through two preset evaluation models on which group of options is the best, and the debate record and the best options selected by the two evaluation models are obtained, which specifically includes: an initialization stage, in which the two evaluation models are divided into an affirmative evaluation model and an adverse evaluation model, and the viewpoint information of the affirmative evaluation model and the adverse evaluation model are determined, wherein the viewpoint information of the affirmative evaluation model includes the best options selected by the affirmative evaluation model from the multiple groups of options and the corresponding reasons, and the viewpoint information of the adverse evaluation model includes the best options selected by the adverse evaluation model from the multiple groups of options and the corresponding reasons, wherein the best option selected by the adverse evaluation model is inconsistent with the best option selected by the affirmative evaluation model; a debate stage, in each round of debate in the debate stage, the affirmative evaluation model is used to determine the best option selected by the adverse evaluation model based on the affirmative evaluation model. Based on the speech information of the negative evaluation model in the previous round, the speech information and the corresponding confidence level of the affirmative evaluation model are determined; and based on the speech information of the positive evaluation model in the previous round, the speech information and the corresponding confidence level of the negative evaluation model are determined through the negative evaluation model, wherein the speech information is used to indicate whether the corresponding evaluation model agrees with the other party's point of view and whether it insists on its own point of view. After a round of debate, the speech information of the evaluation models of both sides is stored to obtain a debate record of the debate; and after each round of debate, the confidence levels corresponding to the affirmative evaluation model and the negative evaluation model are determined. If the confidence levels of the two sides are different, the debate ends, and the optimal option selected by the evaluation model with a higher confidence level is used as the optimal option selected by the two evaluation models. If the confidence levels of the two sides are consistent, the next round of debate begins.

[0104] Optionally, the operation of revising the optimal option based on the debate record by the revision agent to obtain the revised option specifically includes: revising the optimal option based on the objections to the optimal option in the debate record by the revision agent to obtain the revised option.

[0105] Optionally, multiple option generation agents are used to generate multiple groups of options corresponding to the questions based on the question stem content, specifically including: inputting the question stem content into multiple option generation agents, each option generation agent outputting a corresponding group of options and corresponding test answers, and wherein the question-solving agent answers the question stem content and the revised options to obtain the answer results, specifically including: the question stem content and the revised options form a question to be answered, the question-solving agent answers the question to be answered n times, and obtains corresponding n answer results, wherein the process of the question-solving agent answering n times is independent and does not share context. The method comprises the following steps: first, counting the answers to the most frequently appearing questions in n answer results, and judging whether the answers to the most frequently appearing questions are consistent with the answers to the questions corresponding to the revised options; if they are consistent, generating reading comprehension questions based on the questions to be answered and the answers to the most frequently appearing questions; and second, revising the questions to be answered through a revising intelligent agent to obtain revised questions, answering the revised questions again through a question-solving intelligent agent to obtain the results of the revised answers, and generating reading comprehension questions based on the results of the revised answers and the revised questions.

[0106] Therefore, according to this embodiment, the diversity and logical rigor of reading comprehension test question generation can be improved, the coarse-grained retrieval problem existing in the prior art can be solved, and training samples for fine-tuning the pre-trained basic model can be generated through automatic annotation, thereby reducing the cost of generating training samples.

[0107] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0108] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0109] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0110] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0111] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0112] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), a mobile hard drive, a magnetic disk, or an optical disk.

[0113] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for generating reading comprehension test questions based on a large language model and multiple agents, characterized in that: include: Obtain keywords input by the user corresponding to the reading comprehension test questions to be generated; According to the keywords, generating stem content corresponding to the reading comprehension test question through a stem generation model based on RAG, wherein the stem content includes reading materials and questions; Generate an agent through multiple options, and generate multiple groups of options corresponding to the question based on the content of the question stem, wherein the multiple groups of options are different; Using two preset evaluation models, debate is conducted on which of the multiple groups of options is optimal, and a debate record and the optimal option selected by the two evaluation models are obtained; revising the optimal option based on the debate record by a revision agent to obtain a revised option; The intelligent agent answers the question stem and the revised options to obtain an answer result; and The reading comprehension test question is generated according to the question stem content, the revised options and the answer result.

2. The method according to claim 1, characterized in that The operation of generating the stem content corresponding to the reading comprehension test question using the RAG-based stem generation model according to the keyword includes: Searching a preset vector database based on the keyword vector corresponding to the keyword to obtain corresponding search results; Filling the search results into a preset prompt word template to obtain a reconstructed prompt word, wherein the prompt word template includes a real reading comprehension test question as a demonstration; and The reconstructed prompt words are input into the question stem generation model to generate the question stem content.

3. The method according to claim 1, characterized in that Before generating an agent through multiple options and generating multiple groups of options corresponding to the question based on the question stem content, the method further includes: Instantiate the option generation model based on the pre-trained basic model to obtain the multiple option generation agents, wherein the option generation model is obtained by the following operations: Obtaining standard questions from a standard question set, wherein the standard questions are obtained by converting real reading comprehension test questions into structured data including a reading material field, a question field, an option field, and a test answer field; Generate corresponding option generation logic information based on the content of the standard question stem using a preset large language model, wherein the option generation logic information is used to represent the thought process of generating corresponding options based on the content of the standard question stem; Constructing an option generation training set based on the content of the standard questions in the standard question set, the corresponding options, and the corresponding option generation logic information; and A training set is generated according to the options, and the pre-trained basic model is fine-tuned to obtain the option generation model.

4. The method according to claim 1, wherein Before obtaining the answer result by the question-solving agent answering the question stem and the revised options, the method further includes: Instantiate the question-solving model based on the pre-trained basic model to obtain the question-solving agent, wherein the question-solving model is obtained by the following operations: Obtaining standard questions from a standard question set, wherein the standard questions are obtained by converting real reading comprehension test questions into structured data including a reading material field, a question field, an option field, and a test answer field; Based on the content of the standard question, a preset large language model is used to generate information on problem-solving strategies and test answers selected from the options corresponding to the standard question; When the selected test questions and the answers to the test questions corresponding to the standard questions match, constructing a test question training set based on the reading materials, questions, options, test questions and the problem-solving ideas corresponding to the standard questions; and According to the problem-solving training set, the pre-trained basic model is fine-tuned to obtain a problem-solving model.

5. The method according to claim 1, wherein The operation of debating which of the multiple groups of options is optimal by using two preset evaluation models, and obtaining a debate record and the optimal options selected by the two evaluation models specifically includes: an initialization stage, in which the two evaluation models are divided into an affirmative evaluation model and a negative evaluation model, and the opinion information of the affirmative evaluation model and the negative evaluation model are determined, wherein the opinion information of the affirmative evaluation model includes the optimal option selected by the affirmative evaluation model from the multiple groups of options and the corresponding reasons, and the opinion information of the negative evaluation model includes the optimal option selected by the negative evaluation model from the multiple groups of options and the corresponding reasons, wherein the optimal option selected by the negative evaluation model is inconsistent with the optimal option selected by the affirmative evaluation model; In the debate stage, in each round of debate in the debate stage, the affirmative evaluation model determines, based on the speech information of the negative evaluation model in the previous round, the speech information and the corresponding confidence level of the affirmative evaluation model; and the negative evaluation model determines, based on the speech information of the affirmative evaluation model in the previous round, the speech information and the corresponding confidence level of the negative evaluation model, wherein the speech information is used to indicate whether the corresponding evaluation model agrees with the other party's viewpoint and whether it adheres to its own viewpoint. After the end of a round of debate, the speech information of the evaluation models of both parties is stored to obtain a debate record of the round of debate; and After each round of debate, the confidence levels corresponding to the affirmative and negative evaluation models are determined. If the confidence levels of the two sides are different, the debate ends, and the optimal option selected by the evaluation model with a higher confidence level is used as the optimal option selected by the two evaluation models. If the confidence levels of the two sides are the same, the next round of debate begins.

6. The method according to claim 1, wherein The operation of revising the optimal option based on the debate record by the revision agent to obtain the revised option specifically includes: The revision agent revises the optimal option based on the objections to the optimal option in the debate record to obtain a revised option.

7. The method according to claim 1, characterized in that The operation of generating an agent through multiple options and generating multiple groups of options corresponding to the question based on the content of the question stem specifically includes: The question content is input into the multiple option generation agents, each option generation agent outputs a corresponding set of options and a corresponding test answer, and wherein The operation of answering the question stem content and the revised options by the question-solving agent to obtain the answer result specifically includes: combining the question stem content and the revised options into a question to be answered, answering the question to be answered n times by the question-solving agent, and obtaining corresponding n answer results, wherein the process of the question-solving agent answering n times is independent and does not share context, and wherein Generating the reading comprehension test question according to the question stem content, the revised options, and the answer result, specifically includes: Counting the most frequently appearing answers to the test questions among the n answer results, determining whether the most frequently appearing answers to the test questions are consistent with the answers to the test questions corresponding to the revised options, and if they are consistent, generating the reading comprehension test question based on the question to be answered and the most frequently appearing answers to the test questions; and If there is any inconsistency, the question to be answered is revised by the revision agent to obtain a revised question, the revised question is answered again by the question-solving agent to obtain a result of the revised answer, and the reading comprehension test question is generated based on the result of the revised answer and the revised question.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the processor executes the method according to any one of claims 1 to 7.

9. A reading comprehension test generation device based on a large language model and multiple agents, characterized in that: include: An acquisition module, used to acquire keywords input by the user corresponding to the reading comprehension test questions to be generated; A question stem generation module, configured to generate, based on the keywords, question stem content corresponding to the reading comprehension test question using a RAG-based question stem generation model, wherein the question stem content includes reading materials and questions; An option generation module, configured to generate an agent from a plurality of options, and to generate a plurality of groups of options corresponding to the question based on the content of the question stem, wherein the plurality of groups of options are different from each other; A debate module is used to debate which of the multiple groups of options is the best using two preset evaluation models, and obtain a debate record and the best option selected by the two evaluation models; A revision module, configured to revise the optimal option based on the debate record by a revision agent to obtain a revised option; An answering module, configured to answer the question stem and the revised options through a question-answering agent to obtain an answer result; and The test question generating module is used to generate the reading comprehension test question according to the content of the question stem, the revised options and the answer result.

10. A reading comprehension test generation device based on a large language model and multiple agents, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Obtain keywords input by the user corresponding to the reading comprehension test questions to be generated; According to the keywords, generating stem content corresponding to the reading comprehension test question through a stem generation model based on RAG, wherein the stem content includes reading materials and questions; Generate an agent through multiple options, and generate multiple groups of options corresponding to the question based on the content of the question stem, wherein the multiple groups of options are different; Using two preset evaluation models, debate is conducted on which of the multiple groups of options is optimal, and a debate record and the optimal option selected by the two evaluation models are obtained; revising the optimal option based on the debate record by a revision agent to obtain a revised option; The intelligent agent answers the question stem and the revised options to obtain an answer result; and The reading comprehension test question is generated according to the question stem content, the revised options and the answer result.

Images