Intelligent question setting method and system

Abstract

The invention provides an intelligent question setting method and system. The intelligent question setting method comprises the following steps: extracting courseware content; extracting knowledge points in the courseware content, generating a first embedded vector of the knowledge points, and storing the first embedded vector in a vector database; receiving a question setting task, obtaining a to-be-investigated knowledge point in the question setting task, generating a second embedding vector of the to-be-investigated knowledge point, querying a first embedding vector similar to the second embedding vector from the vector database, and taking the queried first embedding vector as a recalled text; a first cue word is generated according to the to-be-investigated knowledge point and the recalled text, the first cue word is input into the large question setting model, test questions output by the large question setting model are obtained, and the first cue word is used for prompting the large question setting model to generate the test questions based on the content in the first cue word. The test questions containing the knowledge points to be investigated can be automatically, efficiently and accurately generated by utilizing the large model.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to an intelligent question generation method and system. Background Technology

[0002] The existing training system faces numerous challenges, such as homogenized content, insufficient practical application, and a lack of diverse assessment methods. In particular, when assessing students' knowledge acquisition, teachers need to prepare a large number of exercises to reinforce learning. Traditional question-setting methods typically rely on manual design by teachers or experts, which is not only time-consuming and labor-intensive but also difficult to accurately assess students' mastery of specific knowledge points. Summary of the Invention

[0003] This invention provides an intelligent question generation method and system to solve the problems of existing manual question generation methods being time-consuming and labor-intensive, and making it difficult to accurately assess students' mastery of specific knowledge points.

[0004] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0005] In a first aspect, embodiments of the present invention provide an intelligent question generation method, including:

[0006] Extract courseware content;

[0007] Extract the knowledge points from the courseware content, generate the first embedding vector of the knowledge points, and store the first embedding vector in the vector database;

[0008] Receive a question-generating task and obtain the knowledge points to be tested in the question-generating task. Generate a second embedding vector of the knowledge points to be tested. Query a first embedding vector similar to the second embedding vector from the vector database. Use the queried first embedding vector as the recall text.

[0009] A first prompt word is generated based on the knowledge point to be examined and the recall text. The first prompt word is then input into the question generation model to obtain the test questions output by the question generation model. The first prompt word is used to prompt the question generation model to generate test questions based on the content in the first prompt word.

[0010] Optionally, the extracted courseware content includes:

[0011] Automatically identify courseware types based on file extensions and / or file header information;

[0012] The extraction tool corresponding to the courseware type is determined, and the courseware content is extracted using the extraction tool. The extraction tool includes a multimodal large model and / or a parsing toolkit. The determination of the extraction tool corresponding to the courseware type includes: determining the extraction tool corresponding to the courseware type according to a pre-set correspondence between the courseware type and the extraction tool, or automatically configuring the extraction tool for the courseware type according to the identified courseware type.

[0013] When the courseware type is an image-based PDF courseware, the parsing toolkit is used to save the PDF courseware page by page as images, prompt words are generated based on the images, and the prompt words containing the images are input into the multimodal large model to recognize the image content;

[0014] When the courseware type is image courseware, prompt words are generated based on the image courseware, and the prompt words containing the image courseware are input into the multimodal large model to recognize the image content; if the multimodal large model recognizes that the image contains a table, flowchart or diagram, the parsing toolkit is used to extract the image containing the table, flowchart or diagram.

[0015] When the courseware type is video or audio courseware, prompt words are generated based on the video or audio courseware, and the prompt words containing the video or audio courseware are input into the multimodal large model. The multimodal large model is then used to convert the video or audio information into text.

[0016] Optionally, the extraction of knowledge points from the courseware content includes:

[0017] A second prompt word is generated based on the reference knowledge points and the courseware content. The second prompt word is used to prompt the extraction of knowledge points from the courseware content based on the content of the second prompt word.

[0018] Input the second prompt word into the knowledge point extraction model to obtain the knowledge points output by the knowledge point extraction model.

[0019] The courseware is designed for the industrial manufacturing sector, and the reference knowledge points include at least one of the following: industry terminology, equipment name, malfunctions, causes of malfunctions, countermeasures, malfunction detection methods, malfunction resolution measures, operating procedures, equipment parts, and abnormal handling.

[0020] Optionally, the generation of the first embedding vector for the knowledge point includes:

[0021] The knowledge points are input into a text embedding model to obtain the first embedding vector of the knowledge points output by the text embedding model. The text embedding model is a large language model.

[0022] Optional, also includes:

[0023] The knowledge points are classified based on a pre-trained text classifier to obtain the type of the knowledge points;

[0024] The types of the knowledge points are stored in the vector database;

[0025] The step of querying the vector database for a first embedding vector similar to the second embedding vector includes:

[0026] Based on the type of the knowledge point to be examined, a first embedding vector similar to the second embedding vector is recalled from the vector database, and the type of the recalled first embedding vector is the same as the type of the knowledge point to be examined.

[0027] Optionally, after generating the first embedding vector of the knowledge point, the method further includes:

[0028] Obtain multiple pre-set reference knowledge points;

[0029] For each of the reference knowledge points, the similarity score between the first embedding vector and the embedding vector of the reference knowledge point is calculated, and the first embedding vector is sorted based on the similarity score of the first embedding vector to obtain the sorting information of the first embedding vector.

[0030] The sorting information of the first embedded vector is stored in the vector database;

[0031] The step of generating a first prompt word based on the knowledge point to be examined and the recalled text, inputting the first prompt word into the question-generating model, and obtaining the test questions output by the question-generating model includes:

[0032] The question-generating model generates questions based on the ranking information of the first embedding vectors corresponding to the recalled text. If the ranking information is sorted in order of similarity score from high to low, the top N first embedding vectors are selected from the multiple first embedding vectors recalled for the knowledge point to be examined, and N is an integer greater than or equal to 1.

[0033] The courseware is designed for the industrial manufacturing sector, and the reference knowledge points include at least one of the following: industry terminology, equipment name, malfunctions, causes of malfunctions, countermeasures, malfunction detection methods, malfunction resolution measures, operating procedures, equipment parts, and abnormal handling.

[0034] Optionally, the first prompt may further include at least one of the following: the type of knowledge point to be tested, the difficulty level of the question, the question type, the proportion of question types, the number of questions, the output format of the test questions, and the test level; wherein, the test level includes at least one of the following: memory-level test questions, comprehension-level test questions, application-level test questions, and innovation-level test questions;

[0035] The first prompt word generated based on the knowledge point to be examined and the recall text includes at least one of the following:

[0036] First instruction information: The instruction combines randomness and preset rules to generate various types of questions;

[0037] The second instruction is to adjust the difficulty level of the test questions according to the learning progress of the candidates, and to randomly generate test questions within the corresponding difficulty level.

[0038] The third instruction is to dynamically adjust the proportion of question types generated based on the learning history and preferences of the candidates to be assessed.

[0039] Fourth instruction information: The instruction is to generate test questions at the test question level that meet the task requirements indicated in the test question task or the cognitive entropy information of the test candidate.

[0040] Among them, the memory-level test questions are those with unique answers and require no reasoning;

[0041] The comprehension-level questions are those with a single answer and require reasoning.

[0042] The application-level test questions are those that require multi-step calculations or modeling;

[0043] The innovative-level test questions are those that require the formulation of new questions, the design of experiments, and / or the demonstration of generalization.

[0044] Optionally, after obtaining the test questions output by the large-scale question-generating model, the method further includes:

[0045] Check for any unqualified test questions and remove them.

[0046] The unqualified test questions include at least one of the following:

[0047] Questions with repeated options;

[0048] Questions with different questions but the same answer choices;

[0049] Questions that do not meet the requirements for setting questions;

[0050] Questions with a similarity greater than a threshold;

[0051] Among these methods, regular expression matching algorithms, semantic similarity detection algorithms, or large question-generating models are used to check for the existence of unqualified test questions.

[0052] Optional, also includes:

[0053] A third prompt word is generated based on the test questions and assessment dimensions. The third prompt word is used to prompt the generated test questions to be assessed.

[0054] The third prompt word is input into the evaluation model to obtain the score output by the evaluation model;

[0055] Obtain scores for multiple evaluation dimensions, and obtain a total score based on the scores for the multiple evaluation dimensions;

[0056] The evaluation dimensions include at least one of the following: content accuracy, diversity, difficulty level, and reference answer.

[0057] Optionally, the question-generating model includes at least one of the following modules:

[0058] The memory layer module is used to generate memory-level test questions, which are test questions with unique answers and require no reasoning.

[0059] The comprehension layer module is used to generate comprehension-level test questions, which are questions with unique answers and require reasoning.

[0060] The application layer module is used to generate application-level test questions, which are test questions that require multi-step calculations or modeling.

[0061] The innovation layer module is used to generate innovation-level test questions, which are test questions that require raising new questions, designing experiments, and / or proving generalizations.

[0062] The question-generating model uses a dynamic routing algorithm to map the first prompt word to the corresponding module based on the task requirements indicated in the first prompt word or the cognitive entropy information of the examinee, and the corresponding module generates the test questions.

[0063] Optionally, the method further includes:

[0064] Obtain the training dataset;

[0065] Based on the training dataset, the initial question-generating model is trained.

[0066] The training dataset includes at least one of the following:

[0067] The first dataset includes multiple memory-level test question samples, and the data source of the memory-level test question samples comes from at least one of the following: enterprise safety manual, relevant safety regulations, and equipment manual;

[0068] The second dataset includes multiple comprehension-level test question samples, the data sources of which are at least one of the following: process specifications, fault case library, and process principle courseware;

[0069] The third dataset includes multiple application-level test question samples, and the data sources of the application-level test question samples are at least one of the following: simulation operation logs, accident records, and work instructions;

[0070] The fourth dataset includes multiple innovative test question samples, the data sources of which are at least one of the following: technology improvement proposals, quality control team results, and patent documents;

[0071] The first dataset is used to train the memory layer module of the question-generating model;

[0072] The second dataset is used to train the understanding layer module of the question-generating model;

[0073] The third dataset is used to train the application layer module of the question-generating model;

[0074] The fourth dataset is used to train the innovation layer module of the question-generating model.

[0075] Optionally, obtaining the training dataset includes:

[0076] Obtain prompt word templates for generating memory-level test question samples, comprehension-level test question samples, application-level test question samples, and innovation-level test question samples;

[0077] For each question level, the courseware content is extracted from the corresponding courseware. Based on the extracted courseware content and the prompt word template, prompt words are generated and input into the large language model to obtain the question samples output by the large language model.

[0078] The training dataset is obtained based on the test sample corresponding to each test level.

[0079] Optionally, obtaining the training dataset further includes at least one of the following:

[0080] The generated test sample is automatically labeled using a large language model;

[0081] The generated test question samples are filtered according to rules to determine whether the test question samples meet the rules of the corresponding test question level;

[0082] Random sampling inspection of the generated test question samples;

[0083] The generated test sample is reclassified using a classifier. The classification result is compared with the label of the test sample, and test sample with consistency less than a preset threshold is discarded.

[0084] The generated test question samples are deduplicated;

[0085] Ensure that the ratio of memory-level test questions, comprehension-level test questions, application-level test questions, and innovation-level test questions in the training dataset is 1:1:1:1.

[0086] Optional, also includes:

[0087] Obtain the test questions output by the aforementioned question-generating model;

[0088] A feedback optimization model is used to predict the feedback labels of the test questions output by the question-generating model; the feedback labels include positive or negative feedback;

[0089] Based on the test questions output by the question-generating model and the feedback tags, the question-generating model is optimized.

[0090] Optional, also includes:

[0091] Acquire multiple historical sample data, including historical test questions output by the question-generating model and historical feedback tags for the historical test questions;

[0092] The initial feedback optimization model is trained based on the multiple historical sample data to obtain the trained feedback optimization model.

[0093] The loss function used to train the feedback optimization model is:

[0094]

[0095] in, Denotes the target network, (s,a,r,s) ′ U(D) represents the empirical data, and U(D) represents the memory unit for replaying the empirical data.

[0096] The step of training the initial feedback optimization model based on the multiple historical sample data includes:

[0097] Experience data is randomly extracted from the playback memory unit for training, and the error is obtained by taking the partial derivative of the loss function with respect to the parameter θ.

[0098] Secondly, embodiments of the present invention provide an intelligent question-generating system, comprising:

[0099] The courseware content extraction module is used to extract courseware content;

[0100] The knowledge point extraction module is used to extract knowledge points from the courseware content, generate a first embedding vector of the knowledge points, and store the first embedding vector in a vector database.

[0101] The recall module is used to receive a question-generating task, obtain the knowledge points to be tested in the question-generating task, generate a second embedding vector of the knowledge points to be tested, query a first embedding vector similar to the second embedding vector from the vector database, and use the queried first embedding vector as the recall text.

[0102] The question generation module is used to generate a first prompt word based on the knowledge point to be tested and the recall text, input the first prompt word into the question generation model, and obtain the question output by the question generation model. The first prompt word is used to prompt the question generation model to generate the question based on the content of the first prompt word.

[0103] Thirdly, embodiments of the present invention provide an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements the steps of the intelligent question generation method described in the first aspect above.

[0104] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the intelligent question generation method described in the first aspect above.

[0105] Fifthly, embodiments of the present invention provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the intelligent question generation method as described in the first aspect above.

[0106] In this embodiment of the invention, the courseware content and knowledge points in the courseware content can be automatically extracted, and the knowledge points are converted into vectors and stored in a vector database. When generating questions, a recall algorithm is used to recall knowledge points similar to the knowledge points to be tested from the vector database. Based on the knowledge points to be tested and the recalled knowledge points, prompt words are generated. The powerful text generation capabilities of the large model are used to automatically, efficiently and accurately generate test questions containing the knowledge points to be tested, and can accurately test students' mastery of specific knowledge points. Attached Figure Description

[0107] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0108] Figure 1 This is a flowchart illustrating the intelligent agent question generation method according to an embodiment of the present invention;

[0109] Figure 2 This is a schematic diagram of the overall framework of the intelligent question generation method according to an embodiment of the present invention;

[0110] Figure 3 This is a schematic diagram of the method for extracting courseware content according to an embodiment of the present invention;

[0111] Figure 4 This is a schematic diagram illustrating the use of a multi-model large language model to extract courseware content according to an embodiment of the present invention;

[0112] Figure 5 This is a schematic diagram of the knowledge point extraction method according to an embodiment of the present invention;

[0113] Figure 6 This is a schematic diagram illustrating the method for automatically generating questions using a large model, as described in an embodiment of the present invention.

[0114] Figure 7 This is a schematic diagram of a method for recalling text according to an embodiment of the present invention;

[0115] Figure 8 This is a schematic diagram of the test item assessment method according to an embodiment of the present invention;

[0116] Figure 9 This is a structural block diagram of the intelligent question generation system according to an embodiment of the present invention;

[0117] Figure 10 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0118] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0119] Please refer to Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating the intelligent agent question-generating method according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall framework of the intelligent question generation method according to an embodiment of the present invention. The intelligent question generation method includes:

[0120] Step S1: Extract the courseware content;

[0121] In this embodiment of the invention, the courseware can be from various fields, such as courseware for displaying industrial production and manufacturing.

[0122] Step S2: Extract the knowledge points from the courseware content, generate the first embedding vector of the knowledge points, and store the first embedding vector in the vector database;

[0123] Step S3: Receive the question-generating task, obtain the knowledge points to be tested in the question-generating task, generate the second embedding vector of the knowledge points to be tested, query the vector database for a first embedding vector similar to the second embedding vector, and use the queried first embedding vector as the recall text;

[0124] In this embodiment of the invention, knowledge points are stored in a vector format, which allows for quick retrieval of the corresponding knowledge points from the vector database during text retrieval.

[0125] Step S4: Generate a first prompt word based on the knowledge point to be examined and the recall text, input the first prompt word into the question generation model, and obtain the test questions output by the question generation model. The first prompt word is used to prompt the question generation model to generate test questions based on the content in the first prompt word.

[0126] In this embodiment of the invention, the courseware content and knowledge points in the courseware content can be automatically extracted, and the knowledge points are converted into vectors and stored in a vector database. When generating questions, a recall algorithm is used to recall knowledge points similar to the knowledge points to be tested from the vector database. Based on the knowledge points to be tested and the recalled knowledge points, prompt words are generated. The powerful text generation capabilities of the large model are used to automatically, efficiently and accurately generate test questions containing the knowledge points to be tested, and can accurately test students' mastery of specific knowledge points (i.e., the knowledge points to be tested).

[0127] The steps described above are explained in detail below.

[0128] I. Extraction of Courseware Content

[0129] Please refer to Figure 3 In this embodiment of the invention, optionally, the extraction of courseware content includes:

[0130] Step S11: Identify the courseware type;

[0131] In this embodiment of the invention, optionally, the courseware type includes at least one of the following: PDF courseware, PPT courseware, Word courseware, image courseware, video courseware, audio courseware, etc.

[0132] In this embodiment of the invention, optionally, the courseware type can be automatically identified by the file extension and / or file header information.

[0133] Step S12: Determine the extraction tool corresponding to the courseware type, and use the extraction tool to extract the courseware content. The extraction tool includes a multimodal large model and / or a parsing toolkit. Determining the extraction tool corresponding to the courseware type includes: determining the extraction tool corresponding to the courseware type according to a pre-set correspondence between the courseware type and the extraction tool, or automatically configuring the extraction tool for the courseware type according to the identified courseware type.

[0134] Optionally, determine the extraction tool corresponding to the courseware type, and use the extraction tool to extract the courseware content, including at least one of the following:

[0135] When the courseware type is PDF, a PDF parsing library (such as PyPDF2, pdfminer, or PyMuPDF, which are parsing toolkits) is used to read the PDF courseware and extract the text content (including titles, paragraphs, lists, etc.) from the PDF courseware, while preserving the structural information of the text during extraction.

[0136] When the courseware type is PPT courseware, use python-pptx (which is a parsing toolkit) to extract information from the PPT courseware.

[0137] When the courseware type is Word courseware, use python-docx (which is a parsing toolkit) to extract the document content.

[0138] When the courseware is an image-based PDF, a parsing toolkit such as pdf2image is used to save the PDF page by page as images (e.g., PNGs). Then, prompts are generated based on these images. These prompts, containing the images, are input into a Multimodal Large Model (MLLM) system to recognize the image content. The prompts can be generated based on a content extraction prompt template, which can include information to prompt the MLLM system about all content within the image. Please refer to [reference needed]. Figure 4 , Figure 4 This is a schematic diagram illustrating the extraction of courseware content using a multi-model large language model according to an embodiment of the present invention.

[0139] When the courseware type is image-based, prompts are generated based on the image courseware, and these prompts containing the image courseware are input into the Multimodal Large Model (MLLM) for image content recognition. The prompts can be generated based on a content extraction prompt template, which may include information to prompt the MLLM to recognize all content within the image.

[0140] For complex images containing tables, flowcharts, diagrams, etc., if the multimodal large model recognizes that the image contains tables, flowcharts, or diagrams, a dedicated OCR (Optical Character Recognition) tool (part of the parsing toolkit) can be used to extract the images containing tables, flowcharts, or diagrams.

[0141] When the courseware type is video or audio courseware, prompt words are generated based on the video or audio courseware, and the prompt words containing the video or audio courseware are input into the multimodal large model. The multimodal large model is then used to convert the video or audio information into text.

[0142] In this embodiment of the invention, optionally, the base of the multimodal large model can be an open-source multimodal large model such as Qwen2-VL-7B-Instruct, Qwen2-VL-72B-Instruct, InternVL2-26B or MiniCPM-V-2.6.

[0143] In this embodiment of the invention, optionally, a unified courseware content format can be set for the parsing toolkit and the multimodal large model output, such as markdown format.

[0144] Step S13: Perform data preprocessing on the extracted courseware content, wherein the data preprocessing includes at least one of the following: removing information unrelated to the courseware content (i.e., removing noise, such as headers, footers, advertisements, etc.), deleting duplicate information (i.e. deduplication), and standardizing the format (such as unifying date format, units, etc.).

[0145] In this embodiment of the invention, optionally, the extracted courseware content is text content.

[0146] II. Knowledge Point Extraction

[0147] Please refer to Figure 5 The extraction of knowledge points from the courseware content includes:

[0148] Step S21: Generate a second prompt word based on the reference knowledge points and the courseware content, wherein the second prompt word is used to prompt the extraction of knowledge points from the courseware content based on the content of the second prompt word;

[0149] Step S22: Input the second prompt word into the knowledge point extraction model to obtain the knowledge points output by the knowledge point extraction model.

[0150] In this embodiment of the invention, optionally, the second prompt word can be generated based on the reference knowledge points, the courseware content, and the knowledge point extraction prompt template.

[0151] In this embodiment of the invention, the reference knowledge points can be pre-set knowledge points. Taking the courseware as an example that displays information in the field of industrial production and manufacturing, the reference knowledge points can include at least one of the following: industry terminology, equipment names, malfunctions, causes of malfunctions, countermeasures, malfunction detection methods, malfunction resolution measures, operating procedures, equipment parts, and abnormal handling.

[0152] Taking industry terminology as an example, a terminology database can be pre-built, containing industry-related professional terms and their definitions. The knowledge point extraction model can extract keywords from the courseware content and calculate the similarity score between the extracted keywords and each term. Based on the similarity score, it is determined whether the courseware content corresponding to the keyword is a knowledge point. In this embodiment of the invention, optionally, the knowledge point extraction model can use algorithms such as TF-IDF, TextRank, or RAKE to extract keywords from the courseware content.

[0153] It should be noted that the knowledge point extraction model in this invention and the question generation model mentioned above can be the same model or different models.

[0154] In this embodiment of the invention, optionally, the intelligent question generation method further includes:

[0155] Step S23: Classify the knowledge points based on a pre-trained text classifier to obtain the type of the knowledge points;

[0156] Step S24: Store the type of the knowledge point in the vector database.

[0157] Categorizing knowledge points can help the question-generating model generate questions of specific knowledge point types when creating questions.

[0158] That is, querying the vector database for a first embedding vector similar to the second embedding vector includes:

[0159] Based on the type of the knowledge point to be examined, a first embedding vector similar to the second embedding vector is recalled from the vector database, and the type of the recalled first embedding vector is the same as the type of the knowledge point to be examined.

[0160] Taking the display industry's manufacturing sector as an example, the knowledge points can be categorized as follows:

[0161] Undesirable phenomena (e.g., defects, contamination, deformation, etc.);

[0162] Causes of defects (e.g., equipment failure, operational errors, material problems, etc.);

[0163] Defect detection methods (e.g., visual inspection, instrument testing, data analysis, etc.);

[0164] Poor handling measures (e.g., repair, replacement, process adjustment, etc.).

[0165] Furthermore, multi-level classification can be used. For example, knowledge points can be divided into low-difficulty, medium-difficulty, and high-difficulty categories based on their difficulty. Then, each level of difficulty can be further categorized into adverse phenomena, causes of adverse phenomena, detection methods for adverse phenomena, and solutions for adverse phenomena. Alternatively, knowledge points can be divided into basic, applied, and comprehensive categories, and then these categories can be further subdivided into adverse phenomena, causes of adverse phenomena, detection methods for adverse phenomena, and solutions for adverse phenomena.

[0166] In this embodiment of the invention, optionally, the text classifier can be a FastText text classifier, etc.

[0167] In this embodiment of the invention, optionally, generating the first embedding vector of the knowledge point includes:

[0168] Step S25: Input the knowledge point into the text embedding model to obtain the first embedding vector of the knowledge point output by the text embedding model.

[0169] Text embedding models are models that transform text data into fixed-length vector representations. By mapping text to a continuous vector space, embedding models can capture semantic relationships between words, phrases, and sentences, providing a foundation for tasks such as text classification, search, and recommendation.

[0170] In this embodiment of the invention, optionally, the text embedding model is a large language model, such as gte-Qwen2-7B-instruct. The input of the large language model is the text description of the knowledge point, and the output is the embedding vector corresponding to the knowledge point.

[0171] In this embodiment of the invention, optionally, after generating the first embedding vector of the knowledge point, the intelligent question generation method further includes:

[0172] Step S26: Obtain multiple pre-set reference knowledge points;

[0173] Step S27: For each of the reference knowledge points, calculate the similarity score between the first embedding vector and the embedding vector of the reference knowledge point, and sort the first embedding vector based on the similarity score of the first embedding vector to obtain the sorting information of the first embedding vector;

[0174] In this embodiment of the invention, optionally, cosine similarity can be used to calculate the similarity score between the first embedding vector and the embedding vector of the reference knowledge point.

[0175] In this embodiment of the invention, optionally, the courseware is courseware displaying the field of industrial production and manufacturing. The reference knowledge points may include at least one of the following: industry terminology, equipment name, malfunctions, causes of malfunctions, countermeasures, malfunction detection methods, malfunction solutions, operating procedures, equipment parts, abnormal handling, and other knowledge points.

[0176] In this embodiment of the invention, the first embedding vectors of each knowledge point can be sorted according to the similarity scores of the first embedding vectors of each knowledge point from high to low or from low to high.

[0177] Step S28: Store the sorting information of the first embedded vector into the vector database.

[0178] In this embodiment of the invention, optionally, the question-generating model can generate questions based on the sorting information of the first embedding vector of the knowledge points.

[0179] That is, the step of generating a first prompt word based on the knowledge point to be examined and the recall text, inputting the first prompt word into the question generation model, and obtaining the test questions output by the question generation model includes: the question generation model generating questions based on the ranking information of the first embedding vector corresponding to the recall text.

[0180] For example, if the sorting information is sorted in descending order of similarity score, the top N first embedding vectors are selected from the multiple first embedding vectors recalled for the knowledge point to be examined, where N is an integer greater than or equal to 1.

[0181] III. Automatic Question Generation

[0182] Please refer to Figure 6 , Figure 6 This is a schematic diagram illustrating the method for automatically generating questions using a large model, as described in an embodiment of the present invention. Figure 6 As can be seen, the first prompt word input into the LLM (Large Question Model) includes, in addition to the knowledge point to be tested and the recall text (context 1 and context 2), at least one of the following: the type of knowledge point to be tested, the difficulty of the question, the question type, the proportion of question types, the number of questions, the output format of the test questions, and the level of the test questions.

[0183] The types of knowledge points to be examined may include at least one of the following: industry terminology, equipment name, malfunction, cause of malfunction, countermeasures, malfunction detection method, malfunction resolution measures, operating procedures, equipment parts, and abnormal handling.

[0184] The difficulty level of the questions can include, for example, low difficulty, medium difficulty, and high difficulty.

[0185] The question types may include at least one of the following: multiple choice, multiple selection, true / false, short answer, complex question-and-answer, fill-in-the-blank, etc. Among them, complex question-and-answer refers to questions and answers with a certain degree of complexity, that is, the answer requires a deep understanding of the article's theme, needs to include processes such as reasoning and induction, and needs to be obtained by reading multiple paragraphs.

[0186] The test questions can be output in JSON format, for example, and the output content includes: questions, options, answers, and explanations.

[0187] The test question levels include at least one of the following: memory-level test questions, comprehension-level test questions, application-level test questions, and innovation-level test questions.

[0188] In this embodiment of the invention, optionally, the first prompt word can be generated based on a large model prompt word template.

[0189] In this embodiment of the invention, optionally, the first prompt word is input into the question-generating model, and the question-generating model automatically generates and outputs test questions related to the recall text based on the output first prompt word.

[0190] In this embodiment of the invention, optionally, the first prompt word generated based on the knowledge point to be examined and the recall text includes at least one of the following:

[0191] First instruction information: The instruction combines randomness and preset rules to generate various types of questions;

[0192] The second instruction is to adjust the difficulty level of the test questions according to the learning progress of the candidates, and to randomly generate test questions within the corresponding difficulty level.

[0193] The third instruction is to dynamically adjust the proportion of question types generated based on the learning history and preferences of the candidates to be assessed.

[0194] Fourth instruction information: The instruction is to generate test questions at the test question level that meet the task requirements indicated in the test question task or the cognitive entropy information of the test candidate.

[0195] Among them, the memory-level test questions are those with unique answers and require no reasoning;

[0196] The comprehension-level questions are those with a single answer and require reasoning.

[0197] The application-level test questions are those that require multi-step calculations or modeling;

[0198] The innovative-level test questions are those that require the formulation of new questions, the design of experiments, and / or the demonstration of generalization.

[0199] Cognitive entropy reflects the probability of occurrence of various knowledge units within a cognitive system. It can be determined through factors such as the learning progress of the individuals being assessed.

[0200] In this embodiment of the invention, the test questions may optionally include at least one of the following: questions, options, answers, and explanations.

[0201] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the text recall method according to an embodiment of the present invention. First, the knowledge point to be examined needs to be input into the encoder to generate the second embedding vector of the knowledge point to be examined (i.e., Figure 7 The embedding vector in the second embedding vector is used to retrieve the first embedding vector from the vector database. The retrieved first embedding vector is then used as the recall text. Figure 7 (Recall top k texts in the middle).

[0202] In this embodiment of the invention, a text embedding model can be used as an encoder to encode knowledge points and generate a second embedding vector for the knowledge point to be examined. The text embedding model can be the same as the text embedding model used to generate the first embedding vector in the above embodiments. For example, the text embedding model can be gte-Qwen2-7B-instruct.

[0203] In this embodiment of the invention, optionally, the cosine similarity between the second embedded vector and each first embedded vector in the vector database can be calculated to obtain the similarity score between the second embedded vector and each first embedded vector in the vector database. The similarity scores of all first embedded vectors are ranked, and the top K first embedded vectors with the highest similarity scores are obtained as the recall text, where K is an integer greater than or equal to 1.

[0204] The first prompt word in the embodiments of the present invention will be described below with reference to specific examples.

[0205] Example 1:

[0206] Taking multiple-choice questions as an example, the first prompt word in this embodiment of the invention is as follows:

[0207] Generate multiple-choice questions based on the following information:

[0208] Knowledge points to be examined: scratches, cracks, glass shards.

[0209] The type of knowledge point to be examined is: special adverse education.

[0210] Recall text: 1. Scratches: Scratches, including longer scratches or minor single-point scratches, less severe than Cr Crack; 2. Cracks: Cracks in the glass, misalignment of lines or patterns, and in severe cases, light transmission through the metal parts…

[0211] Difficulty level: Medium.

[0212] Question type: Multiple choice.

[0213] Number of questions: 2.

[0214] Output in JSON format:

[0215] {

[0216] "question(question)":"Question X",

[0217] "options":["A. Option A","B. Option B","C. Option C","D. Option D"],

[0218] "answer":"A",

[0219] "explain": "Explanation content"

[0220] }

[0221] Example 2:

[0222] Taking a true / false question as an example, the first prompt word in this embodiment of the invention is as follows:

[0223] Generate a true / false question based on the following information:

[0224] Knowledge point to be examined: defect detection.

[0225] The type of knowledge point to be examined is: special adverse education.

[0226] Recall text: 23.8 Panel edge gate has a large area missing, Code2 reserved...

[0227] Difficulty level: High.

[0228] Question type: True or False.

[0229] Number of questions: 1.

[0230] Output in JSON format:

[0231] {

[0232] "question(question)":"Question X",

[0233] "options":["A.XX","B.XX"],

[0234] "answer(answer)":"true / false",

[0235] "explain(explanation)":"Explanation content"}

[0236] Example 3:

[0237] Taking a fill-in-the-blank question as an example, the first prompt word in this embodiment of the invention is as follows:

[0238] Generate fill-in-the-blank questions based on the following information:

[0239] Knowledge points to be examined: TN products.

[0240] Knowledge points to be examined: Key training for new maintenance personnel.

[0241] Recall Text: Normal Dark Mode (ADS / HADS): The liquid crystal molecules are deflected through the electric field between the first and second layers of ITO...

[0242] Difficulty level: High.

[0243] Question type: True or False.

[0244] Number of questions: 1.

[0245] Output in JSON format:

[0246] {

[0247] "question":"(with blank space)",

[0248] "answer":"",

[0249] "explain(explanation)":"Explanation content"}

[0250] Using the first prompt word in the above embodiments, only one type of question is generated each time. In other embodiments of the present invention, multiple types of questions can also be generated each time using the first prompt word.

[0251] Example 4:

[0252] The first prompt word in this embodiment of the invention is as follows:

[0253] ## Generate test questions based on the following information:

[0254] Content: {The entire / multiple courseware}

[0255] Difficulty level: The questions are reasonably distributed from basic to advanced.

[0256] ##Single-choice question design ({singleNum} questions):

[0257] ##Single Choice Question Output Format Guidelines:

[0258] json

[0259] [{

[0260] "question(question)":"Question X",

[0261] "options":["A. Option A","B. Option B","C. Option C","D. Option D"],

[0262] "answer":"A",

[0263] "explain": "Explanation content"}]

[0264] ```

[0265] ## Multiple Choice Question Design ({mutilNum} questions):

[0266] ## Multiple Choice Question Output Format Guidelines:

[0267] json

[0268] [{

[0269] "question(question)":"Question X",

[0270] "options":["A. Option A","B. Option B","C. Option C","D. Option D"],

[0271] "answer":"C,D",

[0272] "explain": "Explanation content"}]

[0273] ```

[0274] ## True / False Question Design ({judgeNum} questions):

[0275] ## Output format guidelines for true / false questions:

[0276] json

[0277] [{

[0278] "question(question)":"Question X",

[0279] "answer":"Correct",

[0280] "explain": "Explanation content"}]

[0281] ```

[0282] ## Fill-in-the-blank question design ({blankNum} questions):

[0283] ## Output format guidelines for true / false questions:

[0284] json

[0285] [{

[0286] "question":"(with blank space)",

[0287] "answer":"",

[0288] "explain": "Explanation content"}]

[0289] ```

[0290] In this embodiment of the invention, during the construction of the first prompt word:

[0291] It can combine randomness and preset rules to ensure the diversity of question types while avoiding repetition and monotony.

[0292] The difficulty level of the test questions (e.g., easy, medium, hard) is adjusted according to the learning progress of the candidates, and test questions are randomly generated within the corresponding difficulty level.

[0293] The proportion of question types generated is dynamically adjusted based on the learning history and preferences of the candidates to be assessed, in order to provide a personalized question-generating experience.

[0294] In this embodiment of the invention, the large-scale question-generating model can be a large language model of a relatively large size, such as Qwen2-72b-chat.

[0295] The following example illustrates the test questions output by the large-scale question-generating model.

[0296] Based on the entire "Spare Parts" B10 Cell PI Cleaner Spare Parts Replacement History - Ver 1.0 courseware, an example of a single-choice option is automatically generated:

[0297] Based on the provided courseware, the following are the generated multiple-choice questions and their correct answers:

[0298] 1. The main reason for replacing the Guide Roller in the PI Cleaner device is:

[0299] A. In order to improve the cleaning efficiency of the equipment

[0300] B. Wear and tear due to prolonged contact with the glass.

[0301] C. In order to replace the new PK solution

[0302] D. To correct the position of the Align Pin

[0303] Correct answer: B

[0304] 2. When replacing the PI Cleaner EUV Bearing and gears, the following protective measures should be taken:

[0305] A. Wear rubber gloves and chemical protective goggles.

[0306] B. Wear a safety helmet and operate the equipment only after power is off.

[0307] C. Wear a safety helmet and rubber gloves.

[0308] D. No special protective measures are required.

[0309] Correct answer: B

[0310] Based on the "[Alarm] 2017Y PI Inkjet Issue Summary Report - Ver 1.0" courseware, here's an example of the automatic question generation effect:

[0311] 1. One of the main problems encountered by Inkjet equipment during production is low Wiping pressure. Which of the following is NOT a reason for this?

[0312] A. The pipe below the Wiper is blocked.

[0313] BMWiping air circuit valve loose

[0314] C.BPC provides insufficient negative pressure.

[0315] D. The gap between the Inkjet Head and Wiping is too large.

[0316] Correct answer: D

[0317] 2. When addressing the issue of low Wiping pressure, which of the following is NOT a possible solution?

[0318] A. Pipeline removal and NMP soaking cleaning

[0319] B. Manual cleaning of BPC

[0320] C. Adjust the Wiping Speed ​​to 300mm / s

[0321] D.BPC Unit CL Repetition Adjustment

[0322] Correct answer: C

[0323] In this embodiment of the invention, optionally, the generated test questions can be post-processed after generation.

[0324] In some embodiments, optionally, after obtaining the test questions output by the large-scale question-generating model, the method further includes:

[0325] Check for any unqualified test questions and remove them.

[0326] The unqualified test questions include at least one of the following:

[0327] Questions with repeated options;

[0328] Questions with different questions but the same answer choices;

[0329] Questions with meaningless questions or options; for example, questions whose questions or options are irrelevant to the exam content.

[0330] Questions that do not meet the requirements for setting questions;

[0331] Questions with a similarity greater than a threshold;

[0332] Among these methods, regular expression matching algorithms, semantic similarity detection algorithms, or large question-generating models are used to check for the existence of unqualified test questions.

[0333] In this embodiment of the invention, optionally, a regular expression matching algorithm, a semantic similarity detection algorithm, or a large question model can be used to check whether there are any unqualified questions.

[0334] Among them, the detection methods for test questions with similarity greater than the threshold can be: using a semantic similarity detection algorithm to calculate the similarity of each test question and remove it, or directly constructing prompt words and using a large model to detect the similarity of test questions.

[0335] IV. Test Item Evaluation

[0336] Please refer to Figure 8 In this embodiment of the invention, optionally, the intelligent question generation method further includes:

[0337] A third prompt word is generated based on the test questions and assessment dimensions. The third prompt word is used to prompt the generated test questions to be assessed.

[0338] The third prompt word is input into the evaluation model to obtain the score output by the evaluation model;

[0339] Obtain scores for multiple evaluation dimensions, and obtain a total score based on the scores for the multiple evaluation dimensions;

[0340] The evaluation dimensions include at least one of the following: content accuracy, diversity, difficulty level, and reference answer.

[0341] Among them, content accuracy means ensuring that the questions in the test paper are consistent with the content of the courseware, and that the knowledge points tested are accurate.

[0342] Diversity: This means that the test paper should cover different types of questions, such as multiple choice, fill-in-the-blank, and short answer questions, in order to comprehensively assess students' comprehension abilities.

[0343] Difficulty level: This refers to the appropriate level of difficulty in the questions, including both basic and challenging questions, to accommodate students of different levels.

[0344] Logicality: This means that the structure of the test paper should be reasonable, and there should be a certain logical relationship between the questions to help students gradually demonstrate their knowledge.

[0345] Suggested answers: This refers to providing clear suggested answers for each question to ensure consistency and fairness in scoring.

[0346] In this embodiment of the invention, the generated test questions are evaluated in multiple dimensions. The score for each dimension is given by the evaluation model. Since the results generated by the evaluation model are variable, the evaluation score for each dimension can be obtained by averaging multiple evaluations, and the total score is obtained based on the scores of multiple dimensions.

[0347] In this embodiment of the invention, each dimension is evaluated using its own third prompt word.

[0348] In this embodiment of the invention, optionally, the third prompt word is generated using a dimension evaluation prompt word template.

[0349] It should be noted that in this embodiment of the invention, the evaluation model can be the same as the question-generating model, or it can be a different model.

[0350] The following example illustrates the test evaluation method of this invention.

[0351] The process of converting the following assessment dimensions into a total score can be carried out through the following steps:

[0352] (1) Dimensional Scoring: Set a scoring standard for each evaluation dimension, such as using a scoring system of 1 to 5 points. For example:

[0353] Content accuracy: 1-5 points;

[0354] Diversity: 1-5 points;

[0355] Difficulty level: 1-5 points;

[0356] Logical reasoning: 1-5 points;

[0357] Suggested answer: 1-5 points.

[0358] (2) Weighting: Assign weights to each evaluation dimension based on its importance. Some evaluation dimensions may be more important than others. For example:

[0359] Content accuracy: 30%;

[0360] Diversity: 20%;

[0361] Difficulty level: 20%;

[0362] Logicality: 15%;

[0363] Suggested answer: 15%.

[0364] (3) Scoring Calculation: Each evaluation dimension is scored, and then the total score is calculated based on the weights. For example, suppose the evaluation results are as follows:

[0365] Content accuracy: 4 points;

[0366] Diversity: 3 points;

[0367] Difficulty level: 4 points;

[0368] Logical reasoning: 5 points;

[0369] Suggested answer: 4 points.

[0370] Calculate the total score: Total score = (4×0.3) + (3×0.2) + (4×0.2) + (5×0.15) + (4×0.05) = 3.55.

[0371] (4) Scoring criteria: Set the evaluation criteria for the total score, for example:

[0372] 5.0-4.5: Excellent;

[0373] 4.4-3.5: Good;

[0374] 3.4-2.5: Pass;

[0375] 2.4-0: Fail.

[0376] In this way, the scores of each assessment dimension can be combined into a total score, which facilitates a comprehensive evaluation of the quality of the test paper.

[0377] In this embodiment of the invention, optionally, test questions can be filtered based on their total score.

[0378] V. Model Training

[0379] In this embodiment of the invention, optionally, the question-generating model includes at least one of the following modules:

[0380] The memory layer module is used to generate memory-level test questions, which are test questions with unique answers and require no reasoning.

[0381] The comprehension layer module is used to generate comprehension-level test questions, which are questions with unique answers and require reasoning.

[0382] The application layer module is used to generate application-level test questions, which are test questions that require multi-step calculations or modeling.

[0383] The innovation layer module is used to generate innovation-level test questions, which are test questions that require raising new questions, designing experiments, and / or proving generalizations.

[0384] The question-generating model uses a dynamic routing algorithm to map the first prompt word to the corresponding module based on the task requirements indicated in the first prompt word or the cognitive entropy information of the examinee, and the corresponding module generates the test questions.

[0385] The question-generating model in this invention can be obtained by fine-tuning the basic model (such as Deepseek) using LoRA (low-rank adaptation). That is, the above module is obtained by fine-tuning using LoRA, and can also be called a LoRA module.

[0386] Each of the above modules is equivalent to a LoRA expert pool, and each module corresponds to a cognitive level. Specifically, the memory layer module corresponds to the cognitive level of memory, the understanding layer module corresponds to the cognitive level of understanding, the application layer module corresponds to the cognitive level of application, and the innovation layer module corresponds to the cognitive level of innovation. These cognitive levels can also be referred to as Bloom cognitive levels.

[0387] The question-generating model employs a dynamic routing algorithm. Based on the task requirements indicated in the first prompt word or the cognitive entropy information of the examinee, the first prompt word is mapped to the corresponding module, and the corresponding module generates test questions, thus achieving multiple levels of difficulty within the same model.

[0388] For example, if the task requirement in the first prompt is a memory task, the dynamic routing algorithm will route the first prompt to the memory layer module, which will then generate memory-level test questions.

[0389] The training method for the large-scale question-generating model is explained below.

[0390] In this embodiment of the invention, optionally, the method further includes:

[0391] Obtain the training dataset;

[0392] Based on the training dataset, the initial question-generating model is trained.

[0393] The training dataset includes at least one of the following:

[0394] The first dataset includes multiple memory-level test question samples. The data source of the memory-level test question samples comes from at least one of the following: enterprise safety manual, relevant safety regulations, equipment manual; the memory-level test question samples are, for example, test questions that directly test safety regulations, equipment numbers, or standard operating procedures (SOPs).

[0395] The second dataset includes multiple comprehension-level test question samples. The data sources for the comprehension-level test question samples are at least one of the following: process specifications, fault case database, and process principle courseware; for example, the comprehension-level test question samples are test questions such as principle explanation, fault explanation, or flowchart reading.

[0396] The third dataset includes multiple application-level test question samples. The data sources of the application-level test question samples are at least one of the following: simulation operation logs, accident records, and work instructions; for example, the application-level test question samples are test questions such as parameter settings and troubleshooting scenarios.

[0397] The fourth dataset includes multiple innovative test question samples, the data sources of which are at least one of the following: technical improvement proposals, quality control (QC) team results, and patent literature; for example, the innovative test question samples are test questions such as process improvement.

[0398] The first dataset is used to train the memory layer module of the question-generating model;

[0399] The second dataset is used to train the understanding layer module of the question-generating model;

[0400] The third dataset is used to train the application layer module of the question-generating model;

[0401] The fourth dataset is used to train the innovation layer module of the question-generating model.

[0402] In this embodiment of the invention, the hyperparameters used for training may include rank, α, and LR. rank is the dimension used to control the low-rank matrix factorization during fine-tuning, affecting the capacity of the model update part. In this embodiment, rank can be set to 64. α is a scaling factor used to control the degree of influence of the fine-tuning parameters on the original model parameters. The increment is adjusted by α / rank, and in this embodiment, α can be set to 128. LR is the learning rate, i.e., the learning speed of the optimizer, affecting the weight update step size. In this embodiment, LR can be set to 3e-5.

[0403] In this embodiment of the invention, optionally, obtaining the training dataset includes:

[0404] Obtain prompt word templates for generating memory-level, comprehension-level, application-level, and innovation-level test question samples; that is, design a prompt word template for each cognitive level of memory, comprehension, application, and innovation.

[0405] For each question level, the courseware content is extracted from the corresponding courseware. Based on the extracted courseware content and the prompt word template, prompt words are generated and input into the large language model to obtain the question samples output by the large language model.

[0406] The training dataset is obtained based on the test sample corresponding to each test level.

[0407] The following example illustrates the prompt word template in an embodiment of the present invention.

[0408] The prompt word template in this embodiment of the invention can be as follows:

[0409] System prompt (fixed)

[0410] ```

[0411] [Role] You are an industrial training data synthesizer, and you must generate test questions according to Bloom's cognitive levels.

[0412] [Output] JSON only, no explanation.

[0413] ```

[0414] The clues for memory-based test questions may include the following:

[0415] Based on the following excerpt from regulations or manuals, please generate one "memory-level" test question directly. The question stem contains no reasoning, and the answer is unique.

[0416] Excerpt: {raw_text}

[0417] Output format: {"text":"...","option":[],"answer":"...","level":0}

[0418] ```

[0419] The prompts for comprehension-level questions may include the following:

[0420] Based on the following process descriptions or failure cases, please generate one "comprehension level" question that requires one step of reasoning and has a unique answer.

[0421] Excerpt: {raw_text}

[0422] Output format: {"text":"...","option":[],"answer":"...","reason":"One-step reasoning process","level":1}

[0423] ```

[0424] The prompts for application-level questions may include the following:

[0425] Based on the following accident logs or simulation records, please generate one "application-level" test question that requires multiple calculations or modeling steps.

[0426] Excerpt: {raw_text}

[0427] Output format: {"text":"...","option":[],"answer":"...","steps":["Step 1","Step 2",...],"level":2}

[0428] ```

[0429] The prompts for innovative-level questions may include the following:

[0430] Based on the following technological improvement proposals or patents, please generate one "Innovation-level" open-ended question, requiring you to propose a hypothesis, design an experiment, or propose an improvement plan.

[0431] Excerpt: {raw_text}

[0432] Output format: {"text":"...","evaluation_criteria":"...","level":3}

[0433] ```

[0434] In this embodiment of the invention, optionally, obtaining the training dataset further includes at least one of the following:

[0435] The generated test sample is automatically labeled using a large language model; the labels can be memory-level test questions, comprehension-level test questions, application-level test questions, or innovation-level test questions.

[0436] The generated test question samples are filtered according to rules to determine whether they conform to the rules of the corresponding test question level. A corresponding rule script can be used to filter the generated test question samples. The rule script includes rules corresponding to each test question level. For example, whether the memory-level test question sample has a unique answer and requires no reasoning; whether the comprehension-level test question sample has a unique answer and requires reasoning; whether the application-level test question sample requires multi-step calculation or modeling; and whether the innovation-level test question sample requires proposing new questions, designing experiments, and / or proving generalizations.

[0437] The generated test question samples are randomly sampled; a preset proportion (e.g., 5%) of test question samples can be extracted from the training dataset by manual sampling to determine whether the sampled test question samples meet the requirements.

[0438] The generated test sample is reclassified using a classifier. The classification result is compared with the label of the test sample, and test sample with consistency less than a preset threshold (e.g., 90%) is discarded. This step is also called cross-validation, which can ensure that the four types of test sample are mutually exclusive. The classifier can be, for example, DeBERTa-v3.

[0439] The generated test question samples are deduplicated; for example, MinHash or 64-bit SimHash can be used to deduplicat the generated test question samples, and the phase velocity threshold can be set to 0.85.

[0440] To ensure that the ratio of memory-level test questions, comprehension-level test questions, application-level test questions, and innovation-level test questions in the training dataset is 1:1:1:1, this prevents expert collapse.

[0441] VI. Feedback Optimization

[0442] In this embodiment of the invention, a feedback optimization model based on reinforcement learning can be used to optimize the large-scale question-generating model. Reinforcement learning involves the interaction between an agent (i.e., the feedback optimization model) and the environment, obtaining rewards through this interaction to guide behavior, with the goal of maximizing the agent's reward. The advantage of reinforcement learning over traditional deep learning models lies in its ability to "learn online," continuously updating itself with newly learned knowledge and making timely adjustments and feedback.

[0443] The feedback optimization model of this invention is divided into an offline part and an online part.

[0444] 1. Online section:

[0445] In this embodiment of the invention, optionally, the intelligent question generation method further includes:

[0446] Obtain the test questions output by the aforementioned question-generating model;

[0447] The feedback optimization model is used to predict the feedback labels of the test questions output by the question-generating model.

[0448] Based on the test questions output by the question-generating model and the feedback tags, the question-generating model is optimized.

[0449] The question-generating model is optimized, that is, the network parameters of the question-generating model are optimized.

[0450] In this embodiment of the invention, the feedback label may include feedback from students or teachers on the test questions, and the feedback may be positive or negative. For example, the feedback value is: 1 (positive feedback), 0 (negative feedback).

[0451] 2. Offline portion:

[0452] In this embodiment of the invention, optionally, the intelligent question generation method further includes:

[0453] Acquire multiple historical sample data, including historical test questions output by the question-generating model and historical feedback tags for the historical test questions;

[0454] The initial feedback optimization model is trained based on the multiple historical sample data to obtain the trained feedback optimization model.

[0455] The loss function used to train the feedback optimization model is:

[0456]

[0457] in, Denotes the target network, (s,a,r,s) ′ U(D) represents the empirical data, and U(D) represents the memory unit for replaying the empirical data.

[0458] The step of training the initial feedback optimization model based on the multiple historical sample data includes:

[0459] Experience data is randomly extracted from the playback memory unit for training, and the error is obtained by taking the partial derivative of the loss function with respect to the parameter θ.

[0460] In this embodiment of the invention, optionally, the historical sample data may also include at least one of the following: student position, professional field, student exam scores, teacher feedback, etc.

[0461] In this embodiment of the invention, the feedback optimization model may optionally be a deep Q-network (DQN).

[0462] In this embodiment of the invention, the terms related to reinforcement learning are defined as follows:

[0463] Agent (g): The feedback optimization model is regarded as an agent, which includes the optimization model, the exploration strategy, and the related data storage.

[0464] Environment (e): In the environment, students / teachers receive test questions and provide corresponding feedback.

[0465] Action(a): In the feedback optimization model, "action" refers to the list of test items obtained by the optimization system, that is, the feature representation of the test items.

[0466] Feedback (r): After receiving the test questions, the student / teacher provides positive or negative feedback. The feedback value is: 1 (positive feedback), 0 (negative feedback).

[0467] State(s): The state is a representation of the student's job position, professional field, student's exam score, and teacher feedback characteristics.

[0468] Taking the feedback optimization model using a deep Q-network as an example, in this embodiment of the invention, optionally, the loss function used when training the feedback optimization model is as follows:

[0469] The current Q-network uses Q(s,a; θ) i The target Q network is represented by ) Let θ represent θ i and Let represent the network parameters in the i-th iteration, and let the loss function be expressed as:

[0470]

[0471] in, Denotes the target network, (s,a,r,s) ′ ) represents empirical data, and U(D) represents the memory unit for replaying empirical data.

[0472] During training, the deep Q-network randomly draws small batches of empirical data from the playback memory unit for training, and obtains the error by taking the partial derivative of the loss function with respect to the parameter θ. Simultaneously, the parameters of the deep Q-network are updated using gradient descent.

[0473] Please refer to Figure 9 This invention also provides an intelligent question-generating system 10, comprising:

[0474] Courseware content extraction module 11 is used to extract courseware content;

[0475] The knowledge point extraction module 12 is used to extract knowledge points from the courseware content, generate a first embedding vector of the knowledge points, and store the first embedding vector in a vector database.

[0476] The recall module 13 is used to receive the question-generating task, obtain the knowledge points to be tested in the question-generating task, generate a second embedding vector of the knowledge points to be tested, query a first embedding vector similar to the second embedding vector from the vector database, and use the queried first embedding vector as the recall text.

[0477] The question generation module 14 is used to generate a first prompt word based on the knowledge point to be tested and the recall text, input the first prompt word into the question generation model, and obtain the question output by the question generation model. The first prompt word is used to prompt the question generation model to generate the question based on the content in the first prompt word.

[0478] In this embodiment of the invention, the courseware content and knowledge points in the courseware content can be automatically extracted, and the knowledge points are converted into vectors and stored in a vector database. When generating questions, a recall algorithm is used to recall knowledge points similar to the knowledge points to be tested from the vector database. Based on the knowledge points to be tested and the recalled knowledge points, prompt words are generated. The powerful text generation capabilities of the large model are used to automatically, efficiently and accurately generate test questions containing the knowledge points to be tested, and can accurately test students' mastery of specific knowledge points.

[0479] Optionally, the courseware content extraction module 11 is used to automatically identify the courseware type through file extension and / or file header information; determine the extraction tool corresponding to the courseware type; and use the extraction tool to extract the courseware content. The extraction tool includes a multimodal large model and / or a parsing toolkit. Determining the extraction tool corresponding to the courseware type includes: determining the extraction tool corresponding to the courseware type based on a pre-set correspondence between the courseware type and the extraction tool, or automatically configuring the extraction tool for the courseware type based on the identified courseware type. Specifically, when the courseware type is an image-based PDF courseware, the parsing toolkit is used to save the PDF courseware page by page as images, prompt words are generated based on the images, and the prompt words containing the images are input into the multimodal large model to recognize the image content.

[0480] When the courseware type is image courseware, prompt words are generated based on the image courseware, and the prompt words containing the image courseware are input into the multimodal large model to recognize the image content; if the multimodal large model recognizes that the image contains a table, flowchart or diagram, the parsing toolkit is used to extract the image containing the table, flowchart or diagram.

[0481] When the courseware type is video or audio courseware, prompt words are generated based on the video or audio courseware, and the prompt words containing the video or audio courseware are input into the multimodal large model. The multimodal large model is then used to convert the video or audio information into text.

[0482] Optionally, the knowledge point extraction module 12 is used to generate a second prompt word based on reference knowledge points and the courseware content, wherein the second prompt word is used to prompt the extraction of knowledge points from the courseware content based on the content in the second prompt word; the second prompt word is input into the knowledge point extraction model to obtain the knowledge points output by the knowledge point extraction model; wherein the courseware is courseware displaying the field of industrial production and manufacturing, and the reference knowledge points include at least one of the following: industry terminology, equipment name, malfunction, cause of malfunction, countermeasures, malfunction detection method, malfunction resolution measures, operating steps, equipment parts, and abnormal handling.

[0483] Optionally, the knowledge point extraction module 12 is used to input the knowledge point into a text embedding model to obtain the first embedding vector of the knowledge point output by the text embedding model, wherein the text embedding model is a large language model.

[0484] Optionally, the intelligent question generation system 10 further includes:

[0485] The classification module is used to classify the knowledge points based on a pre-trained text classifier, obtain the type of the knowledge points, and store the type of the knowledge points in the vector database.

[0486] The recall module 13 is used to recall a first embedding vector similar to the second embedding vector from the vector database according to the type of the knowledge point to be examined, wherein the type of the recalled first embedding vector is the same as the type of the knowledge point to be examined.

[0487] Optionally, the intelligent question generation system 10 further includes:

[0488] A sorting module is used to obtain multiple pre-set reference knowledge points; for each reference knowledge point, it calculates the similarity score between the first embedding vector and the embedding vector of the reference knowledge point, and sorts the first embedding vector based on the similarity score to obtain the sorting information of the first embedding vector; the sorting information of the first embedding vector is stored in the vector database. The question-generating model generates questions based on the sorting information of the first embedding vectors corresponding to the recalled text. If the sorting information is ordered from high to low similarity scores, the top N first embedding vectors are selected from the multiple first embedding vectors recalled for the knowledge point to be examined, where N is an integer greater than or equal to 1, for question generation. Optionally, the courseware is displayed in the field of industrial production and manufacturing, and the reference knowledge points may include at least one of the following: industry terminology, equipment name, malfunctions, causes of malfunctions, countermeasures, malfunction detection methods, malfunction solutions, operating procedures, equipment parts, and abnormal handling.

[0489] Optionally, the first prompt word may further include at least one of the following: the type of knowledge point to be tested, the difficulty level of the question, the question type, the proportion of question types, the number of questions, the output format of the test questions, and the question level; wherein, the question level includes at least one of the following: memory-level questions, comprehension-level questions, application-level questions, and innovation-level questions; the first prompt word generated based on the knowledge point to be tested and the recall text includes at least one of the following:

[0490] First instruction information: The instruction combines randomness and preset rules to generate various types of questions;

[0491] The second instruction is to adjust the difficulty level of the test questions according to the learning progress of the candidates, and to randomly generate test questions within the corresponding difficulty level.

[0492] The third instruction is to dynamically adjust the proportion of question types generated based on the learning history and preferences of the candidates to be assessed.

[0493] Fourth instruction information: The instruction is to generate test questions at the test question level that meet the task requirements indicated in the test question task or the cognitive entropy information of the test candidate.

[0494] Among them, the memory-level test questions are those with unique answers and require no reasoning;

[0495] The comprehension-level questions are those with a single answer and require reasoning.

[0496] The application-level test questions are those that require multi-step calculations or modeling;

[0497] The innovative-level test questions are those that require the formulation of new questions, the design of experiments, and / or the demonstration of generalization.

[0498] Optionally, the intelligent question generation system 10 further includes:

[0499] The post-processing module is used to check for any unqualified test questions and remove the unqualified test questions.

[0500] The unqualified test questions include at least one of the following:

[0501] Questions with repeated options;

[0502] Questions with different questions but the same answer choices;

[0503] Questions that do not meet the requirements for setting questions.

[0504] Optionally, the intelligent question generation system 10 further includes:

[0505] An assessment module is used to generate a third prompt word based on the test questions and assessment dimensions. The third prompt word is used to prompt the assessment of the generated test questions. The third prompt word is input into the assessment model to obtain the score output by the assessment model. Scores for multiple assessment dimensions are obtained, and a total score is obtained based on the scores for the multiple assessment dimensions. The assessment dimensions include at least one of the following: content accuracy, diversity, difficulty level, and reference answer.

[0506] Optionally, the question-generating model includes at least one of the following modules:

[0507] The memory layer module is used to generate memory-level test questions, which are test questions with unique answers and require no reasoning.

[0508] The comprehension layer module is used to generate comprehension-level test questions, which are questions with unique answers and require reasoning.

[0509] The application layer module is used to generate application-level test questions, which are test questions that require multi-step calculations or modeling.

[0510] The innovation layer module is used to generate innovation-level test questions, which are test questions that require raising new questions, designing experiments, and / or proving generalizations.

[0511] The question-generating model uses a dynamic routing algorithm to map the first prompt word to the corresponding module based on the task requirements indicated in the first prompt word or the cognitive entropy information of the examinee, and the corresponding module generates the test questions.

[0512] Optionally, the intelligent question generation system 10 further includes:

[0513] The acquisition module is used to acquire the training dataset;

[0514] The first training module is used to train the initial question-generating model based on the training dataset.

[0515] The training dataset includes at least one of the following:

[0516] The first dataset includes multiple memory-level test question samples, and the data source of the memory-level test question samples comes from at least one of the following: enterprise safety manual, relevant safety regulations, and equipment manual;

[0517] The second dataset includes multiple comprehension-level test question samples, the data sources of which are at least one of the following: process specifications, fault case library, and process principle courseware;

[0518] The third dataset includes multiple application-level test question samples, and the data sources of the application-level test question samples are at least one of the following: simulation operation logs, accident records, and work instructions;

[0519] The fourth dataset includes multiple innovative test question samples, the data sources of which are at least one of the following: technology improvement proposals, quality control team results, and patent documents;

[0520] The first dataset is used to train the memory layer module of the question-generating model;

[0521] The second dataset is used to train the understanding layer module of the question-generating model;

[0522] The third dataset is used to train the application layer module of the question-generating model;

[0523] The fourth dataset is used to train the innovation layer module of the question-generating model.

[0524] Optionally, the acquisition module is used to acquire prompt word templates for generating memory-level test question samples, comprehension-level test question samples, application-level test question samples, and innovation-level test question samples; for each test question level, extract courseware content from the corresponding courseware, generate prompt words based on the extracted courseware content and the prompt word templates, and input the prompt words into the large language model to obtain the test question samples output by the large language model; and obtain the training dataset based on the test question samples corresponding to each test question level.

[0525] Optionally, the acquisition module is further configured to perform at least one of the following:

[0526] The generated test sample is automatically labeled using a large language model;

[0527] The generated test question samples are filtered according to rules to determine whether the test question samples meet the rules of the corresponding test question level;

[0528] Random sampling inspection of the generated test question samples;

[0529] The generated test sample is reclassified using a classifier. The classification result is compared with the label of the test sample, and test sample with consistency less than a preset threshold is discarded.

[0530] The generated test question samples are deduplicated;

[0531] Ensure that the ratio of memory-level test questions, comprehension-level test questions, application-level test questions, and innovation-level test questions in the training dataset is 1:1:1:1.

[0532] Optionally, the intelligent question generation system 10 further includes:

[0533] The feedback optimization module is used to obtain the test questions output by the large-scale question generation model; use the feedback optimization model to predict the feedback labels of the test questions output by the large-scale question generation model, wherein the feedback labels include positive or negative feedback; and optimize the large-scale question generation model based on the test questions output by the large-scale question generation model and the feedback labels.

[0534] Optionally, the intelligent question generation system 10 further includes:

[0535] The second training module is used to acquire multiple historical sample data, including historical test questions output by the question-generating model and historical feedback labels for the historical test questions; and to train the initial feedback optimization model based on the multiple historical sample data to obtain the trained feedback optimization model.

[0536] The loss function used to train the feedback optimization model is:

[0537]

[0538] in, Denotes the target network, (s,a,r,s) ′ U(D) represents the empirical data, and U(D) represents the memory unit for replaying the empirical data.

[0539] The step of training the initial feedback optimization model based on the multiple historical sample data includes:

[0540] Experience data is randomly extracted from the playback memory unit for training, and the error is obtained by taking the partial derivative of the loss function with respect to the parameter θ.

[0541] Please refer to Figure 10 The present invention also provides an electronic device 20, including a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the computer program is executed by the processor 21, it implements the various processes of the above-described intelligent question generation method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0542] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described intelligent question-generating method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0543] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-described... Figure 1 The various processes of the method embodiments shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0544] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0545] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0546] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other modifications under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these modifications are within the protection scope of the present invention.

Claims

1. An intelligent question generation method, characterized in that, include: Extract courseware content; Extract the knowledge points from the courseware content, generate the first embedding vector of the knowledge points, and store the first embedding vector in the vector database; Receive a question-generating task and obtain the knowledge points to be tested in the question-generating task. Generate a second embedding vector of the knowledge points to be tested. Query a first embedding vector similar to the second embedding vector from the vector database. Use the queried first embedding vector as the recall text. A first prompt word is generated based on the knowledge point to be examined and the recall text. The first prompt word is then input into the question generation model to obtain the test questions output by the question generation model. The first prompt word is used to prompt the question generation model to generate test questions based on the content in the first prompt word.

2. The method according to claim 1, characterized in that, The extracted courseware content includes: Automatically identify courseware types based on file extensions and / or file header information; The extraction tool corresponding to the courseware type is determined, and the courseware content is extracted using the extraction tool. The extraction tool includes a multimodal large model and / or a parsing toolkit. The determination of the extraction tool corresponding to the courseware type includes: determining the extraction tool corresponding to the courseware type according to a pre-set correspondence between the courseware type and the extraction tool, or automatically configuring the extraction tool for the courseware type according to the identified courseware type. When the courseware type is an image-based PDF courseware, the parsing toolkit is used to save the PDF courseware page by page as images, prompt words are generated based on the images, and the prompt words containing the images are input into the multimodal large model to recognize the image content; When the courseware type is image courseware, prompt words are generated based on the image courseware, and the prompt words containing the image courseware are input into the multimodal large model to recognize the image content; if the multimodal large model recognizes that the image contains a table, flowchart or diagram, the parsing toolkit is used to extract the image containing the table, flowchart or diagram. When the courseware type is video or audio courseware, prompt words are generated based on the video or audio courseware, and the prompt words containing the video or audio courseware are input into the multimodal large model. The multimodal large model is then used to convert the video or audio information into text.

3. The method according to claim 1, characterized in that, The extraction of knowledge points from the courseware content includes: A second prompt word is generated based on the reference knowledge points and the courseware content. The second prompt word is used to prompt the extraction of knowledge points from the courseware content based on the content of the second prompt word. Input the second prompt word into the knowledge point extraction model to obtain the knowledge points output by the knowledge point extraction model. The courseware is designed for the industrial manufacturing sector, and the reference knowledge points include at least one of the following: industry terminology, equipment name, malfunctions, causes of malfunctions, countermeasures, malfunction detection methods, malfunction resolution measures, operating procedures, equipment parts, and abnormal handling.

4. The method according to claim 1, characterized in that, The first embedding vector for generating the knowledge point includes: The knowledge points are input into a text embedding model to obtain the first embedding vector of the knowledge points output by the text embedding model. The text embedding model is a large language model.

5. The method according to claim 1, characterized in that, Also includes: The knowledge points are classified based on a pre-trained text classifier to obtain the type of the knowledge points; The types of the knowledge points are stored in the vector database; The step of querying the vector database for a first embedding vector similar to the second embedding vector includes: Based on the type of the knowledge point to be examined, a first embedding vector similar to the second embedding vector is recalled from the vector database, and the type of the recalled first embedding vector is the same as the type of the knowledge point to be examined.

6. The method according to claim 1, characterized in that, After generating the first embedding vector of the knowledge point, the process also includes: Obtain multiple pre-set reference knowledge points; For each of the reference knowledge points, the similarity score between the first embedding vector and the embedding vector of the reference knowledge point is calculated, and the first embedding vector is sorted based on the similarity score of the first embedding vector to obtain the sorting information of the first embedding vector. The sorting information of the first embedded vector is stored in the vector database; The step of generating a first prompt word based on the knowledge point to be examined and the recalled text, inputting the first prompt word into the question-generating model, and obtaining the test questions output by the question-generating model includes: The question-generating model generates questions based on the ranking information of the first embedding vectors corresponding to the recalled text. If the ranking information is sorted in order of similarity score from high to low, the top N first embedding vectors are selected from the multiple first embedding vectors recalled for the knowledge point to be examined, and N is an integer greater than or equal to 1. The courseware is designed for the industrial manufacturing sector, and the reference knowledge points include at least one of the following: industry terminology, equipment name, malfunctions, causes of malfunctions, countermeasures, malfunction detection methods, malfunction resolution measures, operating procedures, equipment parts, and abnormal handling.

7. The method according to claim 1, characterized in that, The first prompt word also includes at least one of the following: the type of knowledge point to be tested, the difficulty of the question, the question type, the proportion of question types, the number of questions, the output format of the test questions, and the test question level; wherein, the test question level includes at least one of the following: memory-level test questions, comprehension-level test questions, application-level test questions, and innovation-level test questions; The first prompt word generated based on the knowledge point to be examined and the recall text includes at least one of the following: First instruction information: The instruction combines randomness and preset rules to generate various types of questions; The second instruction is to adjust the difficulty level of the test questions according to the learning progress of the candidates, and to randomly generate test questions within the corresponding difficulty level. The third instruction is to dynamically adjust the proportion of question types generated based on the learning history and preferences of the candidates to be assessed. Fourth instruction information: The instruction is to generate test questions at the test question level that meet the task requirements indicated in the test question task or the cognitive entropy information of the test candidate. Among them, the memory-level test questions are those with unique answers and require no reasoning; The comprehension-level questions are those with a single answer and require reasoning. The application-level test questions are those that require multi-step calculations or modeling; The innovative-level test questions are those that require the formulation of new questions, the design of experiments, and / or the demonstration of generalization.

8. The method according to claim 1, characterized in that, After obtaining the test questions output by the large-scale question-generating model, the process also includes: Check for any unqualified test questions and remove them. The unqualified test questions include at least one of the following: Questions with repeated options; Questions with different questions but the same answer choices; Questions that do not meet the requirements for setting questions; Questions with a similarity greater than a threshold; Among these methods, regular expression matching algorithms, semantic similarity detection algorithms, or large question-generating models are used to check for the existence of unqualified test questions.

9. The method according to any one of claims 1 or 8, characterized in that, Also includes: A third prompt word is generated based on the test questions and assessment dimensions. The third prompt word is used to prompt the generated test questions to be assessed. The third prompt word is input into the evaluation model to obtain the score output by the evaluation model; Obtain scores for multiple evaluation dimensions, and obtain a total score based on the scores for the multiple evaluation dimensions; The evaluation dimensions include at least one of the following: content accuracy, diversity, difficulty level, and reference answer.

10. The method according to claim 1, characterized in that, The question-generating model includes at least one of the following modules: The memory layer module is used to generate memory-level test questions, which are test questions with unique answers and require no reasoning. The comprehension layer module is used to generate comprehension-level test questions, which are questions with unique answers and require reasoning. The application layer module is used to generate application-level test questions, which are test questions that require multi-step calculations or modeling. The innovation layer module is used to generate innovation-level test questions, which are test questions that require raising new questions, designing experiments, and / or proving generalizations. The question-generating model uses a dynamic routing algorithm to map the first prompt word to the corresponding module based on the task requirements indicated in the first prompt word or the cognitive entropy information of the examinee, and the corresponding module generates the test questions.

11. The method according to claim 10, characterized in that, Also includes: Obtain the training dataset; Based on the training dataset, the initial question-generating model is trained. The training dataset includes at least one of the following: The first dataset includes multiple memory-level test question samples, and the data source of the memory-level test question samples comes from at least one of the following: enterprise safety manual, relevant safety regulations, and equipment manual; The second dataset includes multiple comprehension-level test question samples, the data sources of which are at least one of the following: process specifications, fault case library, and process principle courseware; The third dataset includes multiple application-level test question samples, and the data sources of the application-level test question samples are at least one of the following: simulation operation logs, accident records, and work instructions; The fourth dataset includes multiple innovative test question samples, the data sources of which are at least one of the following: technology improvement proposals, quality control team results, and patent documents; The first dataset is used to train the memory layer module of the question-generating model; The second dataset is used to train the understanding layer module of the question-generating model; The third dataset is used to train the application layer module of the question-generating model; The fourth dataset is used to train the innovation layer module of the question-generating model.

12. The method according to claim 11, characterized in that, The acquisition of the training dataset includes: Obtain prompt word templates for generating memory-level test question samples, comprehension-level test question samples, application-level test question samples, and innovation-level test question samples; For each question level, the courseware content is extracted from the corresponding courseware. Based on the extracted courseware content and the prompt word template, prompt words are generated and input into the large language model to obtain the question sample output by the large language model. The training dataset is obtained based on the test sample corresponding to each test level.

13. The method according to claim 12, characterized in that, The acquisition of the training dataset also includes at least one of the following: The generated test sample is automatically labeled using a large language model; The generated test question samples are filtered according to rules to determine whether the test question samples meet the rules of the corresponding test question level; Random sampling inspection of the generated test question samples; The generated test sample is reclassified using a classifier. The classification result is compared with the label of the test sample, and test sample with consistency less than a preset threshold is discarded. The generated test question samples are deduplicated; Ensure that the ratio of memory-level test questions, comprehension-level test questions, application-level test questions, and innovation-level test questions in the training dataset is 1:1:1:

1.

14. The method according to claim 1, characterized in that, Also includes: Obtain the test questions output by the aforementioned question-generating model; The feedback optimization model is used to predict the feedback labels of the test questions output by the question-generating model. The feedback labels include positive or negative feedback; Based on the test questions output by the question-generating model and the feedback tags, the question-generating model is optimized.

15. The method according to claim 14, characterized in that, Also includes: Acquire multiple historical sample data, including historical test questions output by the question-generating model and historical feedback tags for the historical test questions; The initial feedback optimization model is trained based on the multiple historical sample data to obtain the trained feedback optimization model. The loss function used to train the feedback optimization model is: in, Let (s,a,r,s′) represent the target network, (s,a,r,s′) represent the empirical data, and U(D) represent the playback memory unit of the empirical data. The step of training the initial feedback optimization model based on the multiple historical sample data includes: Experience data is randomly extracted from the playback memory unit for training, and the error is obtained by taking the partial derivative of the loss function with respect to the parameter θ.

16. An intelligent question-generating system, characterized in that, include: The courseware content extraction module is used to extract courseware content; The knowledge point extraction module is used to extract knowledge points from the courseware content, generate a first embedding vector of the knowledge points, and store the first embedding vector in a vector database. The recall module is used to receive a question-generating task, obtain the knowledge points to be tested in the question-generating task, generate a second embedding vector of the knowledge points to be tested, query a first embedding vector similar to the second embedding vector from the vector database, and use the queried first embedding vector as the recall text. The question generation module is used to generate a first prompt word based on the knowledge point to be tested and the recall text, input the first prompt word into the question generation model, and obtain the question output by the question generation model. The first prompt word is used to prompt the question generation model to generate the question based on the content of the first prompt word.

17. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the intelligent question generation method as described in any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the intelligent question generation method as described in any one of claims 1 to 15.

19. A computer program product, characterized in that, It includes computer instructions, which, when executed by a processor, implement the steps of the intelligent question generation method as described in any one of claims 1 to 15.

Images