Intelligent proposition method, system and device for simulating expert proposition and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and particularly provides an intelligent proposition method, system and device for simulating expert propositions and a storage medium, and the method comprises the steps: constructing a structured knowledge network fusing knowledge points, cognitive levels and proposition specifications; establishing a material vector database associated with the knowledge point system; according to the target knowledge point, proposition constraint information is extracted from the knowledge network, related materials are retrieved from a vector library, and the related materials are combined into a generative prompt to be input into a large language model to generate a test question first draft; inputting the test question first draft into at least one large language model serving as a simulation answering agent, evaluating the quality of the test questions by analyzing the answering process and result, and outputting the test questions reaching the standard. According to the method, the proposition efficiency and consistency can be remarkably improved, absolute dependence on expert experience is reduced, and automatic output of high-quality test questions is achieved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence, and particularly relates to an intelligent proposition method, system, device and storage medium simulating expert proposition. BACKGROUND

[0002] Traditional test proposition mainly relies on expert experience, and has problems such as low efficiency, high cost, and difficulty in guaranteeing quality consistency. At present, although automatic proposition technology based on large language models has appeared, it usually adopts an end-to-end generation mode, lacks deep combination of education measurement rules and subject knowledge system, and is prone to phenomena such as content "hallucination", logical contradiction, out-of-class or difficulty out of control. At the same time, existing methods mainly focus on generation, lack automatic and simulated evaluation mechanisms for test quality, and it is difficult to ensure the rigor and effectiveness of the test questions. Therefore, there is an urgent need for an automatic proposition method that can simulate expert proposition logic, deeply integrate structured knowledge, and intelligently check the quality of the generated results. SUMMARY

[0003] In view of the above problems of the prior art, the present application provides an intelligent proposition method, system, device and storage medium simulating expert proposition to solve the above technical problems.

[0004] In a first aspect, the present application provides an intelligent proposition method simulating expert proposition, comprising: constructing a structured knowledge network containing a knowledge point system and its attribute association based on a plurality of proposition basis data; collecting proposition materials and performing vector representation to establish a material vector database associated with the knowledge point system; According to the target knowledge point, obtain the proposition constraint information from the structured knowledge network, and retrieve the related materials from the material vector database; combine the proposition constraint information and the related materials into a generation prompt, and input it into a large language model to generate a test question draft; input the test question draft into at least one large language model serving as a simulated answering agent to obtain its answering process and result for the test question; based on the answering process and result, evaluate the proposition quality of the test question draft, and output a qualified test question.

[0005] In an optional embodiment, based on a plurality of proposition basis data, a structured knowledge network containing a knowledge point system and its attribute association is constructed, comprising: obtain a plurality of proposition basis data, wherein the proposition basis data at least includes examination syllabus, electronic textbook and past years' test questions; analyze the examination syllabus, extract knowledge points, cognitive levels and ability requirements, and dynamically calculate the knowledge point weight based on semantic association; Analyzing the electronic textbook to extract core content paragraphs corresponding to knowledge points; Analyzing the past years' true questions to construct structured data representing the association between the test questions, knowledge points, cognitive levels, and difficulty; Based on the extracted knowledge points, weights, core content paragraphs, and structured data, a knowledge graph containing nodes, attributes, and association relationships is constructed and stored, wherein a graph reasoning model is used to infer and complete the logical dependency relationship between knowledge points.

[0006] In an optional implementation, based on the extracted knowledge points, weights, core content paragraphs, and structured data, a knowledge graph containing nodes, attributes, and association relationships is constructed and stored, including: Defining multiple types of nodes, at least including: KnowledgePoint nodes representing knowledge points, TextbookChapter nodes representing textbook chapter content, PastQuestion nodes representing historical test questions, PropositionRule nodes representing proposition specifications, and CognitiveLevel nodes representing cognitive ability levels; Defining relationship types connecting the nodes, at least including: BelongsTo relationships connecting knowledge points and textbook chapters, Examines relationships connecting historical test questions and knowledge points, Requires relationships connecting knowledge points and cognitive ability levels, Follows relationships connecting knowledge points and proposition specifications, and Prerequisite relationships representing the logical dependency between knowledge points; Infer and complete the Prerequisite relationship between knowledge points through a graph reasoning model; The KnowledgePoint node has a dynamic weight attribute, and its weight value is calculated based on multiple factors, including: a frequency factor based on the historical test data statistics, an association factor based on semantic similarity calculation, and a level factor based on the cognitive ability level determination.

[0007] In an optional implementation, the Prerequisite relationship between knowledge points is inferred and completed through a graph reasoning model, including: Based on the existing nodes and relationships in the knowledge graph, extract at least one pair of observation features between knowledge points, and use a probabilistic reasoning model to calculate a first probability value of the existence of prerequisite knowledge relationship between the knowledge point pair; Perform graph representation learning on the knowledge graph, aggregate neighbor information of nodes through message passing to update their vector representations, and based on the updated node vector representations, calculate a second probability value of the existence of prerequisite knowledge relationship between the knowledge point pair through a link prediction model. fuse the first probability value and the second probability value to obtain a comprehensive confidence degree that the prerequisite relationship exists between the knowledge point pair; when the comprehensive confidence degree exceeds a preset threshold and the relationship does not exist in the graph, automatically creating a corresponding Prerequisite relationship.

[0008] In an optional implementation, proposition materials are collected and vectorized to establish a material vector database associated with the knowledge point system, including: collecting original proposition material texts from at least one material source, performing subject recognition and abstract generation on the original proposition material texts to obtain structured material segments; using a pre-trained text embedding model to encode the structured material segments into fixed-dimensional vector representations; based on at least one pre-association strategy, determining a preliminary association relationship between the structured material segments and one or more knowledge points in the knowledge point system, and storing the knowledge point identifiers representing the preliminary association relationship as metadata corresponding to the vector representations.

[0009] In an optional implementation, proposition constraint information is obtained from the structured knowledge network according to a target knowledge point, and relevant materials are retrieved from the material vector database; the proposition constraint information and the relevant materials are combined into a generative prompt, which is input into a large language model to generate a test question draft, including: extracting proposition constraint information from the structured knowledge network according to a target knowledge point, the proposition constraint information including at least the cognitive level, difficulty requirement and applicable question type rule of the target knowledge point; based on the target knowledge point, retrieving at least one semantically related material segment from the material vector database; formatting and combining the proposition constraint information, the at least one material segment and a preset proposition task instruction to construct a generative prompt text; inputting the generative prompt text into a large language model, adjusting the generation parameters of the large language model to control the output characteristics, and batch generating test question drafts conforming to the proposition constraint information.

[0010] In an optional implementation, the proposition quality of the test question draft is evaluated based on the answer process and result, and a qualified test question is output, including: analyzing the answer process and result generated by the simulation answering agent according to a plurality of preset quality dimensions to generate scores of the test question draft on each quality dimension; based on the scores on each quality dimension, calculating a comprehensive quality score of the test question draft; The comprehensive quality score is compared with a preset adoption threshold to determine whether the test question draft is a standard test question.

[0011] In a second aspect, the present application provides an intelligent test question setting system simulating expert setting, comprising: a graph construction module configured to construct a structured knowledge network comprising a knowledge point system and attribute association based on multiple test question setting basis data; a material processing module configured to collect test question setting materials and perform vectorization representation, and establish a material vector database associated with the knowledge point system; a test question generation module configured to acquire test question setting constraint information from the structured knowledge network according to a target knowledge point, and retrieve relevant materials from the material vector database; combine the test question setting constraint information and the relevant materials into a generative prompt, and input the generative prompt into a large language model to generate a test question draft; a test question evaluation module configured to input the test question draft into at least one large language model serving as a simulated answering agent to acquire an answering process and result of the test question; evaluate the test question setting quality of the test question draft based on the answering process and result, and output a standard test question.

[0012] In a third aspect, an apparatus is provided, comprising: a memory configured to store an intelligent test question setting program simulating expert setting; a processor configured to implement the steps of the intelligent test question setting method simulating expert setting when executing the intelligent test question setting program simulating expert setting.

[0013] In a fourth aspect, a computer readable storage medium is provided, and the storage medium stores an intelligent test question setting program simulating expert setting, which implements the steps of the intelligent test question setting method simulating expert setting when executed by a processor.

[0014] The present application has the beneficial effect that the intelligent test question setting method, system, apparatus and storage medium simulating expert setting provided by the present application realize deep semantic organization and efficient retrieval of test question setting basis and materials by constructing a structured knowledge network and a vectorized material library, which regulates the knowledge range and logical basis of test question setting from the root. By combining test question setting constraints and relevant materials into a generative prompt, the large language model is guided to generate test questions accurately and controllably, effectively suppressing "hallucinations" and improving content relevance. The simulated answering evaluation mechanism based on the large language model is innovatively introduced, which realizes automatic and multi-dimensional quality checking of test question logic, difficulty and effectiveness, forming an intelligent closed-loop control of test question setting quality. Finally, while significantly improving test question setting efficiency and consistency, the absolute dependence on expert experience is reduced, and automatic production of high-quality and large-scale test questions is realized. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0016] Figure 1 is a schematic flow chart of a method of an embodiment of the present application.

[0017] Figure 2 is a schematic block diagram of a system of an embodiment of the present application.

[0018] Figure 3 is a structural schematic diagram of an apparatus provided by an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the technical personnel in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort should belong to the scope of protection of the present application.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0021] The intelligent proposition method for simulating expert proposition provided by the embodiments of the present application is executed by a computer apparatus, and accordingly, the intelligent proposition system for simulating expert proposition runs in the computer apparatus.

[0022] Figure 1 is a schematic flow chart of a method of an embodiment of the present application. In which, Figure 1 The execution subject can be an intelligent proposition system for simulating expert proposition. The order of the steps in the flow chart can be changed according to different needs, and some can be omitted.

[0023] As Figure 1 shown, the method comprises: S1. Based on a plurality of proposition basis data, a structured knowledge network containing a knowledge point system and its attribute association is constructed; S2. Proposition material is collected and vectorized, and a material vector database associated with the knowledge point system is established; S3. Obtain proposition constraint information from the structured knowledge network according to the target knowledge point, and retrieve relevant materials from the material vector database; combine the proposition constraint information and the relevant materials into a generative prompt, and input the generative prompt into a large language model to generate a preliminary test question; S4. Input the preliminary test question into at least one large language model serving as a simulated answering agent to obtain the answering process and result of the test question; based on the answering process and result, evaluate the proposition quality of the preliminary test question, and output a qualified test question.

[0024] In an embodiment of the present application, based on step S1, a possible embodiment will be given below to non-restrictively illustrate the specific implementation thereof.

[0025] S101, Obtain a plurality of proposition basis data.

[0026] The system collects structured and unstructured raw data from specified education resource platforms or local storage through a file upload interface or a web crawler. The proposition basis data at least includes: examination syllabus stored in the form of text or table, electronic teaching materials stored in PDF or Word format, and a database of past years' true questions containing questions and answers. These data constitute the basic raw materials for building a knowledge system.

[0027] S102, Analyze the examination syllabus and calculate the weight.

[0028] The system uses a pre-trained language model based on the Transformer architecture (such as BERT) to perform deep semantic analysis on the examination syllabus text. First, through named entity recognition and dependency syntax analysis, discrete knowledge point entities and their corresponding cognitive levels (such as memory, understanding, application, analysis) and ability requirement descriptions are extracted. Subsequently, the system calculates the semantic correlation between knowledge points (such as using word vector cosine similarity), and dynamically assigns an initial weight to each knowledge point in combination with expert prior rules. This weight can be updated in subsequent steps according to the true question analysis results.

[0029] S103, Analyze and extract the content of the electronic teaching materials.

[0030] For electronic teaching material documents, the system first performs format analysis and text cleaning to obtain pure text content. Subsequently, a graph model algorithm such as TextRank or an unsupervised summary model is applied to analyze the text within each chapter or knowledge unit, identify and extract core content paragraphs expressing core concepts, and discard redundant, example, or transitional text. The extracted core paragraphs are mapped and associated with the knowledge points extracted in S102 through chapter titles, keywords, etc.

[0031] S104. Analyze and structure past years' true questions.

[0032] The system analyzes past years' true questions one by one. Using natural language processing technology, it identifies the stem, options, answers and analysis text of each question. Through matching or classification models, it associates each question to the knowledge points defined in S102, and labels the cognitive level and difficulty value obtained through historical score data statistics. Finally, a structured data entry is constructed for each question, clearly representing the four-ary relationship between "question-knowledge point-cognitive level-difficulty", forming a proposition rule database.

[0033] S105. Based on the extracted knowledge points, weights, core content paragraphs and structured data, construct and store a knowledge graph containing nodes, attributes and relationship types, including: Knowledge graph structure definition and storage implementation: The system uses an attribute graph model to formalize the structured knowledge network. In the graph database (this example uses Neo4j), the following main types of nodes are created and maintained: KnowledgePoint (knowledge point node): Core entity with attributes such as kp_id (unique identifier), name (name), weight (dynamic weight), description (description), etc.

[0034] TextbookChapter (Textbook Chapter Node): Store compressed core textbook content with chapter_id, title, compressed_content attributes.

[0035] PastQuestion (Past Question Node): Represent past years' true questions with question_id, content, difficulty attributes.

[0036] PropositionRule (Proposition Rule Node): Store proposition technology rules with rule_id, rule_content (such as "interference items should have a semblance of truth") attributes.

[0037] CognitiveLevel (Cognitive Level Node): As an independent node, such as creating nodes named "memory" and "application", to facilitate hierarchical reasoning.

[0038] At the same time, the system defines and creates the following core relationship types to connect the above nodes to form a knowledge semantic network: BELONGS_TO: connecting KnowledgePoint node and TextbookChapter node, indicating that the knowledge point comes from a textbook chapter.

[0039] EXAMINES: connecting PastQuestion node and KnowledgePoint node, indicating that a question examines a knowledge point. The relationship can have a frequency (number of examinations) attribute.

[0040] REQUIRES: connecting KnowledgePoint node and CognitiveLevel node, indicating that the cognitive level to be mastered to master the knowledge point.

[0041] FOLLOWS: connecting KnowledgePoint node and PropositionRule node, indicating that the proposition for the knowledge point should follow a certain rule.

[0042] PREREQUISITE: connecting two KnowledgePoint nodes, indicating the prerequisite dependency between knowledge points (e.g., "knowledge point A is the prerequisite knowledge of knowledge point B"). This relationship is the focus of the system to automatically infer and complete.

[0043] Knowledge graph relationship automatic completion implementation: To realize the automatic discovery and completion of PREREQUISITE relationship, this embodiment adopts a collaborative mechanism combining probabilistic reasoning and graph representation learning: Feature extraction and probabilistic reasoning (Bayesian module): For any pair of knowledge points (A, B), the system extracts observation features from the graph, such as: whether they belong to the same chapter (co_chapter), whether they co-occur in the past papers (co_occurrence), the order in the textbook directory (order), and the semantic similarity of the names (semantic_sim). Using a Bayesian probabilistic graph model, these features are used as evidence E to calculate the posterior probability P(H_ab|E) of the hidden variable H_ab (i.e., "A is the prerequisite knowledge of B") as the first probability value.

[0044] Graph representation learning and link prediction (GNN module): The system uses a graph neural network (GNN) to perform node representation learning on the knowledge graph. Each node aggregates its neighbor information through multi-layer message passing, and finally obtains a vector representation containing graph structure information. Then, a link prediction model (such as DistMult or neural network) is used to calculate the second probability value of the existence of PREREQUISITE relationship between knowledge point pairs (A, B) using their final vector representations as input.

[0045] Probability fusion and relationship creation: the system performs a weighted average of the first probability value and the second probability value to obtain the comprehensive confidence Final ab that the prerequisite relationship exists for the knowledge point pair. A confidence threshold (for example, 0.75) is set. The system periodically scans all knowledge point pairs. If Final ab exceeds the threshold and the PREREQUISITE relationship does not exist in the graph, a PREREQUISITE relationship edge is automatically created from A to B, and the confidence attribute is attached to the edge, recording the confidence. The addition of the new relationship triggers the retraining of the GNN model, forming a self-enhancing closed loop.

[0046] Dynamic weight calculation implementation of knowledge points: The weight attribute of each KnowledgePoint node is dynamically calculated and updated by the following multi-factor weighted formula: W_kp=α*F_freq+β*F_semantic+γ*F_cog Wherein: frequency factor (F_freq): based on the PastQuestion node and the EXAMINES relationship system. The total number of times that the knowledge point is examined by the past exam is calculated and normalized. For example, the top 10% of the examination frequency F_freq=1.5, the middle 80% is 1.2, and the last 10% is 1.0.

[0047] Semantic association factor (F_semantic): use the Sentence-BERT model to encode the knowledge point name / description and the core theme sentence of the exam outline into vectors respectively, and calculate the cosine similarity. The similarity value is scaled to obtain F_semantic.

[0048] Cognitive level factor (F_cog): according to the CognitiveLevel node assigned by the knowledge point. For example, F_cog=1.0 for the “memory” level, 1.2 for the “understanding”, 1.5 for the “application”, and 1.8 for the “analysis and creation”.

[0049] Hyperparameters: α, β, γ are adjustable weight coefficients, satisfying α+β+γ=1, and the initial values can be set by domain experts (for example, α=0.5, β=0.3, γ=0.2).

[0050] When the system analyzes and enters new past exam papers, it will automatically update the F_freq of the related knowledge points and recalculate their W_kp, realizing the dynamic evolution of the weight.

[0051] In an embodiment of the present application, based on step S2, a possible embodiment will be given below to give a non-limiting description of its specific implementation.

[0052] S201, Material collection and preprocessing.

[0053] The system collects original proposition material texts from multiple material sources (such as authoritative news websites, academic paper databases, topic report libraries, and local teaching material repositories) through web crawlers or API interfaces. These materials may exist in formats such as PDF, Word, HTML, or plain text.

[0054] First, the system uses tool libraries such as Apache Tika for multi-format text extraction and analysis. For PDFs or scans containing mixed text and graphics, the PaddleOCR engine is used for optical character recognition. The recognized text is merged with the main text to form a complete plain text content.

[0055] Subsequently, the system performs intelligent text segmentation on the cleaned long text. To improve the accuracy of subsequent retrieval, the system uses a semantic-based segmentation strategy, preferentially using SemanticChunker or similar models, to calculate the similarity of sentence embedding vectors and cut at the boundary where the semantics change significantly, ensuring that each text block (Chunk) remains coherent in semantics. At the same time, the size of the text block (such as 512-1024 characters) and the overlap area (such as 50-100 characters) are set to maintain the integrity of the context and avoid information fragmentation.

[0056] Each text block is assigned structured metadata, including: source: material source; title: article title; topic: topic label identified and labeled by BERTopic and other topic models; summary: summary generated by BART or T5 and other summary generation models.

[0057] Finally, the system outputs a series of structured material segments with rich metadata for subsequent vectorization processing.

[0058] S202, Material vectorization representation.

[0059] The system selects a pre-trained text embedding model trained on a large-scale Chinese corpus and performs well in semantic retrieval tasks, such as bge-large-zh, text2vec-large-chinese, or M3E. Each structured material segment (including its text content and key metadata, such as the title) is input into the model, and the model outputs a high-dimensional semantic vector (such as 1024 dimensions). This vector can capture deep semantic information of the text segment and be used for subsequent similarity retrieval.

[0060] S203, Knowledge point pre-association and storage.

[0061] To achieve the preliminary association between the materials and the knowledge point system, the system adopts one or more of the following pre-association strategies to label each material segment with a knowledge point: Rule matching based on metadata: If the material segment has highly structured metadata (such as the file name contains "Chapter 3_Newton's Law of Motion.pdf"), the system directly associates it to the KnowledgePoint node named "Newton's Law of Motion" in the knowledge graph through regular expression or keyword matching, and records its kp_id.

[0062] Based on keyword and entity linking: The system uses the TextRank algorithm to extract keywords from the material segment, and uses the NER (Named Entity Recognition) model to identify proper nouns or discipline terms (such as "photosynthesis"). Then, through entity linking technology, link these entities to the corresponding knowledge point nodes in the knowledge graph to obtain their kp_id.

[0063] Based on zero-shot classifier: The system constructs an association problem as a zero-shot text classification task. Using the names and descriptions of all knowledge points in the knowledge graph as class labels, use a text classification model such as deberta or a specially fine-tuned text classification model to calculate the semantic correlation probability of the material segment with each knowledge point. Select the knowledge points whose probability exceeds the preset threshold (such as 0.6) and their kp_id as the association object.

[0064] After association, the system stores the vector representation, original text content, basic metadata, and pre-associated knowledge point identifier list (related_kp_ids) of each material segment. In this embodiment, a vector database such as Milvus or Chroma is used for storage, where the vector is stored in a special vector index, and the text, metadata and related_kp_ids are stored in the associated metadata table or collection, and are associated by unique ID. In this way, a material vector database that is preliminarily associated with the knowledge point system and supports efficient semantic retrieval is constructed.

[0065] In one embodiment of the present application, based on step S3, a possible embodiment will be given below to illustrate the specific implementation scheme.

[0066] S301, proposition constraint information extraction When the system receives a proposition request for a specific target knowledge point (such as "Newton's First Law"), it first queries and reasons in the structured knowledge network (i.e. knowledge graph) according to the unique identifier (kp_id) of the knowledge point to extract the structured constraint information required for generating the test questions. The specific extraction content includes: Cognitive Level: Determine the required cognitive level (e.g., "application") by querying the REQUIRES relationship between the knowledge point node and the CognitiveLevel node.

[0067] Difficulty Requirement: Calculate a target difficulty value (e.g., ranging from 0.0 to 1.0) by integrating the dynamic weight of the knowledge point node, the average difficulty statistics of associated historical questions, and the expert-predefined difficulty coefficient mapping rules.

[0068] Question Type Rule: Obtain the applicable proposition specification by querying the FOLLOWS relationship between the knowledge point node and the PropositionRule node, such as "single-choice questions should contain 4 options, and the correct option is unique" or "essay questions should include clear scoring points."

[0069] In addition, detailed descriptions, core concepts, common misconceptions, and associated pre / post-knowledge point information can be extracted as deep semantic constraints for generating questions.

[0070] S302, Related Material Retrieval.

[0071] The system initiates a retrieval process on the material vector database based on the target knowledge point to obtain original material segments highly related to the knowledge point semantics. The retrieval process is as follows: Construct Enhanced Query Vector: Instead of directly using the knowledge point name as a query, the system combines the knowledge point's name, description, cognitive level, and associated proposition specifications to form a rich query text (e.g., "Understanding and application of 'Newton's First Law,' cognitive level 'application,' need to design single-choice questions with practical scenarios"). Use the same pre-trained text embedding model (such as BGE-M3) as in the material vectorization stage to encode the query text into a high-dimensional query vector.

[0072] Perform Approximate Nearest Neighbor Search: Input the query vector into the HNSW index-based vector database (such as FAISS or Milvus). The system performs an approximate nearest neighbor search and returns the top K (e.g., K=5) material segments with the highest similarity to the query vector. These segments have been segmented into semantically coherent text blocks in the preprocessing stage and are attached with metadata such as source and theme.

[0073] Result Filtering and Optimization: The system can further use the pre-associated knowledge point identifiers (related_kp_ids) in the material segment metadata to filter out directly related segments to the target knowledge point and sort them by semantic similarity scores, finally selecting the most relevant 1-3 material segments for subsequent use.

[0074] S303, Constructing Generative Prompt Text.

[0075] The system combines the proposition constraint information, the retrieved relevant material segments, and the system's built-in proposition task instructions through a pre-set prompt word engineering template to form a high-quality generated prompt text. An example of the template is as follows: You are a subject proposition expert, please generate a test question according to the following requirements:

Proposition Constraints

Reference Materials

Generation Instructions

[0076] The system automatically fills in the outputs of S301 and S302 into the corresponding positions of the template to generate the final large model instructions.

[0077] S304, Batch Generation of Test Questions The system inputs the generated prompt text constructed above into a selected large language model (such as GPT-4, ChatGLM, or Ernie) to balance the standardization, innovation, and diversity of the test questions. The system adjusts the following key generation parameters of the large language model: Temperature (Temperature): Set to 0.8-1.2 to encourage the model to produce moderately creative output while following the constraints.

[0078] top_p: Set to 0.9 to control the diversity of generated content.

[0079] Maximum generation length: Set to 512-1024 tokens according to the question type.

[0080] The system can generate multiple test question drafts (e.g., 3-5 different angle test questions at a time) through repeated calls or one-time generation. All test question drafts are parsed and stored as structured data, including stem, option list, correct answer, analysis, etc., and enter the subsequent test question filtering and evaluation pipeline.

[0081] In one embodiment of the present application, based on step S4, a possible embodiment will be given below to specifically illustrate the non-limiting embodiment.

[0082] S401, test input and simulated answering.

[0083] The system inputs the test draft generated by the foregoing steps (including complete stems, options, preset answers, and analysis) to a simulated evaluation module. The module deploys at least one, usually multiple large language models (LLM) with different parameter scales as simulated test-taker agents (Simulated Test-taker Agents). For example, the system can simultaneously call a lightweight model (such as 1B parameters), a medium-sized model (such as 7B-14B parameters), and a large or professional model (such as more than 72B parameters). After receiving the test, each model will simulate the cognitive process of a real examinee, independently generate its answer to the test, including: the selected answer option (or the answer text for subjective questions), and the key reasoning process (Reasoning Chain) or problem-solving steps. The system records the output results of each agent model.

[0084] S402, multi-dimensional quality scoring.

[0085] The system automatically analyzes the answer process and results of each simulated test-taker agent according to the preset multiple quality dimensions covering educational measurement and test design professionalism, and generates quantitative scores for the test draft in each dimension. The core quality dimensions include: Logical consistency: analyze whether there are logical contradictions or ambiguities between the stem statements, the content of each option, and the reasoning process of the simulated agent. The score is based on the results of contradiction detection model and rule checking.

[0086] Disturbance item effectiveness: evaluate whether the incorrect options (disturbance items) have “truth-likeness”. By analyzing the “hesitation” degree of the simulated agent on the incorrect options (such as the length of the reasoning steps, the expression of confidence), and whether it is selected or not. The disturbance items that are easily excluded or never considered by multiple agents get low scores.

[0087] Answer uniqueness and certainty: check whether all high-capability agents (such as large models) can consistently derive a unique correct preset answer, and the reasoning is sound. If there is a disagreement or too many additional assumptions are needed to get the answer, the score will be reduced.

[0088] Cognitive level compliance: compare whether the simulated agent's problem-solving process actually triggered by the test (such as only needing to reproduce memory, or needing multi-step analysis and application) matches the target cognitive level specified in the test design constraints.

[0089] Language specification and scientificity: Accuracy of terminologies, clarity of expressions, and absence of scientific errors.

[0090] The scoring algorithm of each dimension can combine rule matching, text entailment judgment, sentiment / determinacy analysis, and indicators based on agent behavior statistics.

[0091] S403, comprehensive quality score calculation and threshold determination.

[0092] The system uses a dynamically weighted ensemble learning algorithm to integrate the scores of each quality dimension. The weights can be adaptively adjusted according to the question type, knowledge point domain, or historical review feedback. An example of the calculation formula is as follows: Comprehensive quality score = w1 * logical consistency score + w2 * interference item validity score + … + wn * language specification score Where the sum of the weight coefficients is 1, and important dimensions (such as logical consistency and answer uniqueness) are usually given higher weights.

[0093] After calculating the comprehensive quality score of the test question (usually normalized to 0-100 points), the system compares it with the preset acceptance threshold. This threshold can be dynamically set by the administrator according to the quality requirements of the question bank (for example, set to 80 points). The judgment logic is as follows: If the comprehensive quality score is greater than or equal to the acceptance threshold, the test question draft is determined to be a qualified test question, automatically enters the qualified test question bank, and can be marked with its quality level.

[0094] If the comprehensive quality score is less than the acceptance threshold, the test question is determined to be unqualified. The system can route it to the manual review link, or directly reject it and record the failure reason (such as "interference item invalid"), which can be used for iterative optimization of the proposition prompt or generation model.

[0095] S404, output and closed loop.

[0096] Finally, the system outputs a structured set of all qualified test questions, completing the full process of automated test proposition. At the same time, the data generated during the simulation evaluation process (such as the answer track of each model and the scoring details) is recorded in the log, which can be used for subsequent test proposition quality analysis, model optimization, and iterative update of the knowledge graph, forming a continuously improving intelligent proposition closed loop system.

[0097] In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the intelligent test proposition system simulating expert proposition can be stored in the memory of the computer device and executed by at least one processor to perform the functions of the intelligent test proposition simulating expert proposition (see Figure 1 Description). In some embodiments, the intelligent test proposition system simulating expert proposition can include a

[0098] In this embodiment, the intelligent proposition system simulating expert proposition can be divided into multiple functional modules according to the functions performed by the system, as shown in the figure. Figure 2 The module referred to in the present application refers to a series of computer program segments capable of being executed by at least one processor and capable of completing a fixed function, which are stored in the memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0099] The atlas construction module is used to construct a structured knowledge network containing a knowledge point system and attribute association based on multiple proposition basis data; The material processing module is used to collect proposition materials and perform vectorization representation, and establish a material vector database associated with the knowledge point system; The test question generation module is used to obtain proposition constraint information from the structured knowledge network according to a target knowledge point, and retrieve related materials from the material vector database; the proposition constraint information and the related materials are combined into a generated prompt, which is input into a large language model to generate a test question draft; The test question evaluation module is used to input the test question draft into at least one large language model as a simulated answering agent to obtain the answering process and result of the test question; based on the answering process and result, the proposition quality of the test question draft is evaluated, and a qualified test question is output.

[0100] Figure 3 The intelligent proposition method simulating expert proposition provided by the embodiments of the present application can be applied to a device. Those skilled in the art can understand that the device structure involved in the embodiments of the present application does not constitute a limitation on the device, and the device can include more or fewer components than the illustration, or combine certain components, or different component arrangements. In the embodiments of the present application, the device includes but is not limited to a laptop computer, a desktop computer, a workstation, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or claimed herein.

[0101] The device 300 can include a processor 310, a memory 320, and a communication unit 330. These components communicate through one or more buses, and those skilled in the art can understand that the structure of the server shown in the figure does not constitute a limitation on the present application, which can be a bus structure or a star structure, and can include more or fewer components than the illustration, or combine certain components, or different component arrangements.

[0102] The memory 320 can be used to store instructions for execution by the processor 310. The memory 320 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic storage devices, flash memory, magnetic disks, or optical disks. When the instructions stored in the memory 320 are executed by the processor 310, the device 300 is enabled to perform some or all of the steps in the following method embodiments.

[0103] The processor 310 is the control center of the storage device, which connects various parts of the electronic device through various interfaces and lines, and performs various functions of the electronic device and / or processes data by running or executing software programs and / or modules stored in the memory 320 and calling data stored in the memory. The processor can be composed of integrated circuits (ICs), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs connected together. For example, the processor 310 can only include a central processing unit (CPU). In embodiments of the present application, the CPU can be a single operation core or can include multiple operation cores.

[0104] The communication unit 330 is used to establish a communication channel, so that the storage device can communicate with other devices. It receives user data sent by other devices or sends user data to other devices.

[0105] The present application also provides a computer storage medium, wherein the computer storage medium can store a program, and the program can include some or all steps in the embodiments provided by the present application when executed. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0106] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present application can be implemented by means of software plus necessary universal hardware platforms. Based on such an understanding, the technical solutions in the embodiments of the present application can be embodied in the form of a software product, which is stored in a storage medium, such as a USB flash disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and the like, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a second device, a network device, or the like) to execute all or part of the steps of the methods described in the embodiments of the present application.

[0107] In the present specification, the same or similar parts among various embodiments can be referred to each other. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0108] In several embodiments provided in the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are merely schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed modules can be indirect coupling or communication connection through some interfaces, systems or modules, and can be electrical, mechanical or other forms.

[0109] The modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, that is, they can be located in one place, or can be distributed on a plurality of network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment.

[0110] In addition, each functional module in the various embodiments of the present application can be integrated in one processing module, or each module can exist physically separately, or two or more modules can be integrated in one module.

[0111] Although the present application has been described in detail with reference to the preferred embodiments, it should be understood that the application is not limited to those preferred embodiments. Various modifications and equivalents can be made by those skilled in the art without departing from the spirit and scope of the application. Any and all modifications and equivalents are intended to be included within the scope of the present application.

Claims

1. An intelligent question-setting method simulating expert question setting, characterized in that, include: Based on multiple sets of data, a structured knowledge network containing a knowledge point system and its attribute relationships is constructed. Collect the materials for the questions and represent them in vector form, and establish a material vector database associated with the knowledge point system; Based on the target knowledge points, proposition constraint information is obtained from the structured knowledge network, and relevant materials are retrieved from the material vector database; the proposition constraint information and the relevant materials are combined into generative prompts, which are then input into the large language model to generate the initial draft of the test questions; The initial draft of the test questions is input into at least one large language model that acts as a simulated answer agent to obtain its answer process and results. Based on the solution process and results, the quality of the initial draft of the test questions is evaluated, and qualified test questions are output.

2. The method according to claim 1, characterized in that, Based on data from multiple propositional bases, a structured knowledge network is constructed, encompassing a knowledge point system and its attribute relationships, including: Obtain various data sources for setting exam questions, including at least the exam syllabus, electronic textbooks, and past exam papers. The examination syllabus is analyzed to extract knowledge points, cognitive levels, and ability requirements, and the weights of knowledge points are dynamically calculated based on semantic association. The electronic teaching materials are analyzed to extract the core content paragraphs corresponding to the knowledge points; The past exam questions were analyzed to construct structured data representing the relationship between questions, knowledge points, cognitive levels, and difficulty. Based on the extracted knowledge points, weights, core content paragraphs, and structured data, a knowledge graph containing nodes, attributes, and relationships is constructed and stored. Graph reasoning models are used to infer and complete the logical dependencies between knowledge points.

3. The method according to claim 2, characterized in that, Based on the extracted knowledge points, weights, core content paragraphs, and structured data, a knowledge graph containing nodes, attributes, and relationships is constructed and stored, including: Define multiple types of nodes, including at least: KnowledgePoint nodes representing knowledge points, TextbookChapter nodes representing textbook chapter content, PastQuestion nodes representing historical test questions, PropositionRule nodes representing question-setting rules, and CognitiveLevel nodes representing cognitive ability levels. Define the relationship types connecting the nodes, including at least: the BelongsTo relationship connecting knowledge points and textbook chapters, the Examines relationship connecting historical test questions and knowledge points, the Requires relationship connecting knowledge points and cognitive ability levels, the Follows relationship connecting knowledge points and question specifications, and the Prerequisite relationship used to represent the logical dependencies between knowledge points. Prerequisite relationships between knowledge points are inferred and completed using graph reasoning models; The KnowledgePoint node has a dynamic weight attribute, and its weight value is calculated based on multi-factor weighting. The factors include at least: frequency factors obtained based on the statistical analysis of the historical test data, association factors obtained based on semantic similarity calculation, and hierarchical factors determined based on the cognitive ability level.

4. The method according to claim 3, characterized in that, The graph reasoning model is used to infer and complete the prerequisite relationships between knowledge points, including: Based on the existing nodes and relationships in the knowledge graph, at least one pair of observation features between knowledge points are extracted, and a probabilistic reasoning model is used to calculate the first probability value of the existence of a prior knowledge relationship between the knowledge point pairs. The knowledge graph is subjected to graph representation learning. The neighbor information of the nodes is aggregated through message passing to update their vector representation. Based on the updated node vector representation, a second probability value is calculated through a link prediction model to determine whether there is a prior knowledge relationship between the knowledge point pairs. By combining the first probability value and the second probability value, a comprehensive confidence level is obtained to indicate that there is a prior knowledge relationship between the knowledge point pairs. When the comprehensive confidence level exceeds a preset threshold and the relationship does not exist in the graph, a corresponding prior knowledge relationship is automatically created.

5. The method according to claim 1, characterized in that, Collect test-taking materials and represent them in vector form, and establish a material vector database associated with the knowledge point system, including: Original proposition material texts are collected from at least one source of material, and the original proposition material texts are subjected to topic identification and summary generation to obtain structured material fragments; The structured material fragments are encoded into fixed-dimensional vector representations using a pre-trained text embedding model. Based on at least one pre-association strategy, the preliminary association between the structured material fragment and one or more knowledge points in the knowledge point system is determined, and the knowledge point identifier representing the preliminary association is stored as metadata corresponding to the vector representation.

6. The method according to claim 1, characterized in that, Based on the target knowledge points, propositional constraint information is obtained from the structured knowledge network, and relevant materials are retrieved from the material vector database; The propositional constraints and related materials are combined into generative prompts, which are then input into a large language model to generate a draft of the test questions, including: Based on the target knowledge points, propositional constraint information is extracted from the structured knowledge network. The propositional constraint information includes at least the cognitive level, difficulty requirements, and applicable question type rules of the target knowledge points. Based on the target knowledge points, at least one semantically related material fragment is retrieved from the material vector database; The proposition constraint information, the at least one material fragment, and the preset proposition task instructions are formatted and combined to construct generative prompt text; The generative prompt text is input into a large language model. By adjusting the generation parameters of the large language model to control the output characteristics, initial drafts of test questions that conform to the propositional constraints are generated in batches.

7. The method according to claim 1, characterized in that, Based on the aforementioned solution process and results, the quality of the initial draft of the test questions is evaluated, and qualified test questions are output, including: Based on multiple preset quality dimensions, the solution process and results generated by the simulated answer agent are analyzed to generate scores for the initial draft of the test questions on each quality dimension. Based on the scores across each quality dimension, the overall quality score of the initial draft of the test questions is calculated. The overall quality score is compared with a preset acceptance threshold to determine whether the initial draft of the test questions meets the standards.

8. An intelligent question-setting system that simulates expert question setting, characterized in that, include: The knowledge graph construction module is used to construct a structured knowledge network containing knowledge point systems and their attribute relationships based on multiple propositional data. The material processing module is used to collect the proposition materials and represent them in vector form, and to establish a material vector database associated with the knowledge point system. The question generation module is used to obtain proposition constraint information from the structured knowledge network based on the target knowledge points, and retrieve relevant materials from the material vector database; combine the proposition constraint information and the relevant materials into generative prompts, and input them into the large language model to generate the initial draft of the question; The test question evaluation module is used to input the initial test question into at least one large language model that acts as a simulated answer agent, obtain its answer process and results for the test question; based on the answer process and results, evaluate the quality of the initial test question and output qualified test questions.

9. An intelligent question-setting device that simulates expert question setting, characterized in that, include: The memory is used to store the intelligent question-setting program that simulates expert questions; A processor, configured to implement the steps of the intelligent proposition method for simulating expert propositions as described in any one of claims 1-7 when executing the intelligent proposition program for simulating expert propositions.

10. A computer-readable storage medium storing a computer program, characterized in that, The readable storage medium stores an intelligent question-setting program that simulates expert question setting. When the intelligent question-setting program that simulates expert question setting is executed by a processor, it implements the steps of the intelligent question-setting method that simulates expert question setting as described in any one of claims 1-7.

Images