Question bank generation method and device based on large language model, electronic equipment and computer readable storage medium
By generating question parameters that meet the requirements of metrology through large language models and parameter prediction models, the problems of long question bank construction cycle and high cost are solved, and the question bank generation is achieved quickly and efficiently.
Patent Information
- Application Number
- CN202611229536.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-13
- Publication Date
- 2026-11-06
AI Technical Summary
In existing technologies, multi-stage adaptive testing systems rely on real people to pre-test and obtain question parameters, resulting in long question bank construction cycles, huge costs, and newly added questions lacking historical data support and cannot be used immediately.
Candidate test questions are generated using a large language model, and the semantic and structured features of the text are extracted through a prediction model with preset parameters. The initial parameters that meet the theoretical parameters of the item response are directly predicted, and the questions that meet the requirements of metrology are generated.
It eliminates the need for human pre-testing and massive data calibration, quickly generating question parameters that meet metrological requirements, shortening the question bank construction cycle and reducing costs.
Smart Images
Figure CN123051642A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of educational testing, and more specifically, to a method, apparatus, electronic device, and computer-readable storage medium for generating a question bank based on a large language model. Background Technology
[0002] In related technologies, multi-stage adaptive testing systems rely heavily on the item response theory parameters of the questions as their core foundation. This means that each question added to the database must possess precise parameters such as difficulty, discrimination, and guessing ability. However, obtaining these parameters typically involves large-scale pre-testing with real people, collecting massive amounts of response data, and then statistically calibrating it. This results in extremely long and costly question bank construction cycles, and newly added questions cannot immediately yield effective parameters due to a lack of historical data support. Summary of the Invention
[0003] In view of the above problems, this application proposes a question bank generation method, apparatus, electronic device and computer-readable storage medium based on a large language model, which can solve the above problems.
[0004] In a first aspect, embodiments of this application provide a question bank generation method based on a large language model. The method includes: receiving domain knowledge data and inputting the domain knowledge data into a preset large language model to generate candidate questions; extracting textual semantic features and structural features of the candidate questions, and concatenating the textual semantic features and structural features to obtain an input feature vector; inputting the input feature vector into a preset parameter prediction model, extracting latent representations through a shared fusion network of the preset parameter prediction model, outputting original predicted values from multiple output heads of the preset parameter prediction model, and performing constraint transformation on the original predicted values to obtain initial parameters that conform to the parameter range of the item response theory; and storing the candidate questions and initial parameters in an available question bank.
[0005] Secondly, embodiments of this application also provide a question bank generation device based on a large language model. The device includes: a generation module for receiving domain knowledge data and inputting the domain knowledge data into a preset large language model to generate candidate questions; a concatenation module for extracting textual semantic features and structural features of the candidate questions and concatenating the textual semantic features and structural features to obtain an input feature vector; a constraint module for inputting the input feature vector into a preset parameter prediction model, extracting latent representations through a shared fusion network of the preset parameter prediction model, outputting original predicted values from multiple output heads of the preset parameter prediction model, and performing constraint transformations on the original predicted values to obtain initial parameters that conform to the theoretical parameter range of the item response theory; and a storage module for associating and storing the candidate questions and initial parameters in an available question bank.
[0006] Thirdly, embodiments of this application also provide an electronic device, including a processor, a memory, and one or more application programs; the one or more application programs are stored in the memory and configured to be executed by the processor to implement the above-described question bank generation method based on a large language model.
[0007] Fourthly, embodiments of this application also provide a computer-readable storage medium storing program code, wherein the above-described question bank generation method based on a large language model is executed when the program code is run by a processor.
[0008] The technical solution provided in this application includes the following method: receiving domain knowledge data and inputting the domain knowledge data into a preset large language model to generate candidate test questions; extracting the textual semantic features and structural features of the candidate test questions, and concatenating the textual semantic features and structural features to obtain an input feature vector; inputting the input feature vector into a preset parameter prediction model, extracting the latent representation through the shared fusion network of the preset parameter prediction model, outputting the original predicted values from multiple output heads of the preset parameter prediction model respectively, and performing constraint transformation on the original predicted values to obtain initial parameters that conform to the theoretical parameter range of item response; and storing the candidate test questions and initial parameters in a usable question bank. Thus, by generating candidate test questions through a preset large language model and directly predicting initial parameters that conform to the theoretical parameter range of item response from the textual semantic features and structural features of the test questions using the preset parameter prediction model, without relying on large-scale human pre-testing and statistical calibration of massive amounts of answer data, newly added test questions can immediately obtain accurate initial parameters even without historical data support. This directly solves the technical problems of long question bank construction cycles, huge costs, and the inability to immediately deploy new questions in related technologies. Attached Figure Description
[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments and drawings obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0010] Figure 1 The illustration shows a flowchart of a question bank generation method based on a large language model provided in an embodiment of this application.
[0011] Figure 2 The diagram shows a structural schematic of a question bank generation device based on a large language model provided in an embodiment of this application.
[0012] Figure 3A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown.
[0013] Figure 4 This illustration shows a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation
[0014] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.
[0015] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0016] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.
[0017] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0018] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0019] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.
[0020] This invention provides a question bank generation method based on a large language model. The execution entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, this question bank generation method based on a large language model can be executed by software or hardware installed on a terminal device or a server device. The software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cluster of cloud servers. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0021] Please see Figure 1 , Figure 1 This illustration shows a flowchart of a question bank generation method based on a large language model provided in an embodiment of this application. Figure 1 As shown, the method may include steps 110 to 140.
[0022] In step 110, domain knowledge data is received and input into a preset large language model to generate candidate test questions.
[0023] Domain knowledge data can refer to a collection of knowledge in a specific discipline or professional field. For example, domain knowledge data can be a teaching syllabus or professional documents.
[0024] The pre-defined large language model is a large language model fine-tuned using data from the education field. The pre-defined large language model can be a pre-trained model based on the Transformer architecture, such as the GPT series, BERT series, or LLaMA series.
[0025] Furthermore, in some implementations, the step of inputting domain knowledge data into a preset large language model to generate candidate test questions may also include the following steps: (1) Input the domain knowledge data into the preset large language model for extraction to obtain the knowledge point entities, the knowledge categories to which the knowledge point entities belong, and the cognitive associations included in the domain knowledge data; wherein, the cognitive associations represent the semantic association between different knowledge point entities or between knowledge point entities and knowledge categories; (2) Construct a domain knowledge graph based on knowledge point entities, the knowledge categories to which the knowledge point entities belong, and cognitive relationships; (3) Based on the preset user capability estimate or preset test paper constraints, match nodes from the domain knowledge graph that are compatible with the preset user capability estimate or that are consistent with the preset test paper constraints to determine the target knowledge point entity; (4) Generate candidate questions based on the target knowledge point entity, the knowledge category to which the target knowledge point entity belongs, and the cognitive relationship corresponding to the target knowledge point entity.
[0026] The pre-defined large language model uses natural language processing technology to automatically identify and extract knowledge point entities and their respective knowledge categories from domain knowledge data. For example, knowledge point entities can be the Pythagorean theorem, Newton's second law, etc.; or the knowledge category they belong to can be geometry or mechanics.
[0027] At the same time, the pre-set large language model will also analyze the internal logic between knowledge point entities and extract cognitive relationships. These relationships are represented by the numerical form of semantic relevance, which indicates the degree of closeness between different knowledge point entities or between knowledge point entities and the knowledge categories to which they belong. For example, the relevance between the Pythagorean theorem and a right triangle is 0.95.
[0028] The system uses knowledge point entities, the knowledge categories to which the knowledge point entities belong, and cognitive relationships as nodes and edges to construct a structured domain knowledge graph.
[0029] Before generating candidate test questions, the system will intelligently match them based on the current exam scenario. In the case of an adaptive exam, the system will read the candidate's preset user ability assessment and search for knowledge point nodes in the knowledge graph that match that ability level. For example, a candidate with a preset user ability assessment of 0.5 will be matched with knowledge points of medium difficulty.
[0030] If it is a fixed test paper, nodes that meet the conditions are selected in the knowledge graph according to the preset test paper constraints. The preset test paper constraints can be a set of structured rules used to guide the matching of knowledge graph nodes and the generation of test questions. For example, the preset test paper constraints can be mechanics accounting for 30% and difficulty coefficient 0.6-0.8.
[0031] After accurately locating the target knowledge point entity that meets the candidate's ability or test paper requirements by pre-setting user ability assessment or pre-setting test paper constraints, the target knowledge point entity, the knowledge category to which the target knowledge point entity belongs, and the corresponding cognitive relationship are input into the pre-set large language model. Based on this structured information, the pre-set large language model generates candidate test questions that are deeply bound to the target knowledge point entity.
[0032] For example, when generating Pythagorean theorem questions, the pre-set large language model will refer to its relationship with right triangles to ensure that the background of right triangles is naturally integrated into the question stem. This makes the generated questions not only test the target knowledge points, but also reflect the logical connection between knowledge points, thereby improving the professionalism and scientific nature of the questions.
[0033] In some implementations, the question bank generation method based on a large language model may further include the following steps: (1) Obtain the target test question and the user's answer data for the target test question; the target test question is any test question in the available question bank; (2) When the target test question is an objective question, determine the user's ability estimate based on the item response theory model and the answer data; (3) When the target test question is a subjective question, the answer data is semantically compared and logically scored according to the graded response model or local credit model to determine the user's ability assessment; (4) Update the preset user capability valuation based on capability valuation.
[0034] The target test questions are derived from the set of test questions in the available question bank. They include mature test questions whose parameters have been calibrated after closed-loop anchor question verification, as well as candidate test questions generated by the large language model and entered into the database after parameter constraint processing.
[0035] The target questions can be dynamically selected from the available question bank by an adaptive enhanced equalization exposure controller. During the dynamic selection process, the controller maintains a real-time exposure count for each question and dynamically adjusts the exposure limit according to the question type. For candidate questions generated through this application's solution, the exposure limit is automatically relaxed to accelerate the calibration and iteration of new questions. For mature questions calibrated manually, the preset exposure limit is strictly enforced to avoid overexposure that could lead to security risks in the question bank.
[0036] Under the premise of meeting the exposure threshold, a hybrid algorithm combining maximum information content priority and exposure equalization compensation is adopted. First, the Fisher information content of each question in the available question bank is calculated to estimate the current user's ability, and the top N questions with the highest information content are selected as the candidate set. Then, exposure equalization compensation is applied to the candidate set to reduce the probability of selecting questions with exposure counts close to the upper limit and increase the probability of selecting low-exposure questions. Finally, a question is randomly selected from the compensated candidate set as the target question. After selection, a second security check is performed to ensure that the question is not marked as abnormal. If the check fails, the selection process is retried until a target question that meets the security standard is selected.
[0037] When the target question is an objective question, the system inputs the user's binary answer and the question's IRT parameters into the Item Response Theory (IRT) model, and calculates the user's ability estimate using maximum likelihood estimation or Bayesian estimation methods. If the target question is an anchor question, the user's ability estimate is directly calibrated using its known parameters; if it is a regular question, the estimate is corrected by combining the calibration results of the anchor question to ensure the accuracy of the ability estimate.
[0038] When the target question is a subjective question, the system invokes a large language model scoring agent, fine-tuned for the education domain, to perform semantic comparison and logical scoring on the answer text, outputting a continuous score as the raw score. Subsequently, based on a preset score interval and level correspondence, the raw score is mapped to the target level, and this level is used as an observation input into a level response model or a local credit model to calculate the user's ability estimate for that subjective question. If the answer involves multiple steps, the local credit model is used to independently evaluate the scores of each step before a comprehensive calculation.
[0039] Specifically, in some implementations, the question bank generation method based on a large language model may further include the following steps: (1) Obtain the original score of the target test question. The original score is the continuous score output by the preset large language model after semantic comparison and logical scoring of the answer data. (2) Based on the pre-set correspondence between score ranges and levels, map the original scores to the target levels; (3) Input the target level as an observation into the level response model or local credit model to update the preset user capability estimate.
[0040] When the target question is a subjective question, the system first calls a pre-defined large language model to process the user's answer data. For example, the pre-defined large language model acts as a scoring agent, performing in-depth semantic comparison and logical consistency analysis between the user's answer text and the standard answer and scoring rules. Based on its deep understanding of natural language, the pre-defined large language model evaluates the completeness, logic and accuracy of the answer content, and outputs a continuous raw score.
[0041] The system maps continuous raw scores to discrete target levels based on a pre-defined correspondence between score ranges and levels. For example, the system can preset scores of 0-59 as unqualified, 60-79 as qualified, and 80-100 as excellent. This transforms unstructured text responses into structured level observations, enabling them to be recognized and processed by level response models or local credit models, thereby quantifying and standardizing subjective question response results.
[0042] The system uses the mapped target level as an observation and inputs it into either a level-response model or a local credit model to update the preset user ability estimate. If a level-response model is used, the system calculates the probability of a user answering a question based on the target level and, combined with the question's difficulty and discrimination parameters, uses maximum likelihood estimation or Bayesian estimation to infer the user's ability estimate. If a local credit model is used, it further considers the scores at each step of the answering process and performs a weighted calculation of the ability contribution of each step. The updated user ability estimate will replace the original preset value in real time and serve as the basis for the next round of question recommendations.
[0043] Finally, the system weights and merges the ability estimates calculated from objective and subjective questions to obtain a comprehensive ability estimate for the user, and writes this updated preset user ability estimate into the user's ability profile in real time. This updated preset value will serve as input for matching target knowledge point entities from the domain knowledge graph, enabling personalized test question generation based on the user's real-time ability status.
[0044] In step 120, the textual semantic features and structured features of the candidate test questions are extracted, and the textual semantic features and structured features are concatenated to obtain the input feature vector.
[0045] The system extracts multi-dimensional features from candidate test questions to construct input feature vectors that comprehensively represent the attributes of the test questions. For example, the extraction of textual semantic features utilizes a pre-trained language model to encode the stem, options, and parsing text of the candidate test questions, converting unstructured natural language into high-dimensional dense vectors.
[0046] High-dimensional dense vectors can capture the deep semantic information of test questions, including the contextual meaning of words, the complexity of syntactic structures, and the way knowledge points are expressed, thereby quantifying the position of test questions in semantic space.
[0047] The extraction of structured features involves digitally encoding explicit and regularized attributes in the test questions, including but not limited to question type labels (single-choice, multiple-choice, short-answer), knowledge point labels (Pythagorean theorem, Newton's second law), cognitive level labels (memory, understanding, application), and statistical features such as question stem length and number of options. These structured features are converted into fixed-dimensional vector representations through one-hot encoding, numerical normalization, or embedded lookup tables to preserve the metadata information of the test questions.
[0048] The system concatenates the extracted text semantic feature vectors with the structured feature vectors to obtain the final input feature vector. This concatenation process is not a simple vector join, but rather aligns and integrates semantic and structural information through a pre-defined fusion strategy. The pre-defined fusion strategy can fuse semantic information through direct concatenation, weighted concatenation, or attention mechanisms.
[0049] The concatenated input feature vector contains both the deep semantic meaning of the test questions and retains their explicit structured attributes, thus providing a comprehensive and multidimensional input representation for the subsequent parameter prediction model.
[0050] In step 130, the input feature vector is input into the preset parameter prediction model. The hidden representation is extracted through the shared fusion network of the preset parameter prediction model. The original predicted values are output by multiple output heads of the preset parameter prediction model. The original predicted values are then subjected to constraint transformation to obtain initial parameters that conform to the theoretical parameter range of the project response.
[0051] The shared fusion network can be composed of multiple layers of neural networks (Transformer encoder layers or fully connected layers). After entering the shared fusion network, textual semantic features and structured features are not processed separately through independent branches, but are input into the same set of network layers for feature interaction and fusion. Through the layer-by-layer transformation of the shared fusion network, the input feature vector is mapped into a high-dimensional implicit representation.
[0052] After obtaining the implicit representation, the pre-defined parameter prediction model outputs raw predicted values through multiple independent output heads. Each output head corresponds to a specific parameter in the item response theory: for example, the first output head outputs the raw predicted value of the difficulty parameter, the second output head outputs the raw predicted value of the discrimination parameter, and the third output head outputs the raw predicted value of the guessing parameter. In other words, the raw predicted values include discrimination, difficulty, and guessing.
[0053] The system performs constraint transformation on the raw predicted values output from multiple output heads, mapping them to initial parameters that conform to the theoretical parameter range of the project response. Specifically, in some implementations, the constraint transformation includes at least: constraining the discrimination index to a preset positive interval through a soft addition function and a truncation operation, constraining the difficulty value to a preset real number interval through a truncation operation, and constraining the guessing probability to a preset probability interval through a soft addition function and a scaling operation.
[0054] For the difficulty value, a truncation function is used to constrain the original predicted value to a preset real number interval (e.g., [-3, 3]), ensuring that the difficulty value is within the reasonable range of the IRT model. For the discrimination parameter, a soft addition function is used to map the original predicted value to a positive real number space, and then a truncation operation is used to constrain it to a preset positive interval (e.g., [0.5, 3.0]), ensuring that the discrimination is always positive and meets metrological requirements. For the guessing parameter, a soft addition function is used to map the original predicted value to a probability space, and then a scaling operation is used to constrain it to a preset probability interval (e.g., [0.1, 0.3]), ensuring that the guessing probability is between 0 and 1 and conforms to the probability law of guessing correctly in multiple-choice questions. After the constraint transformation, the original predicted value is converted into initial parameters that conform to the theoretical range of IRT.
[0055] In some implementations, the question bank generation method based on a large language model may further include the following steps: (1) Construct a multi-task loss function; the multi-task loss function includes difficulty prediction loss value, discrimination prediction loss value and guessing prediction loss value; (2) Based on the multi-task loss function, the adaptive moment estimation optimizer is used to train the parameter prediction model to be trained until the validation set loss converges; (3) Solidify the model parameters after training convergence into a preset parameter prediction model.
[0056] The preset parameter prediction model is trained using a multi-task learning framework. Its multi-task loss function is a weighted combination of three parts: difficulty prediction loss, discrimination prediction loss, and guessing prediction loss. In a specific implementation, the multi-task loss function can be expressed as: in, For the total loss, To predict the loss value based on difficulty, To predict the loss value for discrimination, To predict the loss value based on guesswork, , , Configurable weights.
[0057] In one specific implementation method It can be 1, It can be 1.2. It can be 0.8.
[0058] The difficulty prediction loss uses mean squared error loss to measure the numerical deviation between the difficulty parameter predicted by the model and the actual difficulty parameter; the discrimination prediction loss uses smoothed mean absolute error loss to reduce the interference of abnormal samples on the training process while preserving gradient stability; and the guessing prediction loss uses binary cross-entropy loss to adapt to the distribution characteristics of the guessing parameter in the probability interval of 0-1.
[0059] During the training phase of the parameter prediction model to be trained, the system uses an adaptive moment estimation optimizer to optimize the parameter prediction model to be trained. The optimizer dynamically adjusts the learning rate for each parameter in the model by maintaining the first-order moment estimate and second-order moment estimate of the gradient, which effectively solves the problems of inconsistent gradient scale and slow convergence speed in multi-task training.
[0060] Furthermore, the multi-task loss value on the validation set is calculated in real time. When the validation set loss does not decrease significantly for several consecutive iterations, the model is determined to have converged and training is terminated.
[0061] Once the model training converges, the system stores the model parameters with the lowest validation set loss during training, resulting in the final preset parameter prediction model. This storage process is not simply saving the parameters at the end of training, but rather continuously recording the model parameters with the optimal validation set loss during training and loading these optimal parameters as the final weights of the preset parameter prediction model after training.
[0062] The solidified model parameters are encapsulated in a read-only state. In subsequent question generation and parameter prediction tasks, only forward inference is performed, and no further parameter updates are made. This avoids catastrophic forgetting and wasted computing power caused by online training, and ensures the stability and repeatability of parameter prediction results.
[0063] In step 140, candidate questions and initial parameters are associated and stored in the available question bank.
[0064] The system employs a hybrid storage architecture combining relational and vector databases to achieve a structured association between candidate questions and initial parameters. For example, a unique identifier can be assigned to each candidate question from the question bank, and a composite data table containing question text fields and IRT parameter fields can be created. The question text fields store unstructured content such as the question stem, options, standard answer, and explanation; while the IRT parameter fields structurally store the difficulty value, discrimination index, and guessing index (i.e., initial parameters) output by the parameter prediction model.
[0065] A one-to-one mapping relationship is established between the test question text field and the IRT parameter field through a unique identifier, ensuring that the test question content and measurement parameters can be read synchronously during subsequent retrieval and retrieval, avoiding evaluation bias caused by parameter and test question mismatch.
[0066] In some implementations, the question bank generation method based on a large language model may further include the following steps: (1) Determine the prediction confidence of the initial parameters through a prediction model with preset parameters; (2) When the prediction confidence is greater than or equal to the preset confidence threshold, the candidate questions are marked as available and stored in the available question bank; (3) When the prediction confidence is less than the preset confidence threshold, the candidate test item is marked as an observation and stored in the observation item queue.
[0067] The system uses a preset parameter prediction model to assess the uncertainty of the currently generated candidate test questions and calculates the prediction confidence of the initial parameters. The prediction confidence reflects the degree of certainty that the preset parameter prediction model has about the prediction results of the current test question parameters.
[0068] The system compares the predicted confidence level with the preset confidence threshold. When the predicted confidence level is greater than or equal to the preset confidence threshold, it indicates that the preset parameter prediction model has a high degree of confidence in predicting the parameters of the question. The system marks the candidate question as available and directly stores it in the available question bank so that it can immediately participate in the subsequent test paper generation or testing process.
[0069] When the prediction confidence is less than the preset confidence threshold, it indicates that the preset parameter prediction model has a large uncertainty in predicting the parameters of the test questions. The system marks the candidate test questions as being in the observation state and stores them in the observation question queue. Test questions in the observation state usually need to be manually reviewed, tested on a small scale, or further calibrated before they can be converted to the usable state.
[0070] By introducing confidence assessment and a tiered entry mechanism, this embodiment can effectively intercept low-quality generated test questions and prevent unreliable parameters from entering the official question bank. This significantly improves the overall measurement quality and security of the question bank while ensuring the efficiency of question bank generation.
[0071] Please see Figure 2 , Figure 2 This illustration shows a structural diagram of a question bank generation device based on a large language model according to an embodiment of this application. The question bank generation device 200 based on the large language model includes: a generation module 210, a splicing module 220, a constraint module 230, and a storage module 240. Specifically: The generation module 210 is used to receive domain knowledge data and input the domain knowledge data into a preset large language model to generate candidate test questions; The concatenation module 220 is used to extract the textual semantic features and structural features of candidate test questions, and concatenate the textual semantic features and structural features to obtain the input feature vector; The constraint module 230 is used to input the input feature vector into the preset parameter prediction model, extract the hidden representation through the shared fusion network of the preset parameter prediction model, output the original prediction value by the multiple output heads of the preset parameter prediction model, and perform constraint transformation on the original prediction value to obtain the initial parameters that conform to the theoretical parameter range of the project response. Storage module 240 is used to associate candidate questions and initial parameters in the available question bank.
[0072] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0073] In the several embodiments provided in this application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or other forms.
[0074] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.
[0075] Please see Figure 3 , Figure 3 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. The electronic device 300 in this application may include one or more of the following components: a processor 310, a memory 320, and one or more application programs. The one or more application programs may be stored in the memory 320 and configured to be executed by one or more processors 310. The one or more programs are configured to execute the question bank generation method based on a large language model as described in the foregoing method embodiments.
[0076] Processor 310 may include one or more processing cores. Processor 310 connects to various parts within the electronic device 300 using various interfaces and lines, and performs various functions and processes data of the electronic device 300 by running or executing instructions, programs, code sets, or instruction sets stored in memory 320, and by calling data stored in memory 320. Optionally, processor 310 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 310 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 310 and may be implemented separately using a communication chip.
[0077] The memory 320 may include random access memory (RAM) or read-only memory (ROM). The memory 320 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 320 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described below, etc. The data storage area may also store data created by the electronic device 300 during use.
[0078] Please see Figure 4 , Figure 4 The diagram shows a computer-readable storage medium 400 provided in an embodiment of this application. The computer-readable storage medium 400 stores program code, which can be called by a processor to execute the question bank generation method based on a large language model described in the above method embodiment.
[0079] The computer-readable storage medium 400 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 400 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 400 has storage space for program code 410 that performs any of the method steps described above. This program code can be read from or written to one or more computer program devices. The program code 410 may be compressed, for example, in a suitable form.
[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A method for generating a question bank based on a large language model, characterized in that, The method includes: Receive domain knowledge data and input the domain knowledge data into a preset large language model to generate candidate test questions; Extract the textual semantic features and structural features of the candidate test questions, and concatenate the textual semantic features and structural features to obtain the input feature vector; The input feature vector is input into the preset parameter prediction model. The hidden representation is extracted through the shared fusion network of the preset parameter prediction model. The original predicted values are output by the multiple output heads of the preset parameter prediction model. The original predicted values are then subjected to constraint transformation to obtain initial parameters that conform to the theoretical parameter range of the project response. The candidate questions and the initial parameters are associated and stored in the available question bank. 2.The large language model-based question bank generation method of claim 1, wherein, The original predicted values include discrimination, difficulty, and guessing ability; The constraint transformation includes at least: constraining the discrimination degree to a preset positive interval through a soft addition function and a truncation operation, constraining the difficulty value to a preset real number interval through a truncation operation, and constraining the guessing degree to a preset probability interval through a soft addition function and a scaling operation. 3.The big language model-based question bank generation method of claim 1, wherein, The method further includes: Construct a multi-task loss function; the multi-task loss function includes difficulty prediction loss value, discrimination prediction loss value and guessing prediction loss value; Based on the multi-task loss function, an adaptive moment estimation optimizer is used to train the parameter prediction model to be trained until the validation set loss converges. The parameters of the model after training convergence are solidified into the preset parameter prediction model. 4.The big language model-based question bank generation method of claim 1, wherein, The steps involve inputting the domain knowledge data into a preset large language model to generate candidate test questions, including: The domain knowledge data is input into the preset large language model for extraction to obtain the knowledge point entities, the knowledge categories to which the knowledge point entities belong, and the cognitive associations included in the domain knowledge data; wherein, the cognitive associations represent the semantic association between different knowledge point entities, or between the knowledge point entities and the knowledge categories; A domain knowledge graph is constructed based on the knowledge point entities, the knowledge categories to which the knowledge point entities belong, and the cognitive relationships. Based on a preset user capability estimate or a preset test case constraint, nodes that match the preset user capability estimate or the preset test case constraint are matched from the domain knowledge graph to determine the target knowledge point entity. The candidate test questions are generated based on the target knowledge point entity, the knowledge category to which the target knowledge point entity belongs, and the cognitive association relationship corresponding to the target knowledge point entity. 5.The large language model-based question bank generation method of claim 4, wherein, The method further includes: Obtain the target test question and the user's answer data for the target test question; the target test question is any question in the available question bank; When the target test question is an objective question, the user's ability estimate is determined based on the item response theory model and the answer data. When the target test question is a subjective question, the user's ability is estimated by semantic comparison and logical scoring of the answer data according to the graded response model or local credit model. Based on the capability valuation, update the preset user capability valuation. 6.The large language model-based question bank generation method of claim 5, wherein, The method further includes: Obtain the original score of the target test question. The original score is a continuous score output by the preset large language model after semantic comparison and logical scoring of the answer data. Based on the preset correspondence between score ranges and levels, the original score is mapped to the target level; The target level is input as an observation into the level response model or local credit model to update the preset user capability estimate.
7. The question bank generation method based on a large language model according to claim 1, characterized in that, The method includes: The prediction confidence level of the initial parameters is determined by the prediction model with the preset parameters; When the prediction confidence level is greater than or equal to the preset confidence threshold, the candidate question is marked as available and stored in the available question bank; When the prediction confidence is less than the preset confidence threshold, the candidate question is marked as an observation and stored in the observation question queue.
8. A question bank generation device based on a large language model, characterized in that, The device includes: The generation module is used to receive domain knowledge data and input the domain knowledge data into a preset large language model to generate candidate test questions; The concatenation module is used to extract the textual semantic features and structural features of the candidate test questions, and concatenate the textual semantic features and structural features to obtain the input feature vector; The constraint module is used to input the input feature vector into the preset parameter prediction model, extract the hidden representation through the shared fusion network of the preset parameter prediction model, output the original prediction value by the multiple output heads of the preset parameter prediction model, and perform constraint transformation on the original prediction value to obtain the initial parameters that conform to the theoretical parameter range of the project response. A storage module is used to associate and store the candidate questions and the initial parameters in an available question bank.
9. An electronic device, characterized in that, include: One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to perform the question bank generation method based on a large language model as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code, which can be called by a processor to execute the question bank generation method based on a large language model as described in any one of claims 1-7.