Question bank generation method and system based on pre-trained multi-modal large model

Abstract

The application provides a question bank generation method and system based on a pre-trained multi-modal large model, comprising: obtaining set person setting keywords and scene parameters in user requirements; inputting the person setting keywords and scene parameters into a multi-modal large model that has been pre-trained, combining the multi-modal knowledge system of the multi-modal large model itself, and outputting a question bank that meets the user requirements; the multi-modal large model is pre-trained based on multi-modal data in different fields, combined with a reinforcement learning task after unified integration of multi-task links and a corresponding reward function; the multi-modal large model in the application can process multi-modal information at the same time, improving the generalization ability of the model; multi-task link integration can enable deep fusion and collaborative reinforcement training of each link, accurately understand and efficiently complete complex question bank generation tasks; the introduction of a multi-modal knowledge system enables the model to quickly transfer learning based on the correlation therein, improving the universality of question bank generation.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, specifically to a method and system for generating a question bank based on a pre-trained multimodal large model. Background Technology

[0002] In modern workplaces, ensuring that knowledge and skills remain aligned with company requirements often necessitates learning a large amount of new materials and documents. These materials may include technical manuals, operating instructions, legal documents, internal training materials, and more. This ongoing learning need compels offices to seek effective methods to verify employees' understanding and mastery of these materials. This is often done by creating questionnaires based on the latest materials; however, manually creating questionnaires is inefficient and inconsistent in difficulty and quality, impacting the verification and assessment of employees.

[0003] Therefore, the existing method uses a generative model to generate a target questionnaire that matches the set personality keywords; and uses a generative model to generate clue information corresponding to each answer option; based on the personality keywords, questionnaire questions, answer options and clue information, model training data for the generative model is constructed; the generative model is trained using the model training data to obtain the question bank generation model; the question bank generation model is used to output a question bank that matches the type of the target questionnaire. This method can improve the efficiency of question bank generation.

[0004] However, when using generative models to build question bank generation models, there is a large amount of content to process with high relevance requirements, involving multiple steps such as keyword matching, clue information generation, and training data construction based on these data points. This series of steps makes the model training process complex, requiring extensive data preprocessing and parameter tuning to achieve satisfactory generation results. When dealing with real-world application scenarios with massive and diverse data, the model training and optimization process becomes even more cumbersome, complex, and inefficient. Furthermore, using personified keywords and clue matching generates specific types of question banks, resulting in low model generalization ability. When the knowledge domain changes, the model needs to be retrained, limiting the flexibility and application scope of question bank generation. Summary of the Invention

[0005] To overcome the shortcomings of the existing technology, this invention proposes a question bank generation method based on a pre-trained multimodal large model, including:

[0006] Obtain the persona keywords and scene parameters set in the user requirements;

[0007] The persona keywords and scene parameters are input into the pre-trained multimodal large model, and combined with the multimodal large model's own multimodal knowledge system, a question bank that meets the user's needs is output.

[0008] The multimodal large model is obtained by pre-training based on multimodal data from different domains, combined with reinforcement learning tasks and corresponding reward functions that are unified and integrated from multiple tasks.

[0009] Optionally, the pre-training process of the multimodal large model includes:

[0010] Collect multimodal data from different knowledge domains;

[0011] Based on the multimodal data, combined with the adaptive mechanism and self-supervised learning task of the multimodal large model, the multimodal large model is trained to obtain the data generated during the training process of the multimodal large model and the multimodal large model that has completed adaptive self-supervised training.

[0012] The multiple task components in the multimodal large model are integrated into a unified reinforcement learning task;

[0013] The multimodal data and the data generated during the training process of the multimodal large model are used as reinforcement training data;

[0014] The reward function designed based on the objectives and evaluation indicators of each task stage is combined with the reinforcement training data to perform reinforcement learning training on the multimodal large model that has completed adaptive self-supervised training, thereby obtaining a pre-trained multimodal large model.

[0015] The multiple task stages include: character keyword extraction, question generation, answer option construction, and clue generation.

[0016] Optionally, the step of training the multimodal large model based on the multimodal data, combined with the adaptive mechanism and self-supervised learning task of the multimodal large model, to obtain the multimodal large model that has completed adaptive and self-supervised training, includes:

[0017] Based on the multimodal data, a general feature training is performed on the multimodal large model to obtain a multimodal large model that has completed the general feature training;

[0018] By utilizing the adaptive adjustment mechanism in the multimodal large model, the weights and parameters of the multimodal large model that has completed general feature training are adjusted based on the persona keywords in the multimodal data. At the same time, by utilizing the self-supervised learning task preset in the multimodal large model, the inherent laws of the multimodal data are learned, and the multimodal large model that has completed adaptive self-supervised training is obtained.

[0019] Optionally, the reward function designed based on the objective and evaluation index of each task stage, combined with reinforcement training data, is used to perform reinforcement learning training on the multimodal large model that has completed adaptive self-supervised training, to obtain a pre-trained multimodal large model, including:

[0020] Set objectives and evaluation metrics for each of the aforementioned task stages;

[0021] The reward function for each task stage is determined based on the stated objectives and evaluation metrics.

[0022] The reinforcement training data is input into the multimodal large model that has completed adaptive self-supervised training. The multimodal large model generates reward feedback based on the comparison result between the currently generated content and the preset content, combined with the reward function. The internal parameters of the multimodal large model are iteratively adjusted based on the reward feedback until the iteration termination condition is reached, thus obtaining a pre-trained multimodal large model.

[0023] Optionally, before inputting the persona keywords and scene parameters into the pre-trained multimodal large model and combining it with the multimodal large model's own multimodal knowledge system to output a question bank that meets user needs, the method further includes:

[0024] The cue generation sub-model in the multimodal large model is used to monitor changes in the knowledge domain in real time;

[0025] When the knowledge domain is updated, the new multimodal data corresponding to the updated knowledge is integrated into the multimodal knowledge system of the clue generation sub-model to complete the real-time update of the multimodal knowledge system.

[0026] Using the updated multimodal knowledge system, the clue generation rules of the clue generation sub-model are automatically updated to obtain the updated multimodal large model.

[0027] Optionally, the pre-training process of the clue generation sub-model includes:

[0028] Obtain a dataset containing the question content, answer options, and the multimodal data;

[0029] Extract knowledge features associated with the question content and answer options from the multimodal data;

[0030] And from the relevant multimodal data, obtain a multi-level cue information network for the answer options;

[0031] The question content, answer options, knowledge features, and multi-level clue information network are labeled in the dataset to obtain a labeled dataset.

[0032] The question content, answer options, and related knowledge features are used as inputs to the initial clue generation sub-model, and the multi-level clue information network corresponding to the answer options is used as the output. The model is trained by combining the labeled dataset to obtain the pre-trained clue generation sub-model.

[0033] Optionally, the scenario parameters include the difficulty of the question bank, the number of questions, and the distribution of question types;

[0034] The process involves inputting the persona keywords and scene parameters into a pre-trained multimodal large model, and combining this with the multimodal large model's own multimodal knowledge system to output a question bank that meets the user's needs, including:

[0035] The character keywords and scene parameters are input into the pre-trained multimodal large model. The multimodal large model then uses these keywords to perform knowledge retrieval and extraction within its own multimodal knowledge system to obtain the associated knowledge information of the character keywords.

[0036] Using the question generation module of the multimodal large model, corresponding question content is generated based on the associated knowledge information, character keywords, question bank difficulty, number of questions and question type distribution;

[0037] Using the answer option construction module of the multimodal large model, corresponding answer options are generated based on the associated knowledge information, character keywords and question content;

[0038] Using the clue generation sub-model of the multimodal large model, based on the associated knowledge information, character keywords, question content and answer options, the clue information corresponding to the answer options is generated;

[0039] The question bank, which meets the user's needs, is constructed based on the question content, answer options, and corresponding clues.

[0040] Based on the same inventive concept, this invention proposes a question bank generation system based on a pre-trained multimodal large model, comprising:

[0041] The data acquisition module is used to acquire the persona keywords and scene parameters set in the user's requirements;

[0042] The question bank generation module is used to input the character keywords and scene parameters into the pre-trained multimodal large model, and combine the multimodal large model's own multimodal knowledge system to output a question bank that meets the user's needs.

[0043] The multimodal large model is obtained by pre-training based on multimodal data from different domains, combined with reinforcement learning tasks and corresponding reward functions that are unified and integrated from multiple tasks.

[0044] Optionally, the system further includes a model training module for:

[0045] Collect multimodal data from different knowledge domains;

[0046] Based on the multimodal data, combined with the adaptive mechanism and self-supervised learning task of the multimodal large model, the multimodal large model is trained to obtain the data generated during the training process of the multimodal large model and the multimodal large model that has completed adaptive self-supervised training.

[0047] The multiple task components in the multimodal large model are integrated into a unified reinforcement learning task;

[0048] The multimodal data and the data generated during the training process of the multimodal large model are used as reinforcement training data;

[0049] The reward function designed based on the objectives and evaluation indicators of each task stage is combined with the reinforcement training data to perform reinforcement learning training on the multimodal large model that has completed adaptive self-supervised training, thereby obtaining a pre-trained multimodal large model.

[0050] The multiple task stages include: character keyword extraction, question generation, answer option construction, and clue generation.

[0051] Optionally, the model training module is specifically used for:

[0052] Based on the multimodal data, a general feature training is performed on the multimodal large model to obtain a multimodal large model that has completed the general feature training;

[0053] By utilizing the adaptive adjustment mechanism in the multimodal large model, the weights and parameters of the multimodal large model that has completed general feature training are adjusted based on the persona keywords in the multimodal data. At the same time, by utilizing the self-supervised learning task preset in the multimodal large model, the inherent laws of the multimodal data are learned, and the multimodal large model that has completed adaptive self-supervised training is obtained.

[0054] Optionally, the model training module is specifically used for:

[0055] Based on the multimodal data, a general feature training is performed on the multimodal large model to obtain a multimodal large model that has completed the general feature training;

[0056] By utilizing the adaptive adjustment mechanism in the multimodal large model, the weights and parameters of the multimodal large model that has completed general feature training are adjusted based on the persona keywords in the multimodal data. At the same time, by utilizing the self-supervised learning task preset in the multimodal large model, the inherent laws of the multimodal data are learned, and the multimodal large model that has completed adaptive self-supervised training is obtained.

[0057] Optionally, the model training module is specifically used for:

[0058] Set objectives and evaluation metrics for each of the aforementioned task stages;

[0059] The reward function for each task stage is determined based on the stated objectives and evaluation metrics.

[0060] The reinforcement training data is input into the multimodal large model that has completed adaptive self-supervised training. The multimodal large model generates reward feedback based on the comparison result between the currently generated content and the preset content, combined with the reward function. The internal parameters of the multimodal large model are iteratively adjusted based on the reward feedback until the iteration termination condition is reached, thus obtaining a pre-trained multimodal large model.

[0061] Optionally, the system further includes a model update module for:

[0062] The cue generation sub-model in the multimodal large model is used to monitor changes in the knowledge domain in real time;

[0063] When the knowledge domain is updated, the new multimodal data corresponding to the updated knowledge is integrated into the multimodal knowledge system of the clue generation sub-model to complete the real-time update of the multimodal knowledge system.

[0064] Using the updated multimodal knowledge system, the clue generation rules of the clue generation sub-model are automatically updated to obtain the updated multimodal large model.

[0065] Optionally, the system further includes a clue generation sub-model training module, used for:

[0066] Obtain a dataset containing the question content, answer options, and the multimodal data;

[0067] Extract knowledge features associated with the question content and answer options from the multimodal data;

[0068] And from the relevant multimodal data, obtain a multi-level cue information network for the answer options;

[0069] The question content, answer options, knowledge features, and multi-level clue information network are labeled in the dataset to obtain a labeled dataset.

[0070] The question content, answer options, and related knowledge features are used as inputs to the initial clue generation sub-model, and the multi-level clue information network corresponding to the answer options is used as the output. The model is trained by combining the labeled dataset to obtain the pre-trained clue generation sub-model.

[0071] Optionally, the scenario parameters include the difficulty of the question bank, the number of questions, and the distribution of question types;

[0072] The question bank generation module is specifically used for:

[0073] The character keywords and scene parameters are input into the pre-trained multimodal large model. The multimodal large model then uses these keywords to perform knowledge retrieval and extraction within its own multimodal knowledge system to obtain the associated knowledge information of the character keywords.

[0074] Using the question generation module of the multimodal large model, corresponding question content is generated based on the associated knowledge information, character keywords, question bank difficulty, number of questions and question type distribution;

[0075] Using the answer option construction module of the multimodal large model, corresponding answer options are generated based on the associated knowledge information, character keywords and question content;

[0076] Using the clue generation sub-model of the multimodal large model, based on the associated knowledge information, character keywords, question content and answer options, the clue information corresponding to the answer options is generated;

[0077] The question bank, which meets the user's needs, is constructed based on the question content, answer options, and corresponding clues.

[0078] In another aspect, this application also provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus;

[0079] The memory is used to store one or more programs;

[0080] When the one or more programs are executed by the at least one processor, the question bank generation method based on the pre-trained multimodal large model as described above is implemented.

[0081] Furthermore, this application also provides a computer-readable storage medium having an executable program stored thereon, which, when executed, implements the question bank generation method based on a pre-trained multimodal large model as described above.

[0082] Compared with the closest existing technology, the present invention has the following beneficial effects:

[0083] The present invention provides a question bank generation method and system based on a pre-trained multimodal large model, comprising: acquiring persona keywords and scene parameters set in user requirements; inputting the persona keywords and scene parameters into a pre-trained multimodal large model, and combining the multimodal knowledge system of the multimodal large model itself to output a question bank that meets the user requirements; the multimodal large model is pre-trained based on multimodal data from different fields, combined with reinforcement learning tasks and corresponding reward functions after unified integration of multiple task stages; the multimodal large model in the present invention can simultaneously process multimodal information such as text, images, and audio, effectively improving the model's generalization ability; through the integration of multiple task stages, each stage can be deeply integrated and collaboratively reinforced, significantly reducing the training time due to single training and generation and re-integration of multiple data, accurately understanding and efficiently completing complex question bank generation tasks; by introducing a multimodal knowledge system, the model can quickly transfer learning based on the relationships within it, automatically expanding new knowledge, and improving the adaptability and universality of question bank generation. Attached Figure Description

[0084] Figure 1 This is a flowchart illustrating the question bank generation method based on a pre-trained multimodal large model provided by the present invention.

[0085] Figure 2 A schematic diagram of the structure of the question bank generation system based on a pre-trained multimodal large model provided by the present invention;

[0086] Figure 3 This is a schematic diagram of the structure of an electronic device provided by the present invention. Detailed Implementation

[0087] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0088] Example 1

[0089] The present invention provides a method for generating a question bank based on a pre-trained multimodal large model, such as... Figure 1 As shown, it includes:

[0090] S1. Obtain the persona keywords and scene parameters set in the user's requirements;

[0091] S2. Input the persona keywords and scene parameters into the pre-trained multimodal large model, and combine the multimodal large model's own multimodal knowledge system to output a question bank that meets the user's needs;

[0092] The multimodal large model is obtained by pre-training based on multimodal data from different domains, combined with reinforcement learning tasks and corresponding reward functions that are unified and integrated from multiple tasks.

[0093] The pre-training process of the multimodal large model is as follows:

[0094] Step a: Collect multimodal data from different knowledge domains.

[0095] To cater to different knowledge domains and character design needs, we extensively collect data resources in various modalities, including text, images, and audio. For example, we collect textual materials related to subject knowledge from academic literature databases, online knowledge bases, and educational resource platforms; we obtain corresponding images, charts, and other image resources from image databases and multimedia teaching materials; and we acquire audio data from channels such as voice explanations and experimental audio recordings. These data from different sources are then categorized and organized according to knowledge points, character design keywords, etc., and stored in a dedicated data repository.

[0096] The collected multimodal data is cleaned to remove noisy, duplicate, and incomplete data fragments. For text data, grammatical error correction and vocabulary standardization are performed; for image data, image defects are repaired, image resolution and color balance are adjusted; for audio data, noise is eliminated, volume is adjusted, and audio format is standardized. Simultaneously, based on the data content and its knowledge category, keywords, question types, and answer categories are labeled to ensure that subsequent models can accurately identify and utilize this data for training and generation.

[0097] At this point, we have obtained the preprocessed multimodal data.

[0098] Step b: Design a large-scale model architecture capable of fusing multimodal data. This architecture includes multiple sub-modules specifically designed to process different modalities of data. For example, the text processing module employs advanced natural language processing neural network structures, such as a variant of the Transformer (a neural network architecture based on a self-attention mechanism); the image processing module uses a structure combining a Convolutional Neural Network (CNN) and a Visual Transformer; and the audio processing module utilizes a design that fuses an audio feature extraction network with a Recurrent Neural Network (RNN) or its variants. The text processing, image processing, and audio processing modules in this large-scale model architecture fuse information through specific interaction layers, enabling the model to simultaneously understand and process the relationships between multiple modalities of data, resulting in a multimodal large-scale model.

[0099] Step c: Based on the multimodal data, and combining the adaptive mechanism and self-supervised learning task of the multimodal large model, train the multimodal large model to obtain the data generated during the training process of the multimodal large model and the multimodal large model that has completed adaptive self-supervised training.

[0100] Specifically, in step c-1, the multimodal large model is trained with general features based on the multimodal data, such as semantic relationships in text, visual features in images, and acoustic patterns in audio, to obtain a multimodal large model that has completed the training of general features.

[0101] Step c-2: Using the adaptive adjustment mechanism in the multimodal large model, the weights and parameters of the multimodal large model that has completed general feature training are adjusted based on the persona keywords in the multimodal data; at the same time, the self-supervised learning task preset in the multimodal large model is used to learn the inherent rules of the multimodal data, so as to obtain the multimodal large model that has completed adaptive self-supervised training.

[0102] The persona keywords also include application scenario information. During the adaptive adjustment process, when the persona keyword is "mathematics teacher," the model strengthens the network weights for mathematical formula analysis and mathematical concept understanding in the text processing module, and adjusts the information fusion strategy with the image and audio modules to focus more on multimodal information processing related to mathematics teaching. The self-supervised learning tasks in the self-supervised learning process include text masking language modeling, image reconstruction, and audio signal prediction, enabling the multimodal large model to automatically learn the inherent patterns of the data, improving the model's generalization ability and understanding of different modalities.

[0103] This solution combines a large-scale model with multimodal data processing techniques for question generation. Unlike traditional methods that rely solely on a single text modality or simple generative models, this large-scale model can simultaneously process multimodal information such as text, images, and audio (for example, in a question bank involving literary works, it can combine related images and audio recordings to assist in question generation). Based on this, adaptive and self-supervised optimization is performed. By dynamically adjusting the model architecture and parameters during training, it can flexibly adapt to different character keywords and application scenarios, effectively overcoming the complexity of model training and optimization in traditional techniques when dealing with complex needs, and significantly improving model training efficiency and generation results.

[0104] Step d involves integrating multiple task components in the multimodal large model into a unified reinforcement learning task. These multiple task components include: character keyword extraction, question generation, answer option construction, and clue generation.

[0105] Step e: Use the above-mentioned multimodal data and the data generated during the training process of the multimodal large model as reinforcement training data;

[0106] Step f involves designing a reward function based on the objective and evaluation metrics of each task stage, and then using the reinforcement training data to perform reinforcement learning training on the multimodal large model that has completed adaptive self-supervised training, thereby obtaining a pre-trained multimodal large model. Specifically, this includes:

[0107] Step f-1: Set the objectives and evaluation indicators for each of the aforementioned task stages.

[0108] For example, the goal of question generation is to generate questions that match the character's keywords, are of moderate difficulty, and have good discriminatory power. Evaluation indicators include the relevance score between the question and the keywords and the difficulty score.

[0109] The objectives and evaluation metrics for each task stage in this embodiment are as follows:

[0110] I. Character Keyword Extraction Task

[0111] Objective: To accurately capture the core concepts, key knowledge points, and important themes from text materials or knowledge modules, and extract a small number of representative words or phrases that best summarize the core content as keywords for the questions. These keywords must be unique and comprehensive enough to cover key information and facilitate subsequent tasks to revolve around them, ensuring that the generated content closely aligns with the key knowledge points.

[0112] Evaluation metrics include keyword coverage, relevance score, and redundancy. Keyword coverage: This measures the percentage of extracted keywords that cover the core content of the corresponding text. Ideally, it should reach 80% or higher to measure whether key information has been captured and to avoid missing important knowledge points. Relevance score: This uses semantic analysis to compare the semantic relevance between extracted keywords and the original text. The value ranges from 0 to 1, with values ​​closer to 1 indicating stronger relevance. The average relevance score should be no less than 0.7 to ensure the extracted content is closely connected. Redundancy: This calculates the semantic overlap between keywords. Lower redundancy is better, generally aiming to be below 20% to prevent repetition and redundant expressions and to streamline key information.

[0113] II. Question Generation Task Phase

[0114] Objective: Based on the extracted keywords of the persona, cleverly conceive and produce questions that are logically clear, accurately expressed, and meet the requirements of teaching or testing scenarios. The difficulty level should match the level of the target audience, while also having good differentiation among candidates of different knowledge levels, and accurately testing their mastery of the corresponding knowledge points.

[0115] Evaluation indicators include the relevance score between the question and keywords, the difficulty score, and the discrimination index. The relevance score between the question and keywords uses a semantic matching algorithm to quantify the semantic fit between the question text and the assigned keywords. A maximum score of 10 points is given, with 8 points or higher considered excellent, ensuring the question closely adheres to the core concept and does not deviate from the intended direction. The difficulty score combines big data analysis of students' answer accuracy rates with expert evaluation, categorizing questions into three levels: easy (1-3 points), medium (4-7 points), and difficult (8-10 points). The error from the preset target difficulty should not exceed ±1 point, ensuring appropriate difficulty. The discrimination index (e.g., the difference in score rates between high and low groups) is calculated by analyzing the answers of students at different ability levels. The ideal value is in the range of 0.3-0.7; the higher the value, the more effectively it can identify differences in students' knowledge mastery.

[0116] III. Answer Option Construction Task

[0117] Objective: To create a set of answer options for the generated questions, with reasonable distractors, moderate deceptiveness, and a clear correct answer. Each option should fit the context and knowledge scope of the question, comprehensively testing students' ability to distinguish between details and easily confused points of knowledge.

[0118] Evaluation indicators include correctness, deceptiveness score, and option relevance. Correctness: Ensure that only one option fully matches the correct answer standard; multiple correct options or no correct answer are not allowed; the accuracy rate must reach 100%. Deceptiveness score: Analyze the degree to which distractors mislead lower-scoring students; the deceptiveness should not be too low (making it easily identifiable) nor too high (making it difficult to distinguish from the correct answer). The value ranges from 3 to 7 points, determined by sampling evaluation by educational experts. Option relevance: The average score of the relevance between each option and the knowledge points involved in the question and the semantic meaning of the question text should not be lower than 0.6, ensuring that they revolve around the same theme and do not deviate from the scope of the examination.

[0119] Step f-2: Determine the reward function for each task stage based on the stated objective and evaluation metrics.

[0120] Based on the aforementioned objectives and evaluation metrics, a reward function is designed for each task stage. A positive reward is given when the model generates results that meet expectations at a particular task stage, and a negative reward is given otherwise. For example, when the generated question is evaluated as highly relevant to the character's keywords and of appropriate difficulty, the model receives a higher positive reward, thus incentivizing the model to continuously optimize its generation strategy.

[0121] Step f-3: Input the reinforcement training data into the multimodal large model that has completed adaptive self-supervised training. Use the multimodal large model to generate reward feedback based on the comparison result between the currently generated content and the preset content, combined with the reward function. Iteratively adjust the internal parameters of the multimodal large model based on the reward feedback until the iteration termination condition is reached, and obtain the pre-trained multimodal large model.

[0122] Specifically, during reinforcement training, the multimodal large model selects its next action (such as generating new question content, adjusting answer options, or supplementing clue information) based on its current state (including generated character keywords, questions, answer options, and clue information), and receives corresponding reward feedback based on the reward function. Through continuous iterative training, the model gradually learns the optimal generation strategy, enabling collaborative optimization of each task stage. For example, in the early stages of training, the questions and answers generated by the model may not be logically coherent, but with continuous iterative training, the model continuously adjusts its generation strategy based on reward feedback, gradually learning to generate logically rigorous and high-quality questions, answer options, and clue information. This achieves efficient and accurate generation from character keywords to a complete question bank, until the preset iteration termination condition is met, resulting in a pre-trained multimodal large model.

[0123] Thus, a multimodal large model for generating a question bank that meets user needs has been obtained.

[0124] This solution employs an integrated end-to-end reinforcement learning training framework for question bank generation. Unlike traditional data-trained generative model methods, this framework deeply integrates and collaboratively reinforces multiple stages, including character keyword extraction, question generation, answer option construction, and clue information generation. By introducing reinforcement learning algorithms, the model can continuously self-evaluate and optimize its generation strategy during training, enabling each stage to mutually promote and synergistically improve. This allows for accurate understanding and efficient completion of complex question bank generation tasks, significantly enhancing the quality and efficiency of the entire generation process and ensuring a more accurate, logical, and adaptable question bank.

[0125] Optionally, before generating the question bank using the multimodal large model, the multimodal large model needs to be automatically updated using a designed dynamic expansion strategy. Specifically:

[0126] The cue generation sub-model in the multimodal large model is used to monitor changes in the knowledge domain and the inflow of new data in real time.

[0127] When the knowledge domain is updated, the new multimodal data corresponding to the updated knowledge is integrated into the multimodal knowledge system (including knowledge graph) of the clue generation sub-model to complete the real-time update of the multimodal knowledge system; the knowledge update includes new knowledge hotspots, concept updates, or the addition of new multimodal data;

[0128] Using the updated multimodal knowledge system, the clue generation rules of the clue generation sub-model are automatically updated to obtain the updated multimodal large model.

[0129] For example, when new archaeological discoveries are announced, the model can quickly integrate the new archaeological information into the process of generating clues for history-related questions, update the network of question clues related to ancient civilizations, provide learners with the latest knowledge-related clues, and keep the question bank content timely and cutting-edge.

[0130] The pre-training process of the clue generation sub-model, specifically designed to generate answer option clues and establish clue associations within the multimodal large model, includes:

[0131] Obtain a dataset containing the question content, answer options, and the multimodal data;

[0132] Extract knowledge features associated with the question content and answer options from the multimodal data;

[0133] And from the relevant multimodal data, obtain a multi-level cue information network for the answer options;

[0134] The question content, answer options, knowledge features, and multi-level clue information network are labeled in the dataset to obtain a labeled dataset.

[0135] The question content, answer options, and related knowledge features are used as inputs to the initial clue generation sub-model, and the multi-level clue information network corresponding to the answer options is used as the output. The model is trained by combining the labeled dataset to obtain the pre-trained clue generation sub-model.

[0136] The clue generation sub-model employs an attention-based neural network structure. This model focuses on key information related to the quality control of questions and answers, generating clues for each question and answer option. For example, when generating clues for a historical event question, the model mines relevant background information, accounts of related figures, and images or audio recordings of the social landscape of the time from multimodal data, based on key elements such as the time, location, and people involved in the event, constructing a multi-layered clue network.

[0137] The clue generation sub-model in this solution provides an intelligent clue association and dynamic expansion mechanism. Compared to traditional generative models that can only generate relatively fixed and limited clue information, the model in this solution can intelligently construct a multi-layered, multi-angled, and dynamically expandable clue information network based on the question content, answer options, and real-time knowledge updates. For example, when encountering new knowledge hotspots or domain expansions, the model can automatically discover new clue associations and integrate them into the question bank generation process, without the need for large-scale retraining as in traditional techniques. This effectively solves the limitations of the generalization ability of traditional models, greatly enhances the logic, richness, and timeliness of the question bank, and significantly improves learning outcomes.

[0138] In step S2, the persona keywords and scene parameters are input into the pre-trained multimodal large model, and combined with the multimodal knowledge system of the multimodal large model, a question bank that meets the user's needs is output.

[0139] Specifically, the persona keywords and scene parameters are input into the pre-trained multimodal large model. The multimodal large model then uses these keywords to perform knowledge retrieval and extraction within its own multimodal knowledge system, obtaining the associated knowledge information of the persona keywords. The scene parameters include the difficulty level of the question bank (beginner, intermediate, advanced), the number of questions, and the distribution of question types (multiple choice, fill-in-the-blank, short answer, etc.).

[0140] Using the question generation module of the multimodal large model, corresponding question content is generated based on the associated knowledge information, character keywords, question bank difficulty, number of questions and question type distribution;

[0141] Using the answer option construction module of the multimodal large model, corresponding answer options are generated based on the associated knowledge information, character keywords and question content;

[0142] Using the clue generation sub-model of the multimodal large model, based on the associated knowledge information, character keywords, question content and answer options, the clue information corresponding to the answer options is generated;

[0143] The question bank, which meets the user's needs, is constructed based on the question content, answer options, and corresponding clues.

[0144] During the question bank generation process, the model will use the optimal strategy obtained from integrated end-to-end reinforcement learning training to ensure that the generated question bank content is of high quality, logically rigorous, and meets the user-defined scenario parameter requirements.

[0145] The generated question bank is output in a standardized format (such as JSON or database record format), including detailed information such as questions, answer options, clues, question types, and difficulty levels. This question bank content can be directly exported to educational platforms, examination systems, or local storage devices for use in various application scenarios, including teacher instruction, student learning, and exam question creation by examination institutions. Furthermore, the exported question bank content can be visualized, allowing users to preview and evaluate its quality. Users can also further edit and adjust the generated question bank if needed.

[0146] This solution leverages the multimodal fusion and adaptive optimization features of a large model to handle various data types and flexibly adapt to different personas and scenario requirements. Compared to traditional techniques, it significantly reduces training time caused by single data types and rigid models, greatly improving the efficiency of question bank generation. For example, in cross-disciplinary question bank construction, it can quickly integrate textual knowledge with relevant charts, experimental audio, and other information to generate questions, whereas traditional methods require processing different modalities of data separately before integration, resulting in low efficiency. Furthermore, by introducing multi-domain knowledge graph fusion technology, this solution constructs a semantic network covering a wide range of knowledge domains. The multimodal large model can quickly transfer learning based on the relationships in the graph, automatically expanding its understanding and generation capabilities for new knowledge and different question types. This effectively overcomes the shortcomings of existing techniques in model generalization ability. When facing various changes in knowledge domains and different question requirements, it can run stably without frequent retraining, significantly improving the adaptability and versatility of question bank generation.

[0147] By employing a pre-trained multimodal large model combined with domain-specific adaptive fine-tuning, the model extracts and trains only key knowledge points and general clue features, eliminating the need for complex association construction and multi-stage processing of character settings and answer clues. This effectively avoids the complexity of training and optimization caused by the complexity and relevance of the generated content.

[0148] Furthermore, the integrated end-to-end reinforcement learning training framework enables collaborative optimization of each stage, avoiding information transmission loss and redundant calculations caused by independent training of each step in traditional methods. This further improves training speed and generation quality, enabling the question bank to quickly respond to the needs of various application scenarios. Whether it is daily teaching, professional examinations, or question bank updates for online learning platforms, it can be completed efficiently.

[0149] Example 2

[0150] Based on the same inventive concept, this invention also provides a question bank generation system based on a pre-trained multimodal large model, such as... Figure 2 As shown, it includes:

[0151] The data acquisition module is used to acquire the persona keywords and scene parameters set in the user's requirements;

[0152] The question bank generation module is used to input the character keywords and scene parameters into the pre-trained multimodal large model, and combine the multimodal large model's own multimodal knowledge system to output a question bank that meets the user's needs.

[0153] The multimodal large model is obtained by pre-training based on multimodal data from different domains, combined with reinforcement learning tasks and corresponding reward functions that are unified and integrated from multiple tasks.

[0154] In one possible implementation, the system further includes a model training module for:

[0155] Collect multimodal data from different knowledge domains;

[0156] Based on the multimodal data, combined with the adaptive mechanism and self-supervised learning task of the multimodal large model, the multimodal large model is trained to obtain the data generated during the training process of the multimodal large model and the multimodal large model that has completed adaptive self-supervised training.

[0157] The multiple task components in the multimodal large model are integrated into a unified reinforcement learning task;

[0158] The multimodal data and the data generated during the training process of the multimodal large model are used as reinforcement training data;

[0159] The reward function designed based on the objectives and evaluation indicators of each task stage is combined with the reinforcement training data to perform reinforcement learning training on the multimodal large model that has completed adaptive self-supervised training, thereby obtaining a pre-trained multimodal large model.

[0160] The multiple task stages include: character keyword extraction, question generation, answer option construction, and clue generation.

[0161] In one possible implementation, the above model training module is specifically used for:

[0162] Based on the multimodal data, a general feature training is performed on the multimodal large model to obtain a multimodal large model that has completed the general feature training;

[0163] By utilizing the adaptive adjustment mechanism in the multimodal large model, the weights and parameters of the multimodal large model that has completed general feature training are adjusted based on the persona keywords in the multimodal data. At the same time, by utilizing the self-supervised learning task preset in the multimodal large model, the inherent laws of the multimodal data are learned, and the multimodal large model that has completed adaptive self-supervised training is obtained.

[0164] In one possible implementation, the above model training module is specifically used for:

[0165] Based on the multimodal data, a general feature training is performed on the multimodal large model to obtain a multimodal large model that has completed the general feature training;

[0166] By utilizing the adaptive adjustment mechanism in the multimodal large model, the weights and parameters of the multimodal large model that has completed general feature training are adjusted based on the persona keywords in the multimodal data. At the same time, by utilizing the self-supervised learning task preset in the multimodal large model, the inherent laws of the multimodal data are learned, and the multimodal large model that has completed adaptive self-supervised training is obtained.

[0167] In one possible implementation, the above model training module is specifically used for:

[0168] Set objectives and evaluation metrics for each of the aforementioned task stages;

[0169] The reward function for each task stage is determined based on the stated objectives and evaluation metrics.

[0170] The reinforcement training data is input into the multimodal large model that has completed adaptive self-supervised training. The multimodal large model generates reward feedback based on the comparison result between the currently generated content and the preset content, combined with the reward function. The internal parameters of the multimodal large model are iteratively adjusted based on the reward feedback until the iteration termination condition is reached, thus obtaining a pre-trained multimodal large model.

[0171] In one possible implementation, the system further includes a model update module for:

[0172] The cue generation sub-model in the multimodal large model is used to monitor changes in the knowledge domain in real time;

[0173] When the knowledge domain is updated, the new multimodal data corresponding to the updated knowledge is integrated into the multimodal knowledge system of the clue generation sub-model to complete the real-time update of the multimodal knowledge system.

[0174] Using the updated multimodal knowledge system, the clue generation rules of the clue generation sub-model are automatically updated to obtain the updated multimodal large model.

[0175] In one possible implementation, the system further includes a clue generation sub-model training module, used for:

[0176] Obtain a dataset containing the question content, answer options, and the multimodal data;

[0177] Extract knowledge features associated with the question content and answer options from the multimodal data;

[0178] And from the relevant multimodal data, obtain a multi-level cue information network for the answer options;

[0179] The question content, answer options, knowledge features, and multi-level clue information network are labeled in the dataset to obtain a labeled dataset.

[0180] The question content, answer options, and related knowledge features are used as inputs to the initial clue generation sub-model, and the multi-level clue information network corresponding to the answer options is used as the output. The model is trained by combining the labeled dataset to obtain the pre-trained clue generation sub-model.

[0181] In one possible implementation, the aforementioned scenario parameters include the difficulty of the question bank, the number of questions, and the distribution of question types; the question bank generation module is specifically used for:

[0182] The character keywords and scene parameters are input into the pre-trained multimodal large model. The multimodal large model then uses these keywords to perform knowledge retrieval and extraction within its own multimodal knowledge system to obtain the associated knowledge information of the character keywords.

[0183] Using the question generation module of the multimodal large model, corresponding question content is generated based on the associated knowledge information, character keywords, question bank difficulty, number of questions and question type distribution;

[0184] Using the answer option construction module of the multimodal large model, corresponding answer options are generated based on the associated knowledge information, character keywords and question content;

[0185] Using the clue generation sub-model of the multimodal large model, based on the associated knowledge information, character keywords, question content and answer options, the clue information corresponding to the answer options is generated;

[0186] The question bank, which meets the user's needs, is constructed based on the question content, answer options, and corresponding clues.

[0187] Example 3

[0188] like Figure 3 As shown, the present invention also provides an electronic device, which may be a computer device, a microcontroller device, a smart mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, processor, and transceiver component are connected via a bus; the memory can be used to store executable programs, and an exemplary executable program may include instructions; the processor is used to execute the instructions stored in the memory. The memory can also be used to store data, which can be accessed and / or modified when instructions are executed.

[0189] The processor may be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), or application-specific integrated circuits (ASICs).

[0190] Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., are the computing core and control core of the terminal. They are suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the storage medium to realize the corresponding method flow or corresponding function, so as to realize the steps of the question bank generation method based on the pre-trained multimodal large model in the above embodiments.

[0191] Example 4

[0192] Based on the same inventive concept, this invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory). This readable storage medium is a memory device within an electronic device used to store programs and data. It is understood that the storage medium here can include both built-in storage media within the electronic device and extended storage media supported by the electronic device. The storage medium provides storage space, which stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more executable programs (including program code). It should be noted that the storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. Loading and executing one or more instructions stored in the storage medium by the processor can implement the steps of the question bank generation method based on a pre-trained multimodal large model in the above embodiments.

[0193] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0194] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0195] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0196] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit its scope of protection. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that after reading the present invention, they can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the scope of protection of the claims pending approval.

Claims

1. A method for generating a question bank based on a pre-trained multimodal large model, characterized in that, include: Obtain the persona keywords and scene parameters set in the user requirements; The persona keywords and scene parameters are input into the pre-trained multimodal large model, and combined with the multimodal large model's own multimodal knowledge system, a question bank that meets the user's needs is output. The multimodal large model is obtained by pre-training based on multimodal data from different domains, combined with reinforcement learning tasks and corresponding reward functions that are unified and integrated from multiple tasks.

2. The method as described in claim 1, characterized in that, The pre-training process of the multimodal large model includes: Collect multimodal data from different knowledge domains; Based on the multimodal data, combined with the adaptive mechanism and self-supervised learning task of the multimodal large model, the multimodal large model is trained to obtain the data generated during the training process of the multimodal large model and the multimodal large model that has completed adaptive self-supervised training. The multiple task components in the multimodal large model are integrated into a unified reinforcement learning task; The multimodal data and the data generated during the training process of the multimodal large model are used as reinforcement training data; The reward function designed based on the objectives and evaluation indicators of each task stage is combined with the reinforcement training data to perform reinforcement learning training on the multimodal large model that has completed adaptive self-supervised training, thereby obtaining a pre-trained multimodal large model. The multiple task stages include: character keyword extraction, question generation, answer option construction, and clue generation.

3. The method as described in claim 2, characterized in that, The process of training the multimodal large model based on the multimodal data, combined with the adaptive mechanism and self-supervised learning task of the multimodal large model, to obtain the multimodal large model that has completed adaptive and self-supervised training includes: Based on the multimodal data, a general feature training is performed on the multimodal large model to obtain a multimodal large model that has completed the general feature training; By utilizing the adaptive adjustment mechanism in the multimodal large model, the weights and parameters of the multimodal large model that has completed general feature training are adjusted based on the persona keywords in the multimodal data. At the same time, by utilizing the self-supervised learning task preset in the multimodal large model, the inherent laws of the multimodal data are learned, and the multimodal large model that has completed adaptive self-supervised training is obtained.

4. The method as described in claim 2, characterized in that, The reward function designed based on the objectives and evaluation metrics of each task stage, combined with reinforcement training data, is used to perform reinforcement learning training on the multimodal large model that has completed adaptive self-supervised training, resulting in a pre-trained multimodal large model, including: Set objectives and evaluation metrics for each of the aforementioned task stages; The reward function for each task stage is determined based on the stated objectives and evaluation metrics. The enhanced training data is input into the multimodal large model that has completed adaptive self-supervised training. The multimodal large model generates reward feedback based on the comparison result between the currently generated content and the preset content, combined with the reward function. The internal parameters of the multimodal large model are iteratively adjusted based on the reward feedback until the iteration termination condition is reached, thus obtaining a pre-trained multimodal large model.

5. The method as described in claim 1, characterized in that, Before the pre-trained multimodal large model, which inputs the persona keywords and scene parameters, and combines them with its own multimodal knowledge system to output a question bank that meets user needs, the process also includes: The cue generation sub-model in the multimodal large model is used to monitor changes in the knowledge domain in real time; When the knowledge domain is updated, the new multimodal data corresponding to the updated knowledge is integrated into the multimodal knowledge system of the clue generation sub-model to complete the real-time update of the multimodal knowledge system. Using the updated multimodal knowledge system, the clue generation rules of the clue generation sub-model are automatically updated to obtain the updated multimodal large model.

6. The method as described in claim 5, characterized in that, The pre-training process of the clue generation sub-model includes: Obtain a dataset containing the question content, answer options, and the multimodal data; Extract knowledge features associated with the question content and answer options from the multimodal data; And from the relevant multimodal data, obtain a multi-level cue information network for the answer options; The question content, answer options, knowledge features, and multi-level clue information network are labeled in the dataset to obtain a labeled dataset. The question content, answer options, and related knowledge features are used as inputs to the initial clue generation sub-model, and the multi-level clue information network corresponding to the answer options is used as the output. The model is trained by combining the labeled dataset to obtain the pre-trained clue generation sub-model.

7. The method as described in claim 1, characterized in that, The scenario parameters include the difficulty of the question bank, the number of questions, and the distribution of question types; The process involves inputting the persona keywords and scene parameters into a pre-trained multimodal large model, and combining this with the multimodal large model's own multimodal knowledge system to output a question bank that meets the user's needs, including: The character keywords and scene parameters are input into the pre-trained multimodal large model. The multimodal large model then uses these keywords to perform knowledge retrieval and extraction within its own multimodal knowledge system to obtain the associated knowledge information of the character keywords. Using the question generation module of the multimodal large model, corresponding question content is generated based on the associated knowledge information, character keywords, question bank difficulty, number of questions and question type distribution; Using the answer option construction module of the multimodal large model, corresponding answer options are generated based on the associated knowledge information, character keywords and question content; Using the clue generation sub-model of the multimodal large model, based on the associated knowledge information, character keywords, question content and answer options, the clue information corresponding to the answer options is generated; The question bank, which meets the user's needs, is constructed based on the question content, answer options, and corresponding clues.

8. A question bank generation system based on a pre-trained multimodal large model, characterized in that, include: The data acquisition module is used to acquire the persona keywords and scene parameters set in the user's requirements; The question bank generation module is used to input the character keywords and scene parameters into the pre-trained multimodal large model, and combine the multimodal large model's own multimodal knowledge system to output a question bank that meets the user's needs. The multimodal large model is obtained by pre-training based on multimodal data from different domains, combined with reinforcement learning tasks and corresponding reward functions that are unified and integrated from multiple tasks.

9. The system as described in claim 8, characterized in that, It also includes a model training module, used for: Collect multimodal data from different knowledge domains; Based on the multimodal data, combined with the adaptive mechanism and self-supervised learning task of the multimodal large model, the multimodal large model is trained to obtain the data generated during the training process of the multimodal large model and the multimodal large model that has completed adaptive self-supervised training. The multiple task components in the multimodal large model are integrated into a unified reinforcement learning task; The multimodal data and the data generated during the training process of the multimodal large model are used as reinforcement training data; The reward function designed based on the objectives and evaluation indicators of each task stage is combined with the reinforcement training data to perform reinforcement learning training on the multimodal large model that has completed adaptive self-supervised training, thereby obtaining a pre-trained multimodal large model. The multiple task stages include: character keyword extraction, question generation, answer option construction, and clue generation.

10. The system as described in claim 9, characterized in that, The model training module is specifically used for: Based on the multimodal data, a general feature training is performed on the multimodal large model to obtain a multimodal large model that has completed the general feature training; By utilizing the adaptive adjustment mechanism in the multimodal large model, the weights and parameters of the multimodal large model that has completed general feature training are adjusted based on the persona keywords in the multimodal data. At the same time, by utilizing the self-supervised learning task preset in the multimodal large model, the inherent laws of the multimodal data are learned, and the multimodal large model that has completed adaptive self-supervised training is obtained.

Images