Question analysis and explanation generation method and system combined with multi-modal large model

Abstract

The invention provides a multi-modal large model combined question analysis and explanation generation method and system, and relates to the technical field of multi-modal learning, and the method comprises the steps: carrying out the standardization processing of a multi-modal input question, and obtaining the standardized question data; separating structured question information to perform deep feature extraction, driving three-level knowledge structure linkage matching of the question solving thought library, and generating a plurality of initial question solving paths; performing knowledge blind area coverage screening, and positioning a target problem solving path; recursive knowledge point reinforcement is carried out, and a non-blind area credible problem solving chain is generated; step-by-step analysis logic derivation is carried out, and a target analysis logic framework is output; and performing cross-modal directional rendering according to real-time explanation modal selection, and delivering and outputting a multi-modal analysis explanation stream. The technical problems that in the prior art, deep understanding of personalized requirements and knowledge mastering levels of users is lacked, customized answers cannot be provided for the users, and the learning effect of the users and tool practicability are affected are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-modal learning, in particular to a question analysis and explanation generation method and system combined with a multi-modal large model. BACKGROUND

[0002] In the prior art, whether the user is a beginner or an advanced learner, the answer content provided by the problem solving tool is mostly fixed, lacking depth understanding of the user's personalized needs and knowledge mastery level, so it cannot provide customized answers for users. For example, a junior high school student needs basic step and concept explanation, while a high school student needs higher level skill solution and easy-to-mistake point reminder. This leads to a fragmented experience for different users when using the problem solving tool, and cannot accurately meet the needs of different users, affecting the user's learning effect and tool practicality. SUMMARY

[0003] The present application provides a question analysis and explanation generation method and system combined with a multi-modal large model, aiming to solve the technical problem that the prior art lacks depth understanding of the user's personalized needs and knowledge mastery level, cannot provide customized answers for users, and affects the user's learning effect and tool practicality.

[0004] The first aspect of the present application provides a question analysis and explanation generation method combined with a multi-modal large model, the method comprising: standardizing a multi-modal input question to obtain standardized question data; separating structured question information from the standardized question data based on multi-modal semantic understanding, wherein the structured question information includes known conditions, target to be solved, and multi-dimensional exam point labels; performing deep feature extraction on the structured question information to drive three-level knowledge structure linkage matching of a problem solving idea library, generating multiple initial problem solving paths; performing knowledge blind area coverage screening of the multiple initial problem solving paths according to a user's academic profile, positioning a target problem solving path; performing recursive knowledge point reinforcement on a target knowledge point sequence of the target problem solving path, generating a blind area-free credible problem solving chain; defining the known conditions and the target to be solved as reasoning anchor points, performing step-by-step analysis logic deduction based on the blind area-free credible problem solving chain and the multi-dimensional exam point labels, and outputting a target analysis logic framework; performing cross-modal directional rendering of the target analysis logic framework according to real-time user's real-time explanation modal selection, and delivering a multi-modal analysis and explanation flow.

[0005] In a second aspect, a question analysis explanation generation system combined with a multi-modal large model is provided. The system is used for the question analysis explanation generation method combined with the multi-modal large model. The system includes: a standardization processing module configured to perform standardization processing on a multi-modal input question to obtain standardized question data; a question information separation module configured to separate structured question information from the standardized question data based on multi-modal semantic understanding, wherein the structured question information includes known conditions, a target to be solved, and multi-dimensional focus labels; a problem solving path matching module configured to perform deep feature extraction on the structured question information, drive linkage matching of a three-level knowledge structure of a problem solving idea library, and generate a plurality of initial problem solving paths; a coverage screening module configured to perform knowledge blind area coverage screening on the plurality of initial problem solving paths according to a user learning profile, and locate a target problem solving path; a knowledge point supplementing module configured to perform recursive knowledge point supplementing on a target knowledge point sequence of the target problem solving path, and generate a blind area-free reliable problem solving chain; a logic derivation module configured to define the known conditions and the target to be solved as reasoning anchor points, perform step-by-step analysis logic derivation based on the blind area-free reliable problem solving chain and the multi-dimensional focus labels, and output a target analysis logic framework; and a directional rendering module configured to perform cross-modal directional rendering of the target analysis logic framework according to real-time explanation modal selection of a real-time user, and deliver an output multi-modal analysis explanation flow.

[0006] One or more technical solutions provided in the present application have at least the following beneficial effects: By standardizing the multi-modal input questions, the differences between different input forms can be eliminated, the data format is unified, and clear and unified basic data is provided for subsequent analysis and understanding; based on multi-modal semantic understanding, structured information such as known conditions, target to be solved and multi-dimensional test point labels is separated from the standardized question data, which can ensure in-depth analysis of the question content and help accurate matching and deduction of the subsequent problem solving ideas; by matching multiple initial problem solving paths in the problem solving idea library, a number of possible problem solving paths can be selected according to the structured information of the question, providing multiple feasible problem solving strategies for complex questions, so that the problem solving ideas are more comprehensive and diverse; combined with the user's learning profile, the problem solving path with the smallest knowledge blind area coverage is selected according to the user's knowledge mastery, and the most suitable problem solving strategy is recommended for each user, helping the user to solve the problem more effectively; the knowledge blind area of the user is recursively reinforced to generate a blind area-free credible problem solving chain, enhancing the completeness and coherence of the problem solving process; by defining the known conditions and the target to be solved as reasoning anchor points, based on the blind area-free problem solving chain and the multi-dimensional test point label, a complete analysis logic framework is gradually deduced, helping the user to understand the specific logic and knowledge points of each link and improving the user's problem solving ability; according to the user's real-time needs and explanation modal selection, the target analysis logic framework is cross-modal and directional rendering, and a multi-modal analysis explanation flow is output, which not only improves the interactivity and personalized experience of learning, but also allows the user to choose the appropriate learning method according to their own preferences, improving the efficiency and effectiveness of learning.

[0007] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the present application can be implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 The question analysis and explanation generation method flowchart provided by the embodiments of the present application is combined with a multi-modal large model.

[0009] Figure 2 The question analysis and explanation generation system structure diagram provided by the embodiments of the present application is combined with a multi-modal large model.

[0010] Explanation of reference signs: standardization processing module 10, question information separation module 20, problem solving path matching module 30, coverage screening module 40, knowledge point reinforcement module 50, logic deduction module 60, directional rendering module 70. DETAILED DESCRIPTION

[0011] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0012] Example 1, as Figure 1 As shown in the embodiments of this application, a method for generating question analysis and explanations combining a multimodal large model is provided, the method including: The multimodal input questions are standardized to obtain standardized question data.

[0013] Multimodal input questions refer to questions that users can submit through various methods such as text input, photo upload, and voice input, including images of math problems containing formulas. Standardization processing is applied to different types of information, including: for text data, extracting variables, symbols, and operational steps into a standardized format, removing redundant content, and standardizing the expression; for image data, if the question contains images such as mathematical formulas or geometric figures, extracting relevant information using image recognition technology and converting the image content into text or numerical form; for voice input, first converting it into text using speech recognition technology, and then performing further processing. The result of standardization is to transform all types of data into a unified, structured form, facilitating subsequent analysis.

[0014] Based on multimodal semantic understanding, structured question information is separated from the standardized question data, wherein the structured question information includes known conditions, objectives to be found, and multi-dimensional test point tags.

[0015] Multimodal semantic understanding extracts key semantic information from standardized question data. In this step, natural language processing technology is used to perform semantic role labeling. This means that through syntactic analysis, the main semantic components in different sentences are identified. Known conditions refer to the known data or assumptions given in the question. The target to be found refers to the target problem that the question requires to solve or the unknowns to be solved. Multidimensional test point tags are the core knowledge points or skill points abstracted from the questions. Each tag represents a specific knowledge point or concept involved in the question.

[0016] Deep feature extraction is performed on the structured question information to drive the linkage and matching of the three-level knowledge structure of the problem-solving approach library, generating multiple initial problem-solving paths.

[0017] Traditional problem-solving approach libraries rely on manually designed rules and modular knowledge points, while in the large model-based method, deep feature extraction is automated, with large models reasoning and learning. Large models can adaptively understand and extract core features in problems, such as known conditions, target to be solved, related test point labels, and relationships between conditions, by analyzing multi-modal inputs (text, images, videos, etc.). Through deep learning and natural language processing techniques, the model can dynamically extract high-dimensional feature representations from structured problem information. These features are not limited to the problem itself, but can also include contextual information and potential implicit knowledge.

[0018] Specifically, the three-level knowledge structure linkage matching is as follows: in the L1 level knowledge module, the large model automatically locates the relevant domain knowledge modules from its trained knowledge base according to the multi-dimensional test point labels in the question. These modules are no longer fixed, but are dynamically selected based on the reasoning ability of the model; in the traditional method, the problem-solving method is selected based on rules, but under the driving of the large model, the L2 layer selects the solution type based on the model's understanding and reasoning of the question, which means that the model analyzes the known conditions and target to be solved in the question, and selects the most suitable solution type from the possible solutions, and can automatically adjust the number of selected solutions according to the complexity of the question; the large model extracts the confidence weighted templates from the L3 level example template library according to the N solution types and the question feature vector, in this process, the model calculates the weighted cosine similarity according to the relationship between the characteristics of the question and the solution type, and dynamically selects and optimizes the example templates according to these similarities. This selection process is pre-set in the traditional method, while the large model optimizes the accuracy and efficiency of template extraction through training data.

[0019] Overall, traditional problem-solving approach libraries and problem-solving methods rely on manually designed modular knowledge, while large model-based methods dynamically generate the most suitable problem-solving path through deep learning and adaptive reasoning capabilities, flexibly coping with a variety of problems and user needs. In this process, the model continuously adjusts the problem-solving path according to the user's learning progress and error conditions, making the problem-solving process more personalized and efficient.

[0020] According to the user's learning profile, the knowledge blind area coverage screening of the plurality of initial problem-solving paths is performed to locate the target problem-solving path.

[0021] The user learning profile is constructed based on historical learning records, error sets, mastered knowledge points, weak knowledge points, and other information, reflecting the user's knowledge mastery and ability level. Based on the user learning profile, the user's knowledge blind spots in the problem-solving process are identified, i.e., the knowledge points that the user is not familiar with or does not master well. Then, multiple initial problem-solving paths are screened to identify those that can effectively fill the user's knowledge blind spots, ensuring that the user can fill in the knowledge gaps through problem-solving. Finally, the most suitable target problem-solving path is selected, which not only meets the requirements of the problem, but also helps the user effectively fill in their knowledge blind spots and improve their problem-solving ability.

[0022] Recursive knowledge point reinforcement is performed on the target knowledge point sequence of the target problem-solving path to generate a blind-area-free credible problem-solving chain.

[0023] The target knowledge point sequence of the target problem-solving path is extracted, which is a series of knowledge points in this problem-solving path, representing the core concepts, formulas, problem-solving methods, etc. that need to be mastered in the problem-solving process. Recursive knowledge point reinforcement is used to supplement the user's lack of knowledge points in the problem-solving process, with the goal of ensuring that the user masters all the knowledge points required for problem-solving and fills in any blind spots. The recursive reinforcement method is as follows: when a knowledge point is detected as a weak link for the user, relevant basic knowledge points are found from the hierarchical knowledge point defect topology map to supplement them, allowing the user to understand and master the relevant knowledge points. These compensated knowledge points are gradually introduced through recursion to ensure that each knowledge point is understood and not missed. Through recursive knowledge point reinforcement, the blind-area-free credible problem-solving chain generated contains all the knowledge points that the user must master in the problem-solving process.

[0024] The known conditions and target to be solved are defined as reasoning anchor points, and based on the blind-area-free credible problem-solving chain and multi-dimensional exam point tags, step-by-step analysis logic derivation is performed to output a target analysis logic framework.

[0025] In the problem-solving process, the known conditions and target to be solved are the core of reasoning, and the reasoning anchor points refer to these elements that play an anchoring role in the entire reasoning process, serving as the starting point and endpoint of problem-solving. Based on the blind-area-free credible problem-solving chain, the problem-solving logic is derived step by step in a step-by-step manner. These steps are organized according to the logical relationship between the problem and the knowledge points, ensuring that the problem-solving process is clear and rigorous, and ultimately generating a target analysis logic framework. This framework is a clear and structured problem-solving process that demonstrates the reasoning process from the known conditions to the target to be solved.

[0026] According to the real-time user's real-time explanation modal selection, the target analysis logic framework is rendered in a cross-modal targeted manner, and a multi-modal analysis explanation flow is delivered as output.

[0027] Users have different preferences and needs. In order to provide the best explanation experience, according to the user's choice, decide to use different modalities such as text, image, voice or visualization, cross-modal targeted rendering of the target analysis logical framework, targeted rendering refers to the presentation of the target problem-solving logical framework in a specific form, and the explanation content of each step will be optimized according to the selected mode. Finally, the analysis logical framework is combined with the user-selected mode to generate a complete multi-modal analysis explanation flow, which is delivered to the user. Multi-modal analysis explanation flow means that the same problem-solving process can be presented to users in different ways, allowing users to choose the most suitable learning method according to their needs, which can effectively improve user learning efficiency and help them understand the knowledge points behind each step in the problem-solving process.

[0028] Further, according to the user's learning profile, the knowledge blind area coverage screening of the plurality of initial problem-solving paths is performed to locate the target problem-solving path, the method comprising: The history of the real-time user's wrong questions and learning profile data are called locally to perform knowledge gap statistics, and a hierarchical knowledge point defect topology map is constructed. The plurality of initial knowledge point sequences of the plurality of initial problem-solving paths are used to traverse the hierarchical knowledge point defect topology map to perform blind area matching calculation, and a plurality of knowledge point blind area coverage rate quantitative indicators are obtained. According to the ascending order sorting result of the plurality of knowledge point blind area coverage rate quantitative indicators, the target problem-solving path is screened from the plurality of initial problem-solving paths.

[0029] The history of wrong questions is a user's past wrong questions, and each wrong question will mark the user's deficiencies in which knowledge points, and these knowledge points are potential knowledge blind areas; The learning profile data is a user's personalized learning file, which contains the user's learning progress, subject level, mastered knowledge points, preferred problem-solving methods and other information. Analyze these data to count the user's knowledge point gap, that is, to assess the user's deficiencies in which knowledge points and which are the user's weak links that need to be strengthened.

[0030] Based on the statistics of knowledge point gaps, a hierarchical knowledge point defect topology map is constructed. The structure of this map is similar to a knowledge tree, which organizes knowledge points according to hierarchical relationships. Through this hierarchical structure, it can clearly show which basic knowledge points the user has a large blind area, and which advanced knowledge points are difficult due to a weak foundation.

[0031] Each initial problem-solving path contains a series of initial knowledge point sequences, that is, core knowledge points that need to be used in the problem-solving process. By comparing these initial knowledge point sequences with the constructed hierarchical knowledge point defect topology map, the coverage of each problem-solving path on the user's knowledge blind area is evaluated by calculating the overlap of the knowledge points involved in these problem-solving paths and the user's knowledge blind area, and a plurality of knowledge point blind area coverage rate quantitative indicators are quantitatively obtained, which represent how much each problem-solving path covers the user's knowledge blind area. Higher coverage rate means that the path is better in solving the user's blind area.

[0032] According to the plurality of knowledge point blind area coverage rate quantitative indicators, the problem-solving paths are sorted in ascending order according to the coverage rate from low to high. The sorted result can clearly reflect which problem-solving paths are more likely to help users fill their knowledge blind area. After sorting, the target problem-solving path is selected from the sorted result, which is the problem-solving path that covers the most user's knowledge blind area, that is, the problem-solving path with the highest blind area coverage rate. This problem-solving path is determined to be the most suitable path at present, which can help users solve the current problem to the greatest extent and fill their knowledge blind area.

[0033] Further, the local calls the real-time user's historical mistake set and learning profile data to perform knowledge point gap statistics, and a hierarchical knowledge point defect topology map is constructed. The method comprises: Error knowledge points are extracted from the historical mistake set to obtain a plurality of time sequence state labels of a plurality of error knowledge points. Dynamic mastery degree decay calculation is performed according to the plurality of time sequence state labels, and K defect knowledge points are selected from the plurality of error knowledge points. The K defect knowledge points are taken as root nodes, and the learning profile data is traversed to perform dependence backtracking of hierarchical associated discipline knowledge to obtain K associated knowledge point sequences. The K associated knowledge point sequences are subjected to knowledge point topology association processing to construct the hierarchical knowledge point defect topology map.

[0034] The knowledge points involved in each question in the historical mistake set are extracted. For each mistake, the core knowledge point examined by the question is marked. Each question in the historical mistake set records the error type encountered by the user on the question and the knowledge points involved. By analyzing the mistake set, all the error knowledge points are obtained.

[0035] After marking the error knowledge points, time sequence state labels of each error knowledge point are obtained, which means that the mastery state of each error knowledge point is time series labeled to show the user's mastery degree of the knowledge point at a certain time point. For example, the user's mastery of the concept of "limit of function" is poor at the first time, but gradually gets better in the following exercises. The time sequence labels show the user's mastery degree of the knowledge point at different time points.

[0036] The system uses temporal state annotation to calculate the decay of mastery over time for incorrect knowledge points. This means assessing whether the mastery of a particular incorrect knowledge point decreases over time. If a knowledge point is not reviewed or applied within a certain period, a forgetting effect occurs, leading to a decline in mastery. By dynamically calculating this decay, the system evaluates the user's understanding of each knowledge point. If the temporal state annotation of a knowledge point indicates a significant decay in mastery, it is identified as a weak knowledge point. Based on the calculation results, K weak knowledge points (K being a positive integer) are selected from multiple incorrect knowledge points. These K weak knowledge points represent the user's weakest areas, exhibiting strong decay over time or a high frequency of incorrect answers.

[0037] The selected K weak knowledge points are used as the root nodes of the topology graph. Each root node represents a user's weak point in that knowledge point. The user will then backtrack to related knowledge points based on these weaknesses. The learning profile data contains the knowledge points the user has mastered and their mastery level. This data is used to traverse the relationships between knowledge points and backtrack to other knowledge points related to the weak points. Learning records in the learning profile data allow tracking the user's learning progress and identifying supplementary content related to the weak knowledge points. Through this dependency backtracking process, a sequence of K related knowledge points is derived for each weak knowledge point. These related knowledge point sequences are progressively more advanced, representing the foundational or higher-level knowledge required to learn that specific knowledge point.

[0038] By analyzing a sequence of K related knowledge points, these knowledge points are topologically linked to form a hierarchical structure. Topological linking refers to the hierarchical presentation of relationships between knowledge points in a graph, unfolding layer by layer from basic to advanced. The topological graph displays the dependencies between these knowledge points, ensuring that each knowledge point is appropriately connected during the learning process. Through topological linking, a complete hierarchical knowledge point deficiency topological graph is ultimately constructed. This graph presents the hierarchical relationships between the K deficient knowledge points and their related knowledge points, helping to identify which knowledge points the user most needs to strengthen and displaying the learning order for each knowledge point.

[0039] Furthermore, deep feature extraction is performed on the structured question information to drive the three-level knowledge structure linkage matching of the problem-solving approach base, generating multiple initial problem-solving paths. The method includes: Based on the multi-dimensional test point tags, hierarchical mapping is performed on the L1 level knowledge module of the problem-solving approach library to locate the target knowledge module; based on the known conditions and the target to be solved, N solution types are selected and matched in the L2 level problem-solving method layer of the target knowledge module, where N≥5 and N is a positive integer; problem feature vectors are extracted from the structured problem information, where the problem feature vectors include the number of known conditions and the complexity of condition association; based on the N solution types and problem feature vectors, confidence-weighted template extraction is performed in the L3 level example template library to obtain the multiple initial problem-solving paths.

[0040] Based on multi-dimensional test point tags, the target knowledge module is located in the L1 level knowledge module of the problem-solving approach library through hierarchical mapping. This process matches the multi-dimensional test point tags with the knowledge structure in the problem-solving approach library, thereby classifying specific knowledge points into the corresponding modules in the library. The L1 level knowledge module is the most basic level, containing the most basic concepts, definitions, and theorems. Based on the information in the tags, these basic knowledge modules are located, preparing to provide a theoretical foundation for subsequent problem-solving path matching.

[0041] Further matching within the problem-solving approach database involves filtering possible solutions based on the known conditions and the desired outcome of the problem. The known conditions refer to the data or relationships provided in the problem, while the desired outcome is the content the problem requires to be solved. The L2 level problem-solving approach layer is a collection of solutions or methods for a specific problem. It searches for solutions applicable to the known conditions and desired outcome under the target knowledge module, and then selects N solution types. These solution types can be combinations of multiple methods; there may be multiple different approaches to the same problem. The selected solution types N≥5, meaning multiple problem-solving approaches are recommended to ensure users have ample choice and problem-solving strategies.

[0042] The problem feature vector is extracted through the analysis of structured problem information. It mainly reflects the core features of the problem. Among them, the number of known conditions refers to the number of given conditions in the problem; the condition association complexity reflects the interdependence between the conditions in the problem.

[0043] The L3-level example template library, trained and optimized based on a large model, aims to provide high-quality example template resources for generating problem-solving paths. This library contains templates for solutions to various types of problems throughout history. Each template represents a specific type of problem-solving strategy and demonstrates how to deduce the solution process based on the problem's characteristics and known conditions, using the solution steps of a specific problem. By calculating the weighted cosine similarity between the problem's feature vector and the solution type's feature vector, the L3-level template library can efficiently extract solution templates similar to the problem, thus providing support for initial problem-solving paths. During the training of the large model, the L3-level template library relies on L1 and L2 level knowledge modules. Through multi-layered training, the model acquires a comprehensive output capability ranging from basic knowledge to complex problem-solving strategies. This allows the large model to accurately match the most suitable solution template based on problem characteristics and generate personalized problem-solving paths according to the learning needs of different users.

[0044] Confidence-weighted template extraction refers to extracting the most suitable solution template from the L3-level database based on the similarity between the solution type and the problem's feature vectors, and assigning a confidence score to each template. Templates with higher confidence scores indicate that the solution is more applicable to the current problem. Confidence-weighted extraction is based on feature vector similarity calculation, calculating the correlation between each solution type and the problem features, and selecting the most suitable template. Through the above process, multiple initial solution paths are selected from the L3-level example template database, each path corresponding to a possible solution approach.

[0045] Furthermore, based on the N solution types and problem feature vectors, confidence-weighted template extraction is performed in the L3 level example template library to obtain the multiple initial solution paths. The method includes: Extract N L3 level problem feature vectors associated with the N solution types; calculate N weighted cosine similarities between the problem feature vectors and the N L3 level problem feature vectors; based on the descending order of the N weighted cosine similarities, select the top W solution types from the N solution types, where W is a positive integer less than N / 3; extract multiple example templates from the L3 level example template library for the W solution types as the multiple initial solution paths.

[0046] For N solution types, each solution type represents a possible problem-solving approach, and each solution type has its corresponding example problems, that is, the solution steps or templates of similar problems known in the past. For each solution type, multiple problems related to that solution type are found from the L3 level example template library, and the corresponding L3 level example feature vectors are extracted, reflecting the core structure, solution steps, condition types, and other information of the problem.

[0047] For each solution type and its corresponding L3-level problem example feature vector, calculate the cosine similarity between the L3-level problem example feature vector and the problem feature vector of the current problem. Cosine similarity is an index that measures the similarity between two vectors. Since the dimensions and contents of each problem feature vector are not exactly the same, different weights can be set according to different feature dimensions. For example, some conditions have a greater impact on the problem-solving path, so higher weights are given. For each solution type and its associated problem examples, calculate a weighted cosine similarity, which measures the similarity between the current problem and the historical problems associated with this solution type.

[0048] According to the calculated N weighted cosine similarities, sort the weighted cosine similarities of all solution types in descending order. A solution type with a high similarity means that this solution is more in line with the characteristics of the current problem. According to the similarity ranking, select the top W solution types from the N solution types, where W < N / 3, that is, the number of selected solution types is less than one-third of N. The purpose of this screening is to ensure that the recommended solution paths are both representative and not excessive, thus avoiding the user facing too many redundant choices. By restricting the size of W, the most suitable solution can be ensured and information overload can be avoided.

[0049] From the L3-level example template library, for the selected W solution types, extract the corresponding multiple example templates. These example templates represent the typical applications of each solution type. Finally, from these multiple example templates, integrate and generate multiple initial problem-solving paths, each of which contains specific problem-solving steps and methods, and the user can choose these paths to guide their problem-solving.

[0050] Furthermore, perform recursive knowledge point reinforcement on the target knowledge point sequence of the target problem-solving path to generate a blind-spot-free credible problem-solving chain. The method includes: Use the hierarchical knowledge point defect topology map to identify the blind spots of the target knowledge point sequence and locate T blind-spot knowledge points; use the hierarchical knowledge point defect topology map as a navigation taboo to perform equivalent analysis on the T blind-spot knowledge points to generate T recursive compensation knowledge point blocks; introduce the T recursive compensation knowledge point blocks into the target knowledge point sequence and perform recursive knowledge point reinforcement based on the hierarchical topology embedding position to generate the blind-spot-free credible problem-solving chain.

[0051] According to the target knowledge point sequence, traverse in the hierarchical knowledge point defect topology map, and identify T blind-spot knowledge points by comparing with the user's knowledge mastery. These blind spots are the key knowledge points that the user has not fully mastered. T is the number of knowledge points not mastered during the current problem-solving process.

[0052] In a topological graph, hierarchical relationships enable effective navigation. Navigation taboos mean adhering to the structure of the topological graph, avoiding loops or unreasonable jumps between knowledge points, and thus enabling more targeted supplementation. Using a hierarchical knowledge point deficiency topological graph as a guide, we can compensate for T blind spots in knowledge points and avoid repeatedly solving the same problems or neglecting the connections between knowledge points through systematic path guidance.

[0053] For each knowledge point that is difficult to grasp, we provide different expressions or understanding methods through equivalent analysis to help users better master the knowledge point. Equivalent analysis includes presenting knowledge points from multiple perspectives, such as deriving mathematical formulas, practical applications, and graphical understanding, to help users digest the knowledge point. These analysis methods can be customized according to the difficulty, importance, and dependence of the knowledge point on other knowledge points.

[0054] Each blind spot knowledge point generates a recursive compensation knowledge point block. The recursive compensation knowledge point block can be regarded as a module, which contains a detailed analysis of the blind spot knowledge point and the recursive relationship between knowledge points. The recursive compensation process means that it not only fills the gap of a single knowledge point, but also delves into the related prerequisite knowledge points to ensure that the mastery of the blind spot is comprehensive.

[0055] The obtained T recursive compensation knowledge point blocks are introduced into the target knowledge point sequence. These compensation blocks are not simply adding knowledge points, but rather appropriately integrating them into the problem-solving chain based on the logical structure of the problem and the dependencies between knowledge points. This ensures that every missing knowledge point is effectively supplemented, and that the newly introduced knowledge points do not negatively impact the smoothness of the problem-solving path. Hierarchical topological embedding position refers to the relative position and dependencies of each knowledge point in the topological graph. Following these hierarchical structures, reinforcement is performed according to the dependency order and embedding position between knowledge points. Finally, through recursive reinforcement, a blind-spot-free reliable problem-solving chain is generated. This chain contains all necessary knowledge points, and each knowledge point is fully reinforced.

[0056] Furthermore, using the hierarchical knowledge point defect topology map as a navigation taboo, the equivalent analysis of the T blind spot knowledge points is performed to generate T recursive compensation knowledge point blocks. The method includes: T sets of knowledge coordinate labels are deconstructed and separated from the T blind spot knowledge points; using the hierarchical knowledge point defect topology map as navigation taboos, taboo filtering and retrieval are performed in the preset subject knowledge base according to the T sets of knowledge coordinate labels to obtain T sets of authoritative content blocks; the teaching logic structure is analyzed on the T sets of authoritative content blocks to generate the T recursive compensation knowledge point blocks, wherein the recursion depth does not exceed 3 levels.

[0057] Each blind spot knowledge point is deconstructed and separated, transforming them into knowledge coordinate labels. These labels precisely mark the position of a blind spot knowledge point within the subject's knowledge system. They include not only basic information such as name and category, but also related knowledge domains, depth levels, and dependencies with other knowledge points. This deconstruction provides a clearer understanding of each blind spot knowledge point's position within the overall subject framework and its relationship to other knowledge points.

[0058] The search results are filtered for irrelevant or insufficient information to address the current knowledge gap, thus avoiding interference from irrelevant data. After filtering and searching, T sets of authoritative content blocks that meet the requirements are extracted from the pre-built subject knowledge base. These authoritative content blocks are selected to effectively fill the user's knowledge gaps. Each knowledge gap corresponds to one authoritative content block, which possesses high-quality pedagogical value and helps users fill knowledge gaps.

[0059] The extracted T groups of authoritative content blocks undergo a structured analysis based on pedagogical logic. This involves decomposing and reorganizing these blocks according to pedagogical logic to better suit the user's learning process. This structured analysis is based on pedagogical theories and methods, organizing knowledge content hierarchically and step-by-step to help users understand and absorb it more easily. The structured analysis of pedagogical logic includes: the introduction of knowledge points (from simple concepts to complex ones); the decomposition of knowledge points (breaking down complex knowledge points into easier-to-understand parts); and the connections between knowledge points (displaying the relationships between different knowledge points to help users build a complete knowledge framework).

[0060] After structurally analyzing the teaching logic, recursive compensation knowledge point blocks are generated. These blocks are not merely simple knowledge supplements, but rather delve into related sub-knowledge points through recursion, helping users master relevant knowledge layer by layer from basic to complex. Recursive compensation refers to a process of iterative and gradual supplementation to deeply understand the premises and connections of each knowledge point, continuously strengthening the user's learning foundation until all knowledge gaps are completely filled.

[0061] During the recursive compensation process, certain restrictions are followed, namely, the recursion depth does not exceed 3 levels. This is to avoid the learning difficulty being too great due to an overly complex recursive process. A depth of three levels is sufficient to ensure that the basic knowledge points in the blind spots are covered, while maintaining the continuity of learning.

[0062] Furthermore, the method for separating structured question information from the standardized question data based on multimodal semantic understanding includes: After semantic role labeling of the standardized question data, multiple target subjects are located based on dependency syntax; the multiple target subjects are decomposed into known conditions and unsolved targets based on semantic analysis; multi-dimensional test point tags are extracted from the standardized question data to obtain a core test point set; the core test point set is cross-linked and diverged to output a related test point set; the core test point set and the related test point set are combined to extract the multi-dimensional test point tags.

[0063] Semantic role labeling is a natural language processing technique that aims to identify the grammatical and semantic roles of each word in a sentence. For example, in the sentence "students write with pencils," "students" is the agent, "pencil" is the tool, and "words" is the target. In multimodal parsing, standardized question data is semantically labeled to understand the semantic function of each word or phrase in the sentence. The process of semantic role labeling uses NLP models to label sentences, identify different semantic roles, and assign labels to them. Dependency parsing analyzes the dependencies between words in a sentence using syntactic tree structures. Locating multiple target subjects means that dependency parsing can identify the main subjects involved in the question, such as variables, people, and objects. These subjects are key elements in the problem-solving process.

[0064] Further semantic analysis is performed on multiple target subjects to further distinguish the identified target subjects into different types of information. Known conditions refer to the known facts, data, or assumptions given in the question; the desired objective refers to the content that the question requires to be answered or solved. Questions typically involve multiple knowledge points or skills, which can be extracted as test point tags by analyzing the question content. Multi-dimensional test point tags mean not only identifying the knowledge points directly involved in the question but also abstracting them from multiple dimensions, such as mathematical knowledge, physical formulas, common solution methods, and thinking patterns. Through multi-dimensional analysis, these test point tags are integrated into a core test point set, which are essential elements that cannot be ignored in the problem-solving process.

[0065] Cross-referencing and divergence refers to linking and expanding the core test point set with other potential test points. This process involves analyzing existing core test points to infer other knowledge domains or subdomains related to these test points. This process links core test point tags with other potentially related knowledge points through a divergent approach. Through cross-referencing and divergence, a set of related test points containing core test points and their related extended domains is generated. This set examines problems from different angles and dimensions, providing diverse knowledge frameworks for multiple solution methods and multiple paths. The core test point set and the related test point set are then merged, meaning all test points in both are combined and duplicates are removed to obtain the final multi-dimensional test point tags.

[0066] Furthermore, based on the long-term recurrence frequency of the structured question information, standardized question-and-answer storage of the target parsing logic framework is performed.

[0067] Long-term recurrence frequency refers to the frequency with which certain question types, knowledge points, or problem-solving methods reappear during a user's historical learning process. This is based on statistics from the user's learning records, error logs, and problem-solving history. By analyzing long-term recurrence frequency, we can identify which types of questions or knowledge points are frequently asked, and which solutions or approaches are repeatedly reused. These are aspects that require special attention during the problem-solving process. In the problem-solving process, the goal-oriented logical framework refers to the derivation process from known conditions to the problem-solving goal. By analyzing historical user data, a standardized answering framework is formed, standardizing and storing information such as problem-solving steps, methods, and strategies in the system. This means that the depth, difficulty, and details of the answering content can be automatically adjusted based on the user's historical learning situation. Through standardized storage, the relevant answering framework can be quickly retrieved each time a user asks a question, and personalized answers can be provided according to the user's needs and learning progress. This helps improve the user experience, making the problem-solving process more intelligent and personalized, while also improving problem-solving efficiency.

[0068] Example 2, based on the same inventive concept as the problem analysis and explanation generation method combined with the multimodal large model in the previous examples, such as... Figure 2 As shown in the embodiment of this application, a question analysis and explanation generation system combining a multimodal large model is provided. The system includes: The standardization processing module 10 is used to standardize the multimodal input questions to obtain standardized question data; the question information separation module 20 is used to separate structured question information from the standardized question data based on multimodal semantic understanding, wherein the structured question information includes known conditions, the target to be solved, and multi-dimensional test point tags; the problem-solving path matching module 30 is used to perform deep feature extraction on the structured question information, drive the three-level knowledge structure linkage matching of the problem-solving approach library, and generate multiple initial problem-solving paths; the coverage filtering module 40 is used to select the multiple initial problem-solving paths according to the user's learning profile. The system employs a knowledge blind spot coverage filtering mechanism to locate the target problem-solving path. A knowledge point reinforcement module 50 recursively reinforces the target knowledge point sequence of the target problem-solving path, generating a blind spot-free, reliable problem-solving chain. A logic deduction module 60 defines the known conditions and the target to be solved as reasoning anchors, performs step-by-step analytical logic deduction based on the blind spot-free, reliable problem-solving chain and multi-dimensional test point tags, and outputs the target analytical logic framework. A targeted rendering module 70 performs cross-modal targeted rendering of the target analytical logic framework based on the real-time explanation modality selection of the real-time user, delivering a multi-modal analytical explanation stream.

[0069] Furthermore, the coverage filtering module 40 is used to perform the following operation steps: The system locally accesses the historical error set and learning profile data of the real-time user to perform knowledge gap statistics and construct a hierarchical knowledge gap topology map. It then uses multiple initial knowledge point sequences from the multiple initial problem-solving paths to traverse the hierarchical knowledge gap topology map for blind spot matching calculation, obtaining multiple knowledge point blind spot coverage quantification indicators. Based on the ascending sorting results of the multiple knowledge point blind spot coverage quantification indicators, the system selects the target problem-solving path from the multiple initial problem-solving paths.

[0070] Furthermore, the coverage filtering module 40 is used to perform the following operation steps: Error knowledge points are extracted from the historical set of incorrect questions to obtain multiple temporal state labels for multiple error knowledge points; dynamic mastery decay calculation is performed based on the multiple temporal state labels to select K defective knowledge points from the multiple error knowledge points; using the K defective knowledge points as root nodes, the learning profile data is traversed to perform hierarchical dependency backtracking of subject knowledge to obtain K related knowledge point sequences; knowledge point topology association processing is performed on the K related knowledge point sequences to construct the hierarchical knowledge point defect topology map.

[0071] Furthermore, the problem-solving path matching module 30 is used to perform the following operation steps: Based on the multi-dimensional test point tags, hierarchical mapping is performed on the L1 level knowledge module of the problem-solving approach library to locate the target knowledge module; based on the known conditions and the target to be solved, N solution types are selected and matched in the L2 level problem-solving method layer of the target knowledge module, where N≥5 and N is a positive integer; problem feature vectors are extracted from the structured problem information, where the problem feature vectors include the number of known conditions and the complexity of condition association; based on the N solution types and problem feature vectors, confidence-weighted template extraction is performed in the L3 level example template library to obtain the multiple initial problem-solving paths.

[0072] Furthermore, the problem-solving path matching module 30 is used to perform the following operation steps: Extract N L3 level problem feature vectors associated with the N solution types; calculate N weighted cosine similarities between the problem feature vectors and the N L3 level problem feature vectors; based on the descending order of the N weighted cosine similarities, select the top W solution types from the N solution types, where W is a positive integer less than N / 3; extract multiple example templates from the L3 level example template library for the W solution types as the multiple initial solution paths.

[0073] Furthermore, the knowledge point reinforcement module 50 is used to perform the following operation steps: The hierarchical knowledge point defect topology map is used to identify blind spots in the target knowledge point sequence, locating T blind spot knowledge points. Using the hierarchical knowledge point defect topology map as a navigation taboo, the equivalent analysis of the T blind spot knowledge points is performed to generate T recursive compensation knowledge point blocks. The T recursive compensation knowledge point blocks are introduced into the target knowledge point sequence, and recursive knowledge point reinforcement is performed based on the hierarchical topology embedding position to generate the blind spot-free reliable problem-solving chain.

[0074] Furthermore, the knowledge point reinforcement module 50 is used to perform the following operation steps: T sets of knowledge coordinate labels are deconstructed and separated from the T blind spot knowledge points; using the hierarchical knowledge point defect topology map as navigation taboos, taboo filtering and retrieval are performed in the preset subject knowledge base according to the T sets of knowledge coordinate labels to obtain T sets of authoritative content blocks; the teaching logic structure is analyzed on the T sets of authoritative content blocks to generate the T recursive compensation knowledge point blocks, wherein the recursion depth does not exceed 3 levels.

[0075] Furthermore, the question information separation module 20 is used to perform the following operation steps: After semantic role labeling of the standardized question data, multiple target subjects are located based on dependency syntax; the multiple target subjects are decomposed into known conditions and unsolved targets based on semantic analysis; multi-dimensional test point tags are extracted from the standardized question data to obtain a core test point set; the core test point set is cross-linked and diverged to output a related test point set; the core test point set and the related test point set are combined to extract the multi-dimensional test point tags.

[0076] Furthermore, based on the long-term recurrence frequency of the structured question information, standardized question-and-answer storage of the target parsing logic framework is performed.

[0077] Through the foregoing detailed description of the method for generating questions combining multimodal large models, those skilled in the art can clearly understand the question analysis and explanation generation system combining multimodal large models in this embodiment. Since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and relevant parts can be found in the method section.

[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for generating problem analysis and explanation based on multimodal large-scale models, characterized by: The method includes: The multimodal input questions are standardized to obtain standardized question data; Based on multimodal semantic understanding, structured question information is separated from the standardized question data, wherein the structured question information includes known conditions, the target to be found, and multi-dimensional test point tags; Deep feature extraction is performed on the structured question information to drive the three-level knowledge structure linkage matching of the problem-solving approach base and generate multiple initial problem-solving paths; Based on the user's learning profile, the knowledge blind spots of the multiple initial problem-solving paths are covered and filtered to locate the target problem-solving path; The target knowledge point sequence of the target problem-solving path is recursively reinforced to generate a reliable problem-solving chain without blind spots; The known conditions and the target to be found are defined as reasoning anchors. Based on the blind-spot-free reliable problem-solving chain and multi-dimensional test point labels, step-by-step analytical logic derivation is performed to output the target analytical logic framework. Based on the real-time explanation modality selection of the real-time user, cross-modal directional rendering of the target parsing logic framework is performed to deliver a multimodal parsing and explanation stream.

2. The method for generating problem analysis and explanations combining a multimodal large model as described in claim 1, characterized in that, Based on the user's learning profile, the method involves screening for knowledge gaps in the multiple initial problem-solving paths to locate the target problem-solving path. The system locally accesses the historical wrong question set and learning profile data of the real-time user to perform knowledge gap statistics and construct a hierarchical knowledge gap topology map; By using the sequence of multiple initial knowledge points from the multiple initial problem-solving paths to traverse the hierarchical knowledge point defect topology map, blind spot matching calculation is performed to obtain a quantitative index of blind spot coverage for multiple knowledge points. Based on the ascending sorting results of the quantitative indicators of coverage of multiple knowledge point blind spots, the target problem-solving path is selected from the multiple initial problem-solving paths.

3. The method for generating problem analysis and explanations combining a multimodal large model as described in claim 2, characterized in that, The method involves locally accessing the historical error sets and learning profile data of the real-time user to perform knowledge gap statistics and construct a hierarchical knowledge gap topology map. The historical set of incorrect questions is used to extract the erroneous knowledge points, resulting in multiple temporal state annotations for multiple erroneous knowledge points. Based on the multiple time-series state labels, dynamic mastery decay calculation is performed, and K defective knowledge points are selected from the multiple erroneous knowledge points. Using the K deficient knowledge points as the root nodes, the learning profile data is traversed to perform hierarchical dependency backtracking of subject knowledge, resulting in a sequence of K related knowledge points. The K related knowledge point sequences are processed by knowledge point topology association to construct the hierarchical knowledge point defect topology map.

4. The method for generating problem analysis and explanations combining a multimodal large model as described in claim 1, characterized in that, The method involves performing deep feature extraction on the structured question information to drive the three-level knowledge structure linkage matching of the problem-solving approach base, generating multiple initial problem-solving paths. Based on the multi-dimensional test point tags, hierarchical mapping is performed on the L1 level knowledge module of the problem-solving strategy library to locate the target knowledge module; Based on the known conditions and the target to be found, N solution types are selected and matched in the L2 level problem-solving method layer of the target knowledge module, where N≥5 and N is a positive integer; Extract question feature vectors from the structured question information, wherein the question feature vectors include the number of known conditions and the complexity of condition association; Based on the N solution types and problem feature vectors, confidence-weighted template extraction is performed in the L3 level example template library to obtain the multiple initial solution paths.

5. The method for generating problem analysis and explanations combining a multimodal large model as described in claim 4, characterized in that, Based on the N solution types and problem feature vectors, confidence-weighted template extraction is performed in the L3 level example template library to obtain the multiple initial solution paths. The method includes: Extract the feature vectors of N L3 level problem examples associated with the N solution types; Calculate the N weighted cosine similarities between the feature vector of the given question and the feature vectors of the N L3 level question examples; Based on the descending order of the N weighted cosine similarities, the top W solution types are selected from the N solution types, where W is a positive integer less than N / 3; Multiple example templates of the W solution types are extracted from the L3 level example template library as the multiple initial solution paths.

6. The method for generating problem analysis and explanations combining a multimodal large model as described in claim 2, characterized in that, The method involves recursively reinforcing the target knowledge point sequence of the target problem-solving path to generate a reliable problem-solving chain without blind spots. The hierarchical knowledge point defect topology map is used to identify blind spots in the target knowledge point sequence and locate T blind spot knowledge points. Using the hierarchical knowledge point defect topology map as a navigation taboo, perform equivalent analysis of the T blind spot knowledge points to generate T recursive compensation knowledge point blocks; The T recursive compensation knowledge point blocks are introduced into the target knowledge point sequence, and recursive knowledge point reinforcement is performed based on the hierarchical topological embedding position to generate the blind-zone-free reliable problem-solving chain.

7. The method for generating problem analysis and explanations combining a multimodal large model as described in claim 6, characterized in that, Using the hierarchical knowledge point defect topology map as a navigation taboo, the equivalent analysis of the T blind spot knowledge points is performed to generate T recursive compensation knowledge point blocks. The method includes: From the T blind spot knowledge points, deconstruct and separate T sets of knowledge coordinate labels; Using the hierarchical knowledge point defect topology map as a navigation taboo, and based on the T sets of knowledge coordinate labels, taboo filtering and retrieval are performed in the preset subject knowledge base to obtain the T sets of authoritative content blocks; The teaching logic structure is analyzed on the T groups of authoritative content blocks to generate the T recursive compensation knowledge point blocks, wherein the recursion depth does not exceed 3 levels.

8. The method for generating problem analysis and explanations combining a multimodal large model as described in claim 1, characterized in that, The method for separating structured question information from the standardized question data based on multimodal semantic understanding includes: After semantic role labeling of the standardized question data, multiple target subjects are located based on dependency syntax; Based on semantic analysis, the multiple target entities are decomposed into known conditions and targets to be found; Multi-dimensional test point tags are extracted from the standardized question data to obtain a core test point set; Cross-reference and diverge the core test point set to output the associated test point set; The multi-dimensional test point labels are obtained by performing a union extraction on the core test point set and the related test point set.

9. The method for generating problem analysis and explanations combining a multimodal large model as described in claim 1, characterized in that, Based on the long-term recurrence frequency of the structured question information, standardized question-and-answer storage is performed on the target parsing logic framework.

10. A problem analysis and explanation generation system combining multimodal large-scale models, characterized in that: The system is used to implement the problem analysis and explanation generation method combining a multimodal large model as described in any one of claims 1-9, the system comprising: The standardization processing module is used to standardize multimodal input questions to obtain standardized question data. The question information separation module is used to separate structured question information from the standardized question data based on multimodal semantic understanding. The structured question information includes known conditions, objectives to be found, and multi-dimensional test point tags. The problem-solving path matching module is used to perform deep feature extraction on the structured problem information, drive the three-level knowledge structure linkage matching of the problem-solving idea library, and generate multiple initial problem-solving paths; The coverage filtering module is used to cover and filter the knowledge blind spots of the multiple initial problem-solving paths based on the user's learning profile, and to locate the target problem-solving path. The knowledge point reinforcement module is used to recursively reinforce the target knowledge point sequence of the target problem-solving path to generate a reliable problem-solving chain without blind spots. The logic deduction module is used to define the known conditions and the target to be found as reasoning anchors, perform step-by-step parsing logic deduction based on the blind-zone-free reliable problem-solving chain and multi-dimensional test point tags, and output the target parsing logic framework. The directional rendering module is used to perform cross-modal directional rendering of the target parsing logic framework based on the real-time explanation modality selection of the real-time user, and deliver a multimodal parsing and explanation stream.

Images