Higher mathematics problem intelligent data set construction method, system and equipment based on multi-modal interaction and storage medium

Abstract

The invention belongs to the technical field of artificial intelligence, and discloses an advanced mathematics problem intelligent data set construction method, system and device based on multi-modal interaction and a storage medium, and the method comprises the steps: collecting related problems to form an intelligent education data set, carrying out demand analysis, and determining a mathematical operation type, a multi-modal expression form and a dynamic adjustment difficulty strategy; constructing a mathematical basic normal form, and converting the mathematical basic normal form into different expression forms by using a multi-modal template; a difficulty coefficient model is adopted to adjust the difficulty and diversity of mathematical problems in the intelligent education data set in real time; based on the mathematical basic normal form, generating mathematical problem instances by using a random function, and adjusting the ratio of the problem instances of each difficulty level in real time; multi-modal expansion is carried out on a basic input command, and each mathematical problem instance is converted into a command form of different modals. According to the method, the learning efficiency can be optimized, the problem solving speed is improved, the cross-modal processing capability of the model is enhanced, and the application scene is widened.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a method, system, device and storage medium for constructing an intelligent data set for advanced mathematical problems based on multimodal interaction. Background Art

[0002] Artificial intelligence, especially generative artificial intelligence technology, is leading a new round of industrial transformation. Generative artificial intelligence represented by large language models such as ChatGPT has demonstrated extensive application potential in many industries such as transportation, finance, medical care, education, media, and government affairs with its excellent generalization ability, self-supervised learning ability, multi-task processing ability, knowledge integration and reasoning ability, and multi-modal input and output ability. However, large language models are relatively weak in the field of mathematical calculations, especially when dealing with complex mathematical calculation problems, and face many challenges. Existing technologies usually rely on external computing plug-ins or Python interpreters for mathematical calculations. Such methods work well in single calculations, but are inefficient and inaccurate in multi-step logical reasoning or complex mathematical problem processing, making it difficult to meet actual application needs.

[0003] In response to the above problems, patent application CN118152543A discloses a method and device for constructing a large model data set, which constructs several basic mathematical paradigms, generates mathematical problem instances and answer instances, and forms an initial data set; forms an input command set by expanding the diversity of input commands, and generates a basic data set in combination with mathematical problem instances; adjusts the data ratio based on the paradigm difficulty coefficient to construct a training data set; further adjusts the word segmentation rules to optimize the model's word segmentation and calculation processing capabilities for numbers. This method effectively improves the computing power of the large language model in mathematical calculation problems. However, this method still has some shortcomings, such as: 1) Regarding the matching problem of data set difficulty division and model effect, this method optimizes the training data set by adjusting the difficulty coefficient and data ratio, but the basis for the division of the difficulty coefficient is mainly based on the complexity of mathematical derivation logic, lacking a dynamic evaluation of the model learning efficiency, resulting in a mismatch between the data distribution and the actual learning effect of the model, thus affecting the training efficiency and the final computing power improvement effect; 2) Insufficient flexibility in data expansion: When expanding the input command expression, this method mainly relies on preset rules and a fixed set of approximate expressions, and does not fully combine the context to dynamically generate more natural and diverse expressions, which may limit the model's ability to understand natural language problems; 3) Insufficient optimization of the model reasoning process: The answer instance of this method contains an intermediate derivation process, but does not further optimize the hierarchical expression and step-by-step verification mechanism of the derivation process, resulting in insufficient stability of the model when reasoning complex problems. Therefore, there is still much room for improvement in the existing technology in improving the mathematical computing power of large models. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method, system, device and readable storage medium for constructing an intelligent data set for advanced mathematical problems based on multimodal interaction, which solves the problem that the lack of dynamic evaluation of model learning efficiency leads to a mismatch between data distribution and the actual learning effect of the model, thereby affecting the training efficiency and the ultimate computing power improvement effect.

[0005] The method for constructing an intelligent data set for advanced mathematical problems based on multimodal interaction of the present invention comprises the following steps: Step 1: Collect basic operation problems, advanced mathematics problems and multimodal understanding problems to form an intelligent education dataset, analyze the requirements of the intelligent education dataset, determine the mathematical operation type, multimodal expression form and dynamic difficulty adjustment strategy; Step 2: Based on the demand analysis, construct the basic mathematical paradigm and convert it into different expressions using multimodal templates; Step 3: Use the difficulty coefficient model to adjust the difficulty and diversity of math problems in the smart education dataset in real time; specifically: According to the difficulty adjustment strategy, by setting the basic data set, the accuracy, processing time, problem complexity, diversity index, innovation index and stability index in the model training process are collected in real time to calculate the current difficulty coefficient. The expression is: , ; in, A comprehensive indicator that indicates student ability feedback, including knowledge point mastery rate , Problem-solving speed and answer accuracy , , , are weight factors, corresponding to the relative importance of knowledge point mastery rate, problem-solving speed, and answer accuracy in the comprehensive index; D t Indicates the current difficulty coefficient, A t represents the accuracy of the model at time t, β represents the factor that adjusts the diversity V, V t represents the diversity index of problem solving, γ represents the factor that regulates the impact of innovation I on difficulty, and I t Indicator representing the innovativeness of problem solving, T t represents the average time required to solve the problem, λ represents the constant for adjusting the processing time, and C t represents the problem complexity score, ω t represents the dynamic question type weight factor, W represents the question type weight, and η represents the constant contributing to the difficulty of stability; Based on the current difficulty coefficient, the polynomial regression algorithm is used to calculate the target difficulty coefficient. The expression is: ; Among them, D ta represents the target difficulty coefficient, f represents a predefined function, and D t -1 indicates the difficulty coefficient of the previous time step, A t -1 indicates the accuracy of the model at the previous time point, represents the time interval, θ represents the parameter of the adjustment function; Based on the deviation between the target difficulty coefficient and the current difficulty coefficient, combined with the knowledge coverage, problem-solving speed and accuracy of the question type, a reinforcement learning method is used to optimize question selection and dynamically adjust question distribution; Step 4: Based on the basic mathematical paradigm, use random functions to generate mathematical problem instances, and adjust the ratio of problem instances of different difficulty levels in real time; Step 5: Convert each mathematical problem instance into a command form of different modes by performing multimodal expansion on the basic input commands.

[0006] Furthermore, in step 1, the basic operation problems, advanced mathematics problems and multimodal understanding problems are collected to form an intelligent education data set, specifically: Collect basic operation problems of addition, subtraction, multiplication and division, advanced mathematics problems of algebra, geometry, probability statistics and calculus, and multimodal understanding problems of text, image and voice through online education platforms; Query existing models for mathematical problems in the field of advanced mathematics, select the most viewed advanced mathematics problems; analyze the successes and failures of existing models in handling these cases.

[0007] Furthermore, in step 1, the analysis of the intelligent education data set requirements and determination of the mathematical operation type and multimodal expression form are specifically as follows: By listing integer operations, fraction operations, and decimal operations, we can filter out key knowledge points in algebra, geometry, probability statistics, and calculus. Vote on key issues in each area and determine the priority of each area based on the number of votes; Use LaTeX text formulas to establish a sample library of formula texts with standard writing standards; Use Mathematica software to generate mathematical images and unify the drawing standards and templates for geometric figures and function images; We investigated existing speech recognition and synthesis technologies through questionnaires, and used speech synthesis software TTS to write speech scripts covering various mathematical expressions. The dynamic difficulty adjustment strategy is specifically as follows: According to the complexity and abstractness of mathematical problems, the difficulty coefficient calculation method is adopted, and the validity of the difficulty index model is tested in combination with historical data; Build a preliminary adaptive algorithm model based on the machine learning framework TensorFlow, and test the effect of algorithm adjustment difficulty in a simulated environment; According to the difficulty index, the number of each difficulty question is divided into different difficulty levels from the existing database.

[0008] Furthermore, in step 2, based on the demand analysis, a basic mathematical paradigm is constructed and converted into different expressions using a multimodal template, specifically: By sorting out the types of mathematical operations at each level, they are divided into subcategories: basic operations, algebraic operations, geometric operations, functions, and calculus. For each subcategory, multiple sample questions of different difficulty levels are designed that cover all core mathematical knowledge points. Through the hierarchical derivation and verification mechanism, complex mathematical derivation problems are decomposed into multiple sub-steps, each of which clearly defines the logic and key steps of the problem; Verify each sub-step step by step, and build a hierarchical verification framework to record the correctness verification of each step in the derivation process to form a complete derivation path; Generate variant problems by changing values, variables, and structural elements. Write a problem-solving step-by-step description document for each type of problem to clarify the problem-solving logic and key steps. Use LaTeX to format formulas and create a mathematical formula library that is easy to search for basic operations to advanced functions; use text description templates and write specific text descriptions for each example problem based on the framework; Based on the content of the questions, we use Mathematica to draw clear and accurately labeled mathematical graphics, use LaTeX online conversion tools to convert mathematical formulas into high-definition pictures, write voice scripts for each mathematical problem, and use TTS to synthesize clear and fast speech that is suitable for the average comprehension speed of advanced mathematics students.

[0009] Furthermore, in step 4, based on the basic mathematical paradigm, a random function is used to generate mathematical problem instances, and the ratio of problem instances of different difficulty levels is adjusted in real time, specifically: By setting the basic paradigm of mathematical problems as P={P1,P2…,P n}, a composite function is used to adjust the difficulty dynamics, the expression is: , in, represents the value of the composite difficulty function, i, j, k represent the distinction between different mathematical paradigms, difficulty levels and other classifications, x represents the average accuracy of the model on this paradigm, y = (S, T) represents a vector, S represents the student ability prediction index, T represents time sensitivity, Indicates the parameters related to adjusting the model performance A, Indicates the adjustment of the student's ability S in the difficulty calculation. represents the weight of controlling time sensitivity T, ζ represents the weight of the interaction term affecting model performance A and time T, represents a bias value that dynamically balances mathematical paradigms, difficulty levels, and problem classifications, Represents diversity, express The degree of influence of nonlinearity in the difficulty calculation; According to each paradigm P i , construct a multi-level random generator, combining polynomial, trigonometric and logarithmic functions, and calculate mathematical problem examples. The expression is: , in, represents the value of the nonlinear function, a, b, c, d represent the random parameters used when generating the problem through Gaussian distribution, Represents the unknowns in the problem instance; according to Based on the calculation results, an adaptive grouping algorithm is used to dynamically allocate the number of instances of different difficulty levels, and multimodal transformation is performed on each generated mathematical instance.

[0010] Furthermore, in step 5, each mathematical problem instance is converted into a command form of different modes by performing multimodal expansion on the basic input command, specifically: Combining preset Chinese commands and standard mathematical expressions, using context analysis technology (such as BERT or GPT models), dynamically generate a variety of input command expressions, including synonym replacement, sentence changes, and context supplements, to improve the naturalness and diversity of commands; By applying image conversion algorithms to each basic mathematical paradigm, we construct diverse image instructions for each problem instance and dynamically generate image annotations based on the semantic content of the problem. Use TTS technology to convert text modality into voice commands, record voice commands with different accents, speaking speeds, and intonations, integrate text, images, and voice instructions for each math problem instance, and create a complete set of multimodal command sets.

[0011] The present invention also provides an intelligent data set system for advanced mathematical problems based on multimodal interaction, which is used to implement the method described, including: Requirements analysis and framework design module: Analyze the data set requirements, determine the type of mathematical operations, multimodal expressions, and dynamic difficulty adjustment strategies; Multimodal expression generation module: Based on demand analysis, it constructs basic mathematical paradigms and converts them into different expression forms using multimodal templates; Difficulty dynamic adjustment module: adopts difficulty coefficient model to adjust the difficulty and diversity of mathematical problems in the data set in real time; Example generation and management module: Based on the constructed basic mathematical paradigm, it uses random functions to generate mathematical problem instances and adjusts the ratio of problem instances of different difficulty levels in real time; Multimodal instruction conversion module: By performing multimodal expansion on basic input commands, each mathematical problem instance is converted into a command form of different modes.

[0012] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned construction method when executing the computer program.

[0013] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned construction method are implemented.

[0014] The beneficial effects of the present invention are as follows: the present invention introduces a dynamic adjustment mechanism to optimize the difficulty coefficient and data ratio of the mathematical paradigm in real time according to the model learning efficiency, solves the problem of mismatch between data distribution and model learning effect in the prior art, and significantly improves the training efficiency and computing power. Secondly, a multimodal expression generation mechanism is adopted to dynamically generate more natural and diverse input commands in combination with the context, which enhances the model's ability to understand natural language problems and improves the flexibility and naturalness of multimodal expansion; Thirdly, the present invention constructs a hierarchical derivation and step-by-step verification mechanism for answer examples, decomposes the derivation process of complex mathematical problems into multiple levels, and gradually verifies the key steps, further improving the stability and accuracy of the model in complex reasoning problems. In addition, the present invention uses multimodal interaction technology to support graphic and text fusion input, significantly enhancing the scalability of the data set construction method, breaking through the limitation of single text input in the prior art, and at the same time improving the construction efficiency and reducing labor costs through a semi-automatic data generation framework. The present invention has achieved technical breakthroughs in dynamic adjustment, data diversity, reasoning stability and multimodal scalability, and comprehensively improved the performance and practicality of large language models in the field of mathematical computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of the method for constructing a large model data set in Example 1; Figure 2This is a process diagram of multimodal conversion of mathematical problems in Example 1. DETAILED DESCRIPTION

[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0017] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0018] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0019] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a method for constructing an intelligent data set for advanced mathematical problems based on multimodal interaction, comprising the following steps:

[0020] S1. Collect basic operation problems, advanced mathematics problems and multimodal understanding problems to form intelligent education data sets, analyze the needs of intelligent education data sets, determine the types of mathematical operations, multimodal expressions and dynamic difficulty adjustment strategies; Collect basic operation problems of addition, subtraction, multiplication and division, advanced mathematics problems of algebra, geometry, probability statistics and calculus, and multimodal understanding problems of text, image and voice through online education platforms; The collected data are divided into three categories: basic operations, advanced mathematical problems, and multimodal understanding. The existing models are queried for mathematical problems in the field of advanced mathematics, and the advanced mathematical problems with the highest viewing rate are selected; the successes and failures of the existing models in handling these cases are analyzed.

[0021] S2. Based on demand analysis, construct basic mathematical paradigms and convert them into different expressions using multimodal templates: By sorting out the types of mathematical operations at each level, the problems are divided into basic operations, algebraic operations, geometric operations, functions and calculus subcategories. Each subcategory is designed with multiple example questions of different difficulty levels and covering all core mathematical knowledge points. Through a hierarchical derivation and verification mechanism, complex mathematical derivation problems are decomposed into multiple sub-steps, and each step clearly defines the logic and key steps of the problem. Verify each sub-step step by step, and build a hierarchical verification framework to record the correctness verification of each step in the derivation process to form a complete derivation path; generate variant problems by changing numerical values, variables, and structural elements, write a problem-solving step-by-step description document for each type of problem, clarify the problem-solving logic and key steps, use LaTeX to arrange formulas, create an easy-to-query mathematical formula library from basic operations to advanced functions, use text description templates, and write specific text descriptions for each example problem based on the framework. According to the content of the problem, use Mathematica to draw clear and accurately labeled mathematical graphics, use LaTeX online conversion tools to convert mathematical formulas into high-definition pictures, write voice scripts for each mathematical problem, and use TTS to synthesize clear and fast speech that is suitable for the average comprehension speed of students with advanced mathematics levels.

[0022] The difficulty classification standards for mathematical problems include developing a set of detailed problem difficulty classification standards, which can score the difficulty of problems in each subcategory based on the complexity of the problem (such as the number of calculation steps, the number of concepts involved), innovation (whether the problem requires creative thinking to solve), and comprehensiveness (whether it spans multiple mathematical fields), and ensure that there are representative problems at each difficulty level.

[0023] The rules for generating variant problems are designed for each type of operation. For example, basic operations can change the value size, introduce negative numbers or fractions; algebraic operations can adjust the coefficients of equations and introduce nonlinear terms; geometric operations can change the size, position, and angle of figures; functions and calculus can adjust function parameters and consider different boundary conditions. This ensures that the generated variant problems are rich and diverse, and cover key knowledge points.

[0024] The writing standards of the problem-solving step-by-step instruction document are established, and a problem-solving step-by-step document template is established, which includes a restatement of the problem, an overview of the problem-solving ideas, a detailed description of the key steps (each step is accompanied by a formula written in LaTeX and necessary explanations), the answer and a brief analysis, to ensure that the document language is concise and suitable for learners of different levels.

[0025] The organization and retrieval mechanism of the LaTeX formula library creates a structured formula library, stores it by subject and difficulty level, uses the LaTeX tag system, sets keywords and descriptions for each formula, and develops a simple search engine or indexing system so that users can quickly find the required formulas.

[0026] Multimodal text description templates, design a set of text description templates suitable for different types of mathematical problems, including problem background, problem statement, problem solving clues, etc., to ensure that the template covers all key information points, make mathematical graphics, and use Mathematica to develop drawing guidelines, including color coding, annotation style, scale selection, etc., to ensure that all graphics meet educational standards and readability principles. Each graphic must be accompanied by a clear legend and necessary text description. When converting formulas to pictures, use the LaTeX online conversion tool to set a unified image output format (such as SVG or PNG), resolution, and color mode to ensure consistency and high quality.

[0027] Write synthetic speech scripts and use TTS scripts to develop style guidelines, emphasizing the educational and comprehensible nature of the language, paying attention to the arrangement of speech speed, pauses, and stress, record samples and conduct user testing, and adjust speech speed and clarity based on feedback.

[0028] By listing the types of integer operations, fraction operations, and decimal operations, we screened out the key knowledge points in the fields of algebra, geometry, probability statistics, and calculus, voted on the issues in each field, determined the priority of each field based on the number of votes, used LaTeX text formulas, established a sample library of formula texts with standard writing standards, evaluated the applicability of Mathematica software in generating mathematical images, unified the drawing standards and templates for geometric figures and function images, investigated the existing speech recognition and synthesis technologies, and used speech synthesis software TTS to write voice scripts.

[0029] The difficulty level of mathematical problems can be judged based on the experience of professionals in the field. The difficulty coefficient calculation method is used in combination with historical data to test the effectiveness of the difficulty index model. A preliminary adaptive algorithm model is built based on the machine learning framework TensorFlow. The effect of the algorithm adjusting the difficulty is tested in a simulation environment. According to the difficulty index, the number of each difficulty problem is divided into different difficulty levels from the existing database.

[0030] The average comprehension speed is 60 words per minute.

[0031] S3, using the difficulty coefficient model to adjust the difficulty and diversity of mathematical problems in the data set in real time; According to the difficulty adjustment strategy, by setting the basic data set, the accuracy, processing time, problem complexity, diversity index, innovation index and stability index in the model training process are collected in real time to calculate the current difficulty coefficient. The expression is: , ; in, A comprehensive indicator that indicates student ability feedback, including knowledge point mastery rate , Problem-solving speed and answer accuracy , , , are weight factors, corresponding to the relative importance of knowledge point mastery rate, problem-solving speed, and answer accuracy in the comprehensive index, D t Table current difficulty coefficient, A t represents the accuracy of the model at time t, β represents the factor that adjusts the diversity V, V t represents the diversity index of problem solving, γ represents the factor that regulates the impact of innovation I on difficulty, and I t Indicator representing the innovativeness of problem solving, T t represents the average time required to solve the problem, λ represents the constant for adjusting the processing time, and C t represents the problem complexity score, ω t represents the dynamic question type weight factor, W represents the question type weight, and η represents the constant contributing to the difficulty of stability; Based on the current difficulty coefficient, the polynomial regression algorithm is used to calculate the target difficulty coefficient. The expression is: ; Among them, D ta represents the target difficulty coefficient, f represents a predefined function, and D t -1 indicates the difficulty coefficient of the previous time step, A t -1 indicates the accuracy of the model at the previous time point, represents the time interval, θ represents the parameter of the adjustment function; Based on the deviation between the target difficulty coefficient and the current difficulty coefficient, combined with the knowledge coverage, problem-solving speed and accuracy of the question type, a reinforcement learning method is used to optimize question selection and dynamically adjust question distribution.

[0032] Polynomial regression algorithm, using cross-validation to determine the best polynomial order to avoid overfitting or underfitting, and using least squares or other optimization algorithms to estimate the coefficients of polynomial regression; Calculate the target difficulty coefficient and define the specific form of the function f. For example, it can be a linear regression, a quadratic polynomial, or a higher-order polynomial, depending on the trend and complexity of the historical data. Specify how to choose the parameter θ, which may involve hyperparameter tuning and performance evaluation on the validation set. Reinforcement learning method, define the state space of reinforcement learning, including the current difficulty coefficient, indicators for solving various types of problems including accuracy and problem-solving speed, actions that can be taken, increase the proportion of a certain type of problem, adjust the difficulty of specific problems, design a reward mechanism, encourage the model to choose a combination of problems that can improve learning efficiency and learning outcomes, select the reinforcement learning algorithm Q-learning, and adjust and optimize according to the type of problem; Dynamic question type weight factor ω t And the update rules for the question type weight W, design a set of rules or algorithms to dynamically adjust these two weight factors, which may be based on the performance of historical data, student feedback or learning effectiveness analysis. You can consider using online learning methods, such as online gradient descent, to instantly adjust the weights according to the new data after each learning interaction.

[0033] Compared with the prior art, the dynamic adjustment capability of the method of the present invention is significantly enhanced: dynamic adjustment is performed based on multi-dimensional parameters and real-time feedback, while the prior art lacks a dynamic adjustment mechanism; The formula is clear and comprehensive: It provides a complex mathematical formula that dynamically optimizes the difficulty of the dataset by combining model performance and student ability feedback; Data generation and multimodal expansion: The diversity and applicability of the dataset are significantly improved through random generation functions and multimodal expansion methods; Improve model performance: Through dynamic adjustment and multimodal expansion, the model's learning efficiency and reasoning ability in the field of mathematical computing are significantly improved.

[0034] S4. Based on the basic mathematical paradigm, use random functions to generate mathematical problem instances and adjust the ratio of problem instances of different difficulty levels in real time; By setting the basic paradigm of mathematical problems as P={P1,P2…,P n}, a composite function is used to adjust the difficulty dynamics, the expression is: , in, represents the value of the composite difficulty function, i, j, k represent the distinction between different mathematical paradigms, difficulty levels and other classifications, x represents the average accuracy of the model on this paradigm, y = (S, T) represents a vector, S represents the student ability prediction index, T represents time sensitivity, Indicates the parameters related to adjusting the model performance A, Indicates the size of adjusting the student's ability S in the difficulty calculation, represents the weight of controlling time sensitivity T, ζ represents the weight of the interaction term affecting model performance A and time T, A bias value that represents a dynamic balance between mathematical paradigm, difficulty level, and problem classification, Represents diversity, express The degree of influence of nonlinearity in the difficulty calculation; According to each paradigm P i , construct a multi-level random generator, combining polynomial, trigonometric and logarithmic functions, and calculate mathematical problem examples. The expression is: , in, represents the value of the nonlinear function, a, b, c, d represent the random parameters used when generating the problem through Gaussian distribution, Represents the unknowns in the problem instance; according to Based on the calculation results, an adaptive grouping algorithm is used to dynamically allocate the number of instances of different difficulty levels, and multimodal transformation is performed on each generated mathematical instance.

[0035] In the composite difficulty function The parameter values ​​are α, β, γ, η, ζ, θ ijk Provide recommended initial values ​​or methods for determining parameters such as those obtained through experimental calibration, expert evaluation, or analysis of historical data, and describe how these parameters will be adjusted over time to accommodate changes in student progress and learning needs; Quantification method of student ability prediction index S and time sensitivity T, define the specific calculation method of student ability prediction index S, which may involve students' past grades, homework completion speed, test performance, etc., clarify how time sensitivity T reflects the changes in students' learning efficiency in a specific time period, which may require collecting students' learning behavior data in different time periods; Design an adaptive grouping algorithm based on Based on the output of the test, dynamically adjust the ratio of questions of different difficulty levels, which may include setting difficulty thresholds, dynamically dividing difficulty intervals according to students' current abilities, and adjusting interval boundaries according to learning progress.

[0036] S5, by performing multi-modal expansion on basic input commands, each mathematical problem instance is converted into a command form of different modalities; Combining preset Chinese commands and standard mathematical expressions, using context analysis technology (such as BERT or GPT models), dynamically generate a variety of input command expressions, including synonym replacement, sentence changes, and context supplements, to improve the naturalness and diversity of commands; By applying image conversion algorithms to each basic mathematical paradigm, we construct diverse image instructions for each problem instance and dynamically generate image annotations based on the semantic content of the problem. Use TTS technology to convert text modalities into voice commands, record voice commands with different accents, speaking speeds, and intonations, integrate text, images, and voice commands for each math problem instance, and create a complete set of multimodal command sets; The dynamic input generation mechanism driven by context analysis technology improves the flexibility and naturalness of multimodal expansion; For synonym replacement and sentence structure changes, formulate a set of rules or use natural language processing technology (such as syntactic analysis, vocabulary network) to generate variants of text commands. For example, you can use a synonym library to replace keywords and use template sentences to rearrange sentence structures, keeping the semantics unchanged while increasing diversity. Algorithms for converting basic mathematical paradigms to images: select or develop image generation algorithms suitable for different mathematical paradigms. For example, for geometric problems, you can use function drawing software or AI painting API to generate graphics; for algebraic problems, design algorithms to convert equations into visual flowcharts or logic diagrams, and clarify the selection basis, input parameters, and expected output format of the algorithm; Selection and configuration parameters of TTS technology, select a suitable TTS engine, consider its voice naturalness, language and dialect support, set parameters such as timbre (children, adults), speech speed (fast, medium, slow), pitch (smooth, fluctuating), etc., to suit students of different ages and learning preferences, and ensure that the TTS output is semantically consistent with the original text command and is legible; To record voice commands with multiple accents, pitches, and speech speeds, if real human voices are used, readers with multiple accents need to be organized and recorded separately according to the set parameters. If synthesis technology is used, different accent models are set in the TTS engine, and pitch curves and speech speed parameters are adjusted to generate diverse voice files.

[0037] This embodiment also provides an intelligent data set system for advanced mathematical problems based on multimodal interaction, including: Requirements analysis and framework design module: Analyze the data set requirements, determine the type of mathematical operations, multimodal expressions, and dynamic difficulty adjustment strategies; Multimodal expression generation module: Based on demand analysis, it constructs basic mathematical paradigms and converts them into different expression forms using multimodal templates; Difficulty dynamic adjustment module: adopts difficulty coefficient model to adjust the difficulty and diversity of mathematical problems in the data set in real time; Example generation and management module: Based on the constructed basic mathematical paradigm, it uses random functions to generate mathematical problem instances and adjusts the ratio of problem instances of different difficulty levels in real time; Multimodal instruction conversion module: By performing multimodal expansion on basic input commands, each mathematical problem instance is converted into a command form of different modes.

[0038] This embodiment also provides a computer device suitable for the case of a large model dataset construction method, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the large model dataset construction method proposed in the above embodiment.

[0039] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0040] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for constructing a large model data set proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0041] Example 2

[0042] Referring to Table 1, which is the second embodiment of the present invention, in order to further verify the advancement of the present invention, experimental simulation data of a large model data set construction method are provided.

[0043] Four typical large models (named A, B, C, and D) were selected as experimental objects. A represents the GPT-4 model, which is a generative pre-trained transformation model developed by OpenAI. B represents the BERT model, which is a model developed by Google for natural language understanding. C represents the T5 model, which is a framework developed by Google that unifies the processing methods of various NLP tasks and converts all tasks into text generation problems. D represents the DALL-E model, which is a generative model developed by OpenAI. These models are representative and different in the field of mathematical processing. The performance differences between them after using traditional data sets and data sets constructed by applying the method of the present invention are compared, especially in solving advanced mathematical problems and multimodal understanding.

[0044] First, according to the method described in the present invention, we conducted a detailed analysis of the existing capabilities of large models and determined their strengths and weaknesses in three aspects: basic operations, advanced mathematical problems, and multimodal understanding. Based on this, we selected ten typical problems in four fields: algebra, geometry, probability statistics, and calculus. Each problem was designed with five difficulty levels from basic to advanced, with a total of fifty problem templates, based on which we performed multimodal conversion and dynamic difficulty adjustment.

[0045] Next, we used tools such as LaTeX and Mathematica to create a high-quality mathematical formula text and image resource library, and used TTS technology to generate a variety of voice commands. Through the difficulty coefficient model, the difficulty of the questions is dynamically adjusted according to the real-time performance of the model, ensuring that each model can be challenged and continuously improve during the training process.

[0046] Finally, the present invention uses random functions to generate mathematical problem instances with different difficulty and diversity, and achieves a reasonable distribution of instance difficulty through an adaptive grouping algorithm. Specifically, as shown in Table 1:

[0047] Table 1 Experimental record table

[0048] By comparing the data before and after the improvement, it can be clearly observed that after using the data set constructed by the present invention, the accuracy of all models in basic operations, advanced mathematical problems, and multimodal understanding has been significantly improved, and the average problem-solving time has also been reduced. Taking A as an example, its basic operation accuracy increased from 0.92 to 0.94, the accuracy of advanced mathematical problems increased from 0.78 to 0.85, the accuracy of multimodal understanding increased from 0.65 to 0.75, and the average problem-solving time was shortened from 15 seconds to 11 seconds. This directly reflects the remarkable effect of the present invention in promoting model learning efficiency and improving the model's ability to handle complex tasks.

[0049] In particular, the problem-solving diversity, innovation, and stability indicators have all increased, which proves the effectiveness of the difficulty coefficient model and dynamic adjustment strategy, ensuring the challenge, diversity, and adaptability of the problem set, avoiding model overfitting, and improving generalization ability. For example, the solution diversity of D increased from 2.0 to 2.4, and the innovation increased from 1.1 to 1.4, indicating that the model's ability to handle a wider range of and novel problems has been enhanced.

[0050] In summary, the method of the present invention not only successfully improves the performance of the model in mathematical operations, multimodal understanding and problem-solving speed, but also significantly enhances the model's ability to handle complex and diverse problems, bringing substantial innovation and improvement to the model's comprehensive learning efficiency and application breadth. This series of improvements fully demonstrates the important value of the present invention in promoting the application of AI in mathematics and other fields.

[0051] The above description is only a preferred embodiment of the present invention and is not intended to be a further limitation of the present invention. All equivalent changes made using the contents of the present specification and drawings are within the protection scope of the present invention.

Claims

1. A method for constructing an intelligent data set for advanced mathematical problems based on multimodal interaction, characterized in that: The following steps are involved: Step 1: Collect basic operation problems, advanced mathematics problems and multimodal understanding problems to form an intelligent education data set, analyze the requirements of the intelligent education data set, determine the mathematical operation type, multimodal expression form and dynamic difficulty adjustment strategy; Step 2: Based on the demand analysis, construct the basic mathematical paradigm and convert it into different expressions using multimodal templates; Step 3: Use the difficulty coefficient model to adjust the difficulty and diversity of math problems in the smart education dataset in real time; specifically: According to the difficulty adjustment strategy, by setting the basic data set, the accuracy, processing time, problem complexity, diversity index, innovation index and stability index in the model training process are collected in real time to calculate the current difficulty coefficient. The expression is: ， ； in, A comprehensive indicator that indicates student ability feedback, including knowledge point mastery rate , Problem-solving speed and answer accuracy , , , are weight factors, corresponding to the relative importance of knowledge point mastery rate, problem-solving speed, and answer accuracy in the comprehensive index; D t Indicates the current difficulty coefficient, A t represents the accuracy of the model at time t, β represents the factor that adjusts the diversity V, V t represents the diversity index of problem solving, γ represents the factor that regulates the impact of innovation I on difficulty, and I t Indicator representing the innovativeness of problem solving, T t represents the average time required to solve the problem, λ represents the constant for adjusting the processing time, and C t represents the problem complexity score, ω t represents the dynamic question type weight factor, W represents the question type weight, and η represents the constant contributing to the difficulty of stability; Based on the current difficulty coefficient, the polynomial regression algorithm is used to calculate the target difficulty coefficient. The expression is: ； Among them, D ta represents the target difficulty coefficient, f represents a predefined function, and D t -1 indicates the difficulty coefficient of the previous time step, A t -1 indicates the accuracy of the model at the previous time point, represents the time interval, θ represents the parameter of the adjustment function; Step 4: Based on the basic mathematical paradigm, use random functions to generate mathematical problem instances, and adjust the ratio of problem instances of different difficulty levels in real time; Step 5: Convert each mathematical problem instance into a command form of different modes by performing multimodal expansion on the basic input commands.

2. The method for constructing an intelligent data set for advanced mathematical problems based on multimodal interaction according to claim 1, characterized in that: In step 1, the basic operation problems, advanced mathematics problems and multimodal understanding problems are collected to form an intelligent education data set, specifically: Collect basic operation problems of addition, subtraction, multiplication and division, advanced mathematics problems of algebra, geometry, probability statistics and calculus, and multimodal understanding problems of text, image and voice through online education platforms; Query existing models for mathematical problems in the field of advanced mathematics, and select the most viewed advanced mathematics problems; Analyze the successes and failures of existing models in handling these cases.

3. The method for constructing an intelligent data set for advanced mathematical problems based on multimodal interaction according to claim 2 is characterized in that: In step 1, the requirements of the intelligent education data set are analyzed to determine the mathematical operation type and multimodal expression form, specifically: By listing integer operations, fraction operations, and decimal operations, we can filter out key knowledge points in algebra, geometry, probability statistics, and calculus. Vote on key issues in each area and determine the priority of each area based on the number of votes; Use LaTeX text formulas to establish a sample library of formula texts with standard writing standards; Use Mathematica software to generate mathematical images and unify the drawing standards and templates for geometric figures and function images; We investigated existing speech recognition and synthesis technologies through questionnaires, and used speech synthesis software TTS to write speech scripts covering various mathematical expressions. The dynamic difficulty adjustment strategy is specifically as follows: According to the complexity and abstractness of mathematical problems, the difficulty coefficient calculation method is adopted, and the validity of the difficulty index model is tested in combination with historical data; Build a preliminary adaptive algorithm model based on the machine learning framework TensorFlow, and test the effect of algorithm adjustment difficulty in a simulated environment; According to the difficulty index, the number of each difficulty question is divided into different difficulty levels from the existing database.

4. The method for constructing an intelligent data set for advanced mathematical problems based on multimodal interaction according to claim 3 is characterized in that: In step 2, based on the demand analysis, a basic mathematical paradigm is constructed and converted into different expressions using a multimodal template, specifically: By sorting out the types of mathematical operations at each level, they are divided into subcategories: basic operations, algebraic operations, geometric operations, functions, and calculus. For each subcategory, multiple sample questions of different difficulty levels are designed that cover all core mathematical knowledge points. Through the hierarchical derivation and verification mechanism, complex mathematical derivation problems are decomposed into multiple sub-steps, each of which clearly defines the logic and key steps of the problem; Verify each sub-step step by step, and build a hierarchical verification framework to record the correctness verification of each step in the derivation process to form a complete derivation path; Generate variant problems by changing values, variables, and structural elements. Write a problem-solving step-by-step description document for each type of problem to clarify the problem-solving logic and key steps. Use LaTeX to format formulas and create a mathematical formula library that is easy to search for basic operations to advanced functions; use text description templates and write specific text descriptions for each example problem based on the framework; Based on the content of the questions, we use Mathematica to draw clear and accurately labeled mathematical graphics, use LaTeX online conversion tools to convert mathematical formulas into high-definition pictures, write voice scripts for each mathematical problem, and use TTS to synthesize clear and fast speech that is suitable for the average comprehension speed of advanced mathematics students.

5. The method for constructing an intelligent data set for advanced mathematical problems based on multimodal interaction according to claim 4 is characterized in that: Step 4 is as follows: By setting the basic paradigm of mathematical problems as P={P1,P2…,P n }, a composite function is used to adjust the difficulty dynamics, the expression is: ， in, represents the value of the composite difficulty function, i, j, k represent the distinction between different mathematical paradigms, difficulty levels and other classifications, x represents the average accuracy of the model on this paradigm, y = (S, T) represents a vector, S represents the student ability prediction index, T represents time sensitivity, Indicates the parameters related to adjusting the model performance A, Indicates the adjustment of the student's ability S in the difficulty calculation. represents the weight of controlling time sensitivity T, ζ represents the weight of the interaction term affecting model performance A and time T, represents a bias value that dynamically balances mathematical paradigms, difficulty levels, and problem classifications, Represents diversity, express The degree of influence of nonlinearity in the difficulty calculation; According to each paradigm P i , construct a multi-level random generator, combining polynomial, trigonometric and logarithmic functions, and calculate mathematical problem examples. The expression is: ， in, represents the value of the nonlinear function, a, b, c, d represent the random parameters used when generating the problem through Gaussian distribution, Represents the unknowns in the problem instance; according to Based on the calculation results, an adaptive grouping algorithm is used to dynamically allocate the number of instances of different difficulty levels, and multimodal transformation is performed on each generated mathematical instance.

6. The method for constructing an intelligent data set for advanced mathematical problems based on multimodal interaction according to claim 5, characterized in that: In step 5, each mathematical problem instance is converted into a command form of different modes by performing multi-modal expansion on the basic input command, specifically: Combining preset Chinese commands and standard mathematical expressions, using context analysis technology, dynamically generates a variety of input command expressions, including synonym replacement, sentence changes and context supplements, to improve the naturalness and diversity of commands; By applying image conversion algorithms to each basic mathematical paradigm, we construct diverse image instructions for each problem instance and dynamically generate image annotations based on the semantic content of the problem. Use TTS technology to convert text modality into voice commands, record voice commands with different accents, speaking speeds, and intonations, integrate text, images, and voice instructions for each math problem instance, and create a complete set of multimodal command sets.

7. An intelligent data set system for advanced mathematical problems based on multimodal interaction, characterized by: The method for implementing any one of claims 1 to 6 comprises: Requirements analysis and framework design module: Analyze the data set requirements, determine the type of mathematical operations, multimodal expressions, and dynamic difficulty adjustment strategies; Multimodal expression generation module: Based on demand analysis, it constructs basic mathematical paradigms and converts them into different expression forms using multimodal templates; Difficulty dynamic adjustment module: adopts difficulty coefficient model to adjust the difficulty and diversity of mathematical problems in the data set in real time; Example generation and management module: Based on the constructed basic mathematical paradigm, it uses random functions to generate mathematical problem instances and adjusts the ratio of problem instances of different difficulty levels in real time; Multimodal instruction conversion module: By performing multimodal expansion on basic input commands, each mathematical problem instance is converted into a command form of different modes.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the construction method described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the construction method described in any one of claims 1 to 6 are implemented.

Images