Mathematical description language large model building method, medium and system

By constructing a multi-level mathematical logical reasoning network model and dynamically adjusting the video memory allocation based on the contribution value evaluation model, the problem of low computing resource utilization caused by unreasonable video memory allocation during the training of the mathematical description language in the existing technology is solved, and efficient computing resource utilization and model prediction accuracy are achieved.

CN120181223AActive Publication Date: 2025-06-20WUHAN TECHN COLLEGE OF COMM
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202510228132.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-02-28
Publication Date
2025-06-20
Estimated Expiration
2045-02-28

AI Technical Summary

Technical Problem

In the prior art, the computing resource utilization rate is low due to unreasonable video memory allocation during the training of the mathematical description language.

Method used

By constructing a mathematical logical reasoning network model that includes semantic understanding layer, mathematical symbol mapping layer, logical relationship conversion layer, mathematical unit organization layer and unit relationship construction layer, and dynamically adjusts the memory allocation strategy, calculation thread allocation and training batch settings based on the contribution value evaluation model.

Benefits of technology

It realizes efficient utilization of computing resources, improves model training efficiency, ensures model prediction accuracy, and solves the problem of low utilization of video memory resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120181223A_ABST
    Figure CN120181223A_ABST
Patent Text Reader

Abstract

The invention provides a mathematical description language large model establishment method, medium and system, and belongs to the technical field of computer models.The method comprises the steps that firstly, corpus information is obtained by constructing a training data set, then word segmentation and feature extraction are conducted on a natural language description text, and an imprecise description matrix is constructed; and performing structured processing on the standard mathematical description language text to form an accurate description matrix. On the basis, a video memory pool is established, training resources are pre-allocated, and a multi-level mathematical logic reasoning network model is constructed. Through contribution value evaluation and accuracy calculation, resource optimization allocation based on importance is realized, and efficient utilization of resources is ensured by adopting a video memory pool management mechanism. According to the whole scheme, efficient training of the mathematical description language conversion model is achieved through a quantitative evaluation mechanism and a dynamic optimization strategy, and the technical problem that in the prior art, in the mathematical description language large model training process, due to unreasonable video memory allocation, the computing resource utilization rate is low is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer models, and more particularly, relates to a method, medium, and system for establishing a large model of a mathematical description language. Background Art

[0002] In the fields of artificial intelligence and natural language processing, converting natural language descriptions into standard mathematical description languages is an important task, which is of great significance for intelligent education, intelligent tutoring, and scientific research. Traditional methods for converting mathematical description languages mainly include rule-based methods and statistics-based methods. The rule-based method maps natural language to mathematical expressions by manually designing grammar rules and conversion templates. This method requires a large amount of manual rule writing and is difficult to handle complex mathematical concepts and logical relationships. The statistics-based method uses machine learning algorithms to learn conversion models from large-scale corpora, including methods such as conditional random fields and support vector machines. Although such methods have a certain degree of adaptability, they have high requirements for the quality and quantity of training data and are difficult to capture deep semantic information.

[0003] With the development of deep learning technology, neural network-based mathematical description language conversion models have gradually become a research hotspot. Such methods achieve end-to-end conversion from natural language to mathematical descriptions by constructing deep neural networks. However, the training process of deep learning models requires a large amount of computing resources, especially video memory resources. Existing training methods usually adopt a fixed video memory allocation strategy, without considering the differences in the importance of each layer of the model, resulting in low utilization efficiency of video memory resources. At the same time, due to the strict grammar rules and logical constraints of mathematical description languages, the model needs to handle multiple tasks such as semantic understanding, symbol mapping, and logical relationship construction simultaneously, and the resource requirements for these tasks vary significantly.

[0004] Current video memory management methods mainly include two categories: static allocation and dynamic allocation. The static allocation method pre-allocates a fixed amount of video memory for each layer of the model. Although it is simple to implement, it is difficult to adapt to changes in resource requirements during training, easily causing video memory waste or insufficiency. The dynamic allocation method adjusts video memory allocation according to demand during training, but frequent video memory application and release operations will bring additional computational overhead and affect training efficiency. In addition, existing methods lack a quantitative evaluation mechanism for the contribution of each layer of the model, and cannot achieve importance-based resource optimization allocation, making it difficult to solve the problem of low utilization efficiency of video memory resources. That is to say, there is a technical problem in the prior art that the utilization efficiency of computing resources is low due to unreasonable video memory allocation during the training process of large models of mathematical description languages. Summary of the Invention

[0005] In view of this, the present invention provides a method, medium and system for establishing a large model of mathematical description language, which can solve the technical problem of low utilization rate of computing resources caused by unreasonable video memory allocation during the training process of the large model of mathematical description language in the prior art.

[0006] The present invention is implemented as follows: In the first aspect of the present invention, a method for establishing a large model of mathematical description language includes the following steps: obtaining natural language description texts and their corresponding standard mathematical description language texts in a training corpus to establish a training data set; establishing a video memory pool and pre-allocating video memory blocks required for training, dividing the video memory blocks into basic video memory blocks and dynamic video memory blocks, where the basic video memory blocks are used to store fixed parameters, and the dynamic video memory blocks are used to store intermediate calculation results; constructing a mathematical logic reasoning network model including a semantic understanding layer, a mathematical symbol mapping layer, a logical relationship conversion layer, a mathematical unit organization layer, and a unit relationship construction layer; establishing a unit contribution value evaluation model based on the accuracy score of the verification prediction result, and using the gradient backpropagation algorithm to calculate the contribution weights of each layer; determining the video memory allocation amount, the number of calculation threads, and the number of training batches according to the contribution weights calculated by the unit contribution value evaluation model; optimizing the video memory allocation of the video memory pool according to the video memory allocation amount until the accuracy score of the verification prediction result reaches a preset accuracy threshold.

[0007] Among them, the step of establishing the training data set is specifically to construct a training corpus by establishing a mathematical knowledge graph, extract mathematical concepts, theorems, formulas, and problem-solving methods from mathematics textbooks, papers, and professional literature, construct a set of mathematical knowledge nodes, establish inclusion relationships, inheritance relationships, combination relationships, and conversion relationships between mathematical knowledge nodes, and perform structured storage on the mathematical knowledge nodes and their associated relationships.

[0008] Among them, the preset accuracy threshold includes a preset semantic accuracy threshold, a preset symbol accuracy threshold, a preset structure accuracy threshold, and a preset logical accuracy threshold, where the preset semantic accuracy threshold is 0.85, the preset symbol accuracy threshold is 0.9, the preset structure accuracy threshold is 0.8, and the preset logical accuracy threshold is 0.85.

[0009] Among them, the semantic understanding layer uses a bidirectional long short-term memory network to extract the context semantic information of the text, the mathematical symbol mapping layer uses an attention mechanism to realize the mapping conversion from semantic features to mathematical symbols, the logical relationship conversion layer uses a graph neural network to establish the logical relationship between mathematical symbols, the mathematical unit organization layer uses a recurrent neural network to organize mathematical logic units, and the unit relationship construction layer uses a tree-shaped recurrent neural network to establish the hierarchical relationship between mathematical logic units.

[0010] Among them, the total size of the video memory block is determined according to the size of the imprecise description matrix, the size of the precise description matrix, and the preset storage requirements for intermediate calculation results. The basic video memory block accounts for 60% of the total video memory, and the dynamic video memory block accounts for 40% of the total video memory.

[0011] Among them, the method for determining the contribution weight is that the semantic understanding layer mainly affects semantic accuracy, the mathematical symbol mapping layer mainly affects symbol accuracy, the logical relationship conversion layer mainly affects logical accuracy, and the mathematical unit organization layer and the unit relationship construction layer mainly affect structural accuracy.

[0012] Among them, the method for determining the video memory allocation amount is that the greater the contribution weight, the more the video memory allocation amount, and the specific ratio is the square root of the contribution weight; the method for determining the number of calculation threads is that the greater the contribution weight, the more the allocated calculation threads, and the specific number is the basic number of threads multiplied by the contribution weight; the method for determining the number of training batches is that the greater the contribution weight, the more the training batches, and the specific number of batches is the basic number of batches multiplied by the contribution weight.

[0013] Among them, the optimization of video memory allocation includes merging the free video memory in the video memory block, recycling the idle video memory in the video memory block, reallocating the priority of the video memory block according to the contribution weight, and updating the usage status of the video memory block, where the usage status includes usage time, access frequency, and idle ratio.

[0014] The second aspect of the present invention provides a computer-readable storage medium, in which program instructions are stored. When the program instructions run on a computer, they are used to execute the above-mentioned method for establishing a mathematical description language large model.

[0015] The third aspect of the present invention provides a system for establishing a mathematical description language large model, including the above-mentioned computer-readable storage medium. The system can be any one of a computer, a server, and a single-chip microcomputer. The computer-readable storage medium is set inside the system, and a microprocessor for executing the program instructions stored in the computer-readable storage medium is set inside the system.

[0016] Compared with the prior art, the present invention provides a method, medium, and system for establishing a mathematical description language large model. The method for establishing a mathematical description language large model proposed by the present invention realizes the efficient utilization of computing resources by constructing a mathematical logical reasoning network model including a semantic understanding layer, a mathematical symbol mapping layer, a logical relationship conversion layer, a mathematical unit organization layer, and a unit relationship construction layer, combined with a video memory optimization allocation strategy based on contribution values.

[0017] This method first establishes a unit contribution value evaluation model and uses the gradient backpropagation algorithm to quantitatively evaluate the influence degree of each network layer on the model performance. Then, based on the contribution weights, it dynamically adjusts the video memory allocation strategy, computing thread allocation, and training batch settings. Through the video memory pool management mechanism, dynamic scheduling and optimized allocation of video memory resources are achieved, effectively avoiding video memory fragmentation and resource waste. At the same time, this method adopts a multi-level model structure design. For the characteristics of the mathematical description language, special processing modules are respectively constructed, and through reasonable resource allocation, the computing efficiency of each module is ensured.

[0018] The solution of the present invention combines video memory allocation with the model performance contribution degree, establishes a quantitative decision-making mechanism for resource allocation, and solves the problem of low resource utilization efficiency caused by traditional fixed allocation or simple dynamic allocation methods. At the same time, through the overall management and optimized scheduling of the video memory pool, efficient utilization of video memory resources is achieved, and the model training efficiency is improved. Based on the strategies of accuracy evaluation and incremental training, the prediction accuracy of the model is ensured, providing reliable technical support for the mathematical description language conversion task. Brief Description of the Drawings

[0019] Figure 1 It is a flowchart of the method of the present invention. Detailed Embodiments

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0021] As Figure 1 shown, it is a flowchart of a method for establishing a large mathematical description language model provided by the first aspect of the present invention. This method includes the following steps:

[0022] S01. Obtain the natural language description text and its corresponding standard mathematical description language text in the training corpus, and establish a training dataset. The training dataset includes the natural language description text, the standard mathematical description language text, and a mathematical logic unit structure. The mathematical logic unit structure includes a set of mathematical symbols, a set of mathematical functions, a set of mathematical operations, a set of mathematical relations, and a set of logical inference rules;

[0023] S02. Perform word segmentation on the natural language description text, extract semantic features, numerical features, and logical relationship features, and construct an imprecise description matrix. The imprecise description matrix is composed of a semantic vector matrix, a numerical feature matrix, and a relationship feature matrix. The semantic vector matrix is used to represent mathematical concepts in natural language, the numerical feature matrix is used to represent numerical information in natural language, and the relationship feature matrix is used to represent logical relationships in natural language;

[0024] S03. Structurally process the text of the standard mathematical description language, extract mathematical symbols, operation symbols, and mathematical logic units, and construct an accurate description matrix. The accurate description matrix includes a mathematical symbol matrix, an operation symbol matrix, and a mathematical logic unit matrix. The mathematical logic unit matrix is used to store the hierarchical relationship and call relationship between mathematical logic units;

[0025] S04. Establish a video memory pool and pre-allocate video memory blocks required for training. Determine the total size of the video memory blocks according to the size of the inaccurate description matrix, the size of the accurate description matrix, and the preset storage requirements for intermediate calculation results. Divide the video memory blocks into basic video memory blocks and dynamic video memory blocks. The basic video memory blocks are used to store the inaccurate description matrix, the accurate description matrix, and fixed parameters, and the dynamic video memory blocks are used to store intermediate calculation results;

[0026] S05. Construct a mathematical logic reasoning network model. The mathematical logic reasoning network model includes a semantic understanding layer, a mathematical symbol mapping layer, a logical relationship conversion layer, a mathematical unit organization layer, and a unit relationship construction layer;

[0027] S06. Input the inaccurate description matrix into the mathematical logic reasoning network model. Convert the semantic vector matrix into a mathematical concept representation through the semantic understanding layer, map the mathematical concept representation into a mathematical symbol sequence through the mathematical symbol mapping layer, construct the logical relationship between the mathematical symbol sequences through the logical relationship conversion layer, organize the mathematical symbol sequences and the logical relationship into mathematical logic units through the mathematical unit organization layer, and establish the hierarchical relationship and call relationship between the mathematical logic units through the unit relationship construction layer to generate a mathematical logic unit tree structure and a predicted mathematical description text;

[0028] S07. Calculate the accuracy score between the predicted mathematical description text and the text of the standard mathematical description language. The accuracy score includes a semantic accuracy score, a symbol accuracy score, a structure accuracy score, and a logical accuracy score. Optimize and train the mathematical logic reasoning network model according to the accuracy score;

[0029] S08. Obtain the natural language description text in the validation dataset, and repeat steps S02 and S06 to obtain a validation prediction result;

[0030] S09. Calculate the accuracy score of the validation prediction result. When the accuracy score of the validation prediction result is lower than the preset accuracy threshold, add the validation dataset to the training dataset for incremental training. The preset accuracy threshold includes a preset semantic accuracy threshold, a preset symbol accuracy threshold, a preset structure accuracy threshold, and a preset logical accuracy threshold;

[0031] S10, establishing a unit contribution value evaluation model based on the accuracy score of the verification prediction result, and using a gradient back propagation algorithm to calculate the contribution weights of the semantic understanding layer, the mathematical symbol mapping layer, the logical relationship conversion layer, the mathematical unit organization layer, and the unit relationship construction layer;

[0032] S11, determining the amount of video memory allocated, the number of computing threads, and the number of training batches for the semantic understanding layer, the mathematical symbol mapping layer, the logical relationship conversion layer, the mathematical unit organization layer, and the unit relationship construction layer according to the contribution weights calculated by the unit contribution value evaluation model;

[0033] S12, optimizing the video memory allocation of the video memory pool according to the video memory allocation amount, wherein the video memory allocation optimization includes merging the idle video memory in the video memory block, recycling the idle video memory in the video memory block, reallocating the priority of the video memory block according to the contribution weight, and updating the usage status of the video memory block, until the accuracy score of the verification prediction result reaches the preset accuracy threshold, thereby obtaining a trained mathematical description language model;

[0034] Among them, the mathematical logic unit tree structure includes a unit type identifier, input parameter information, output result information, calling method information and constraint information, the unit type identifier is used to identify the functional category of the mathematical logic unit, the input parameter information is used to receive data input, the output result information is used to return the calculation result, the calling method information is used to define the interactive interface between the mathematical logic units, and the constraint information is used to limit the value range of the input parameter information.

[0035] The specific implementation methods of the above steps are described in detail below. The specific implementation method of step S01 is to construct a training corpus by establishing a mathematical knowledge graph. First, mathematical concepts, theorems, formulas and problem-solving methods are extracted from mathematical textbooks, papers and professional literature to construct a set of mathematical knowledge nodes; then the association relationship between mathematical knowledge nodes is established, including the inclusion relationship, inheritance relationship, combination relationship and conversion relationship between concepts; then the mathematical knowledge nodes and their association relationships are structured and stored to form a mathematical knowledge graph; based on the mathematical knowledge graph, mathematical problems described in natural language and their standard mathematical expressions are collected from network resources, teaching materials and test question banks to establish an initial corpus; the initial corpus is cleaned to remove duplicate samples, incomplete samples and erroneous samples; finally, the cleaned corpus is divided into a training set and a validation set in a ratio of 8:2, the training set is used for model training, and the validation set is used for model evaluation. The purpose of this step is to construct a high-quality training data set to provide a reliable data foundation for subsequent model training.

[0036] The specific implementation of step S02 is as follows: First, the jieba tokenization tool is used to tokenize the natural language description text to obtain a sequence of words. Then, the pre-trained word vector model word2vec is used to extract the semantic features of the word sequence, map each word to a 300-dimensional word vector, and construct a semantic vector matrix. Next, regular expressions are used to extract numerical information in the text, including integers, decimals, fractions, percentages, etc., and a numerical feature matrix is constructed after normalizing the numerical information. Finally, dependency parsing technology is used to extract logical relationships in the word sequence, including causal relationships, conditional relationships, parallel relationships, etc., and the logical relationships are encoded into a relationship feature matrix. The role of this step is to convert the natural language description text into a feature representation that can be processed by a computer, providing input for the subsequent conversion of mathematical descriptions.

[0037] The specific implementation of step S03 is as follows: First, lexical analysis is performed on the standard mathematical description language text to identify mathematical symbols, operation symbols, keywords, and identifiers. Then, syntactic analysis is carried out to construct an abstract syntax tree representing the hierarchical structure of the mathematical expression. Next, semantic analysis is performed to extract the dependency relationships and call relationships between mathematical logic units. Finally, the analysis results are organized into an accurate description matrix, where the mathematical symbol matrix stores symbol types and attribute information, the operation symbol matrix stores operation precedence and associativity information, and the mathematical logic unit matrix stores the hierarchical relationships and call relationships between units. The purpose of this step is to convert the standard mathematical description language text into a structured matrix representation, providing the target output for model training.

[0038] The specific implementation of step S04 is as follows: First, estimate the video memory requirements during the training process according to the size of the training dataset and the model structure, and calculate the total size of the required video memory blocks. Then, the video memory blocks are divided into basic video memory blocks and dynamic video memory blocks, where the basic video memory blocks account for 60% of the total video memory and are used to store model parameters and fixed data, and the dynamic video memory blocks account for 40% of the total video memory and are used to store intermediate calculation results. Next, a video memory pre-allocation strategy is adopted to pre-allocate the video memory space required for training to avoid frequent video memory allocation and release during the training process. Finally, a video memory management mechanism is established to achieve dynamic scheduling and recycling of video memory blocks. The role of this step is to optimize the video memory usage efficiency and improve the model training speed.

[0039] The specific implementation of step S05 is as follows: First, a semantic understanding layer is constructed, and a bidirectional long short-term memory network is used to extract the context semantic information of the text. Then, a mathematical symbol mapping layer is constructed, and an attention mechanism is used to realize the mapping transformation from semantic features to mathematical symbols. Next, a logical relationship conversion layer is constructed, and a graph neural network is used to establish the logical relationship between mathematical symbols. Then, a mathematical unit organization layer is constructed, and a recurrent neural network is used to organize mathematical symbols and their logical relationships into mathematical logic units. Finally, a unit relationship construction layer is constructed, and a tree-shaped recurrent neural network is used to establish the hierarchical relationship between mathematical logic units. The purpose of this step is to construct a neural network model that can understand natural language and generate mathematical descriptions.

[0040] The specific implementation of step S06 is as follows: First, the non-exact description matrix is input into the mathematical logic reasoning network model, and the deep semantic features of the text are extracted through the semantic understanding layer. Then, the semantic features are converted into a sequence of mathematical symbols through the mathematical symbol mapping layer, and the cross-entropy loss function is used to optimize the mapping accuracy. Next, the logical relationship between mathematical symbols is constructed through the logical relationship conversion layer, and the relation classification loss function is used to optimize the relation recognition accuracy. Then, the mathematical symbols and their logical relationships are organized into mathematical logic units through the mathematical unit organization layer, and the structure similarity loss function is used to optimize the organization rationality. Finally, a tree structure of mathematical logic units is generated through the unit relationship construction layer, and the tree-shaped distance loss function is used to optimize the correctness of the tree structure. The role of this step is to realize the conversion process from natural language to mathematical description.

[0041] The specific implementation of step S07 is as follows: First, the semantic precision score is calculated, and the cosine similarity is used to calculate the semantic similarity between the predicted text and the standard text. The semantic precision threshold is set to 0.85. Then, the symbol precision score is calculated, and the edit distance is used to calculate the matching degree between the predicted symbol sequence and the standard symbol sequence. The symbol precision threshold is set to 0.9. Next, the structure precision score is calculated, and the tree structure similarity is used to calculate the consistency between the predicted tree structure and the standard tree structure. The structure precision threshold is set to 0.8. Finally, the logical precision score is calculated, and the graph matching algorithm is used to calculate the correspondence between the predicted logical relationship and the standard logical relationship. The logical precision threshold is set to 0.85. The purpose of this step is to evaluate the accuracy of the model prediction results and provide guidance for model optimization.

[0042] The specific implementation of step S08 is as follows: First, the natural language description text in the validation dataset is obtained. Then, step S02 is repeated to perform word segmentation on the validation text, extract semantic features, numerical features, and logical relationship features, and construct a non-exact description matrix. Next, step S06 is repeated to input the non-exact description matrix into the trained mathematical logic reasoning network model to generate a tree structure of mathematical logic units and a predicted mathematical description text. The role of this step is to verify the generalization ability of the model and test the performance of the model on unseen data.

[0043] The specific implementation of step S09 is to first calculate the accuracy scores for verifying the prediction results, including semantic accuracy, symbol accuracy, structural accuracy, and logical accuracy; then compare the accuracy scores with preset thresholds, where the semantic accuracy threshold is 0.85, the symbol accuracy threshold is 0.9, the structural accuracy threshold is 0.8, and the logical accuracy threshold is 0.85; if any accuracy score is lower than the corresponding threshold, add the verification sample to the training dataset; then retrain the expanded training dataset and optimize the model parameters using the gradient descent algorithm. The purpose of this step is to improve the prediction accuracy of the model through incremental training.

[0044] The specific implementation of step S10 is to first construct a unit contribution value evaluation model using a multi-layer perceptron structure, with the intermediate features of each layer as the input and the accuracy score as the output; then use the gradient backpropagation algorithm to calculate the contribution degree of each layer to the accuracy score, where the semantic understanding layer mainly affects semantic accuracy, the mathematical symbol mapping layer mainly affects symbol accuracy, the logical relationship conversion layer mainly affects logical accuracy, and the mathematical unit organization layer and the unit relationship construction layer mainly affect structural accuracy; then determine the contribution weights of each layer according to the magnitude of the gradient values. The role of this step is to quantitatively evaluate the importance of each network layer and provide a basis for resource allocation.

[0045] The specific implementation of step S11 is to first determine the video memory allocation amount for each layer according to the contribution weights. The greater the contribution weight, the more video memory is allocated, and the specific ratio is the square root of the contribution weight; then determine the number of computing threads for each layer according to the contribution weights. The greater the contribution weight, the more computing threads are allocated, and the specific number is the basic number of threads multiplied by the contribution weight; then determine the number of training batches for each layer according to the contribution weights. The greater the contribution weight, the more training batches, and the specific number of batches is the basic number of batches multiplied by the contribution weight. The purpose of this step is to reasonably allocate computing resources according to the importance of each layer and improve the training efficiency.

[0046] The specific implementation of step S12 is to first defragment the video memory blocks in the video memory pool and merge adjacent free video memory blocks; then recycle the video memory blocks that have not been used for a long time and mark them as available; then sort the video memory blocks according to the contribution weights, and the layers with higher contribution weights are given priority to obtain video memory resources; finally, regularly update the usage status of the video memory blocks, including usage time, access frequency, and free ratio. The role of this step is to optimize the usage efficiency of video memory resources and ensure the smooth progress of model training. This step will continue until all the accuracy scores of the verification prediction results reach the preset thresholds, indicating that the model training is completed.

[0047] The mathematical logic unit tree structure in this method adopts the object-oriented design concept, and each mathematical logic unit contains a complete function definition and interface description. The unit type identifier is used to distinguish different types of mathematical operations or functions, facilitating targeted processing by the model; the input parameter information defines the data types and format requirements that the unit can receive, ensuring the standardization of data input; the output result information specifies the return form of the calculation result, facilitating subsequent units to call data; the call method information defines the interaction rules between units, ensuring the correctness of data flow; the constraint condition information limits the value range of input parameters, ensuring the validity of calculation results. Through this structured design, the modular organization and flexible call of mathematical logic units are realized.

[0048] The imprecise description matrix includes a semantic vector matrix, a numerical feature matrix, and a relationship feature matrix. The semantic vector matrix is specifically represented as follows:

[0049]

[0050] In the formula, s ij is the vector value of the i-th word in the j-th semantic dimension; m is the length of the word sequence; n is the semantic vector dimension, with a value of 300; w k is the weight coefficient of the k-th context word; v k is the word vector of the k-th context word; K is the context window size; α is the semantic adjustment factor.

[0051] Among them, the derivation process of the semantic vector matrix equation in the imprecise description matrix is as follows:

[0052] 1. First, obtain the basic word vectors based on the Word2Vec model, and get the initial vector representation of each word through training on a large-scale corpus;

[0053] 2. Considering the influence of context semantic information, introduce the window size K, and the value range of K is 3 to 7, and determine the optimal value through grid search;

[0054] 3. To reflect the importance degree of context words in different positions, introduce the weight coefficient w k , and its calculation formula is:

[0055]

[0056] In the formula, k is the context word position, c is the target word position, ∈ is the smoothing factor, and the value range is 0.01 to 0.1;

[0057] 4. Finally, introduce the semantic adjustment factor α to optimize the vector representation, and optimize it through the backpropagation algorithm.

[0058] The numerical feature matrix is specifically represented as follows:

[0059]

[0060] In the formula, n ij is the value of the i-th numerical value on the j-th feature dimension; p is the number of numerical values; q is the number of feature dimensions; x t is the t-th numerical sample; μ is the numerical mean; T is the total number of samples; β is the numerical adjustment factor.

[0061] Among them, the establishment principle of the numerical feature matrix equation is as follows:

[0062] 1. Standardize the original numerical values to eliminate the influence of dimensions;

[0063] 2. Calculate the statistical features of the numerical samples, including mean, variance, kurtosis, skewness, etc.;

[0064] 3. Use variance to characterize the numerical distribution characteristics. The reason for choosing variance instead of standard deviation is that variance is more sensitive to outliers;

[0065] 4. Introduce the numerical adjustment factor β to adjust the feature distribution, and its value is determined by minimizing the reconstruction error.

[0066] The relationship feature matrix is specifically represented as follows:

[0067]

[0068] In the formula, r ij is the relationship feature value between the i-th word and the j-th word; u and v are the lengths of the word sequences; c l is the weight coefficient of the l-th type of relationship; d l is the feature vector of the l-th type of relationship;; is the total number of relationship types; γ is the relationship adjustment factor.

[0069] Among them, the establishment principle of the relationship feature matrix equation is as follows:

[0070] 1. Identify the relationship types between words based on dependency syntactic analysis;

[0071] 2. Encode the relationship types as vector representations, using the one-hot encoding method;

[0072] 3. Introduce the relationship weight coefficient c l , indicating the importance of different types of relationships, and its calculation formula is:

[0073]

[0074] In the formula, f l is the frequency of the l-th type of relationship in the training set;

[0075] 4. Finally, optimize the feature representation through the relationship adjustment factor γ.

[0076] The precise description matrix includes a mathematical symbol matrix, an operation symbol matrix, and a mathematical logic unit matrix. The mathematical symbol matrix is specifically represented as follows:

[0077]

[0078] In the formula, m ij is the value of the i-th symbol in the j-th feature dimension; a is the length of the symbol sequence; b is the feature dimension; f h is the weight coefficient of the h-th type of symbol; g h is the feature vector of the h-th type of symbol; H is the total number of symbol types; δ is the symbol adjustment factor.

[0079] Among them, the derivation process of the mathematical symbol matrix equation in the precise description matrix is as follows:

[0080] 1. Construct a mathematical symbol dictionary, including common mathematical symbols and their attribute information;

[0081] 2. Encode the symbol attribute information into vector representation;

[0082] 3. Introduce the symbol weight coefficient f h , reflecting the usage frequency of different types of symbols;

[0083] 4. Optimize the symbol representation through the symbol adjustment factor δ.

[0084] The operation symbol matrix is specifically represented as follows:

[0085]

[0086] In the formula, o ij is the value of the i-th operator in the j-th feature dimension; e is the length of the operator sequence; f is the feature dimension; q p is the weight coefficient of the p-th type of operation; z p is the feature vector of the p-th type of operation; P is the total number of operation types; θ is the operation adjustment factor.

[0087] Among them, the establishment principle of the operation symbol matrix equation is as follows:

[0088] 1. Define the operator precedence system, including four arithmetic operations, power operations, function operations, etc.;

[0089] 2. Encode the operator attributes into vector representation;

[0090] 3. Introduce the operation weight coefficient q p , reflecting the complexity of different operation types;

[0091] 4. Optimize the arithmetic representation by adjusting the factor θ.

[0092] The specific representation of the mathematical logic unit matrix is as follows:

[0093]

[0094] In the formula, u ij is the relationship value between the i-th logic unit and the j-th logic unit; g and h are the numbers of logic units; k y is the weight coefficient of the y-th type of relationship; l y is the eigenvector of the y-th type of relationship; Y is the total number of relationship types; λ is the unit adjustment factor.

[0095] Among them, the principle of establishing the mathematical logic unit matrix equation is as follows:

[0096] 1. Construct a relationship graph between logic units;

[0097] 2. Encode the relationship type as a vector representation;

[0098] 3. Introduce the relationship weight coefficient k y , reflecting the importance of different relationship types;

[0099] 4. Optimize the relationship representation by the unit adjustment factor λ.

[0100] The specific representation of the accuracy score calculation formula is as follows:

[0101] Semantic accuracy score:

[0102] In the formula, A s is the semantic accuracy score; x i and y i are the i-th dimensional semantic vectors of the predicted text and the standard text respectively; n is the dimension of the semantic vector; η is the semantic accuracy adjustment factor.

[0103] Symbol accuracy score:

[0104] In the formula, A m is the symbol accuracy score; D i is the number of edit operations at the i-th position; L p and L s are the lengths of the predicted sequence and the standard sequence respectively; ξ is the symbol accuracy adjustment factor.

[0105] Structure accuracy score:

[0106] In the formula, A t is the structure accuracy score; N cis the set of common nodes of the prediction tree structure and the standard tree structure; N p and N s are the node sets of the prediction tree structure and the standard tree structure respectively; ω is the structure accuracy adjustment factor.

[0107] Logical accuracy score:

[0108] In the formula, A l is the logical accuracy score; E c is the set of common edges of the prediction logical relationship and the standard logical relationship; E p and E s are the edge sets of the prediction logical relationship and the standard logical relationship respectively; φ is the logical accuracy adjustment factor.

[0109] In the calculation of the accuracy score, the semantic accuracy score is calculated using the cosine similarity. The reason for choosing the cosine similarity is that it is not sensitive to the vector length and can better reflect the semantic similarity degree; the symbol accuracy score is calculated based on the edit distance, and through normalization, the score range is between 0 and 1; the structure accuracy score uses the Dice coefficient to calculate the tree structure similarity, and this coefficient is symmetric and not sensitive to sample imbalance; the logical accuracy score uses the Jaccard coefficient to calculate the graph structure similarity, and this coefficient can effectively measure the overlapping degree of two sets.

[0110] The specific formula for the contribution weight is as follows:

[0111]

[0112] In the formula, W i is the contribution weight of the i-th layer; L is the total loss function; O i is the output of the i-th layer; ψ is the contribution weight adjustment factor.

[0113] Among them, the contribution weight is calculated using the method based on the gradient of the loss function, and the specific steps are as follows:

[0114] 1. Calculate the partial derivative of the loss function with respect to the output of each layer;

[0115] 2. Calculate the relative contribution of the output of each layer to the loss function;

[0116] 3. Introduce the contribution weight adjustment factor for optimization;

[0117] This equation can quantitatively evaluate the influence degree of each network layer on the model performance.

[0118] The specific formula for the video memory allocation amount is as follows:

[0119]

[0120] Wherein, M i is the video memory allocation amount of the i-th layer; M base is the basic video memory amount; W i is the contribution weight of the i-th layer; σ is the video memory allocation adjustment factor.

[0121] The calculation formula for the number of computing threads is specifically expressed as follows:

[0122] T i = T base ×W i + τ;

[0123] Wherein, T i is the number of computing threads of the i-th layer; T base is the basic number of threads; W i is the contribution weight of the i-th layer; τ is the thread number adjustment factor.

[0124] The calculation formula for the number of training batches is specifically expressed as follows:

[0125] B i = B base ×W i + ρ;

[0126] Wherein, B i is the number of training batches of the i-th layer; B base is the basic number of batches; W i is the contribution weight of the i-th layer; ρ is the batch number adjustment factor.

[0127] The calculation equations for the video memory allocation amount, the number of computing threads, and the number of training batches are all designed based on the contribution weight. Among them, the square root relationship is adopted for the video memory allocation amount to reduce the uneven resource allocation caused by the weight difference; the linear relationship is adopted for the number of computing threads and the number of training batches to highlight the priority of the important layers. The purpose of designing these equations is to achieve the reasonable allocation of computing resources and improve the training efficiency.

[0128] The value ranges of the adjustment factors are 0.1 to 0.5, and the optimal values are determined through cross-validation; the basic video memory amount, the basic number of threads, and the basic number of batches are determined according to the hardware configuration.

[0129] The values of all adjustment factors are determined through cross-validation, and the specific steps are as follows:

[0130] 1. Set the candidate value range of the adjustment factor to 0.1 to 0.5, with a step size of 0.1;

[0131] 2. Conduct a grid search on the validation set;

[0132] 3. Select the combination of adjustment factor values that makes the model performance optimal.

[0133] The determination methods for each basic parameter are as follows:

[0134] 1. The basic available memory is determined according to 20% of the GPU video memory size;

[0135] 2. The basic number of threads is determined according to 50% of the CPU core number;

[0136] 3. The basic number of batches is determined according to 10% of the training dataset size.

[0137] The construction principle of the above equations is as follows:

[0138] 1. The inexact description matrix adopts the form of weighted summation, takes into account the influence of context information, and introduces a tuning factor for fine-tuning;

[0139] 2. The exact description matrix also adopts the form of weighted summation, takes into account the importance of different types of features, and is optimized through a tuning factor;

[0140] 3. Different similarity measurement methods are used for calculating the accuracy score: cosine similarity is used for semantic accuracy, edit distance is used for symbol accuracy, tree structure similarity is used for structural accuracy, and graph matching similarity is used for logical accuracy;

[0141] 4. The contribution weight is calculated using the gradient method, taking into account the sensitivity of the loss function to the outputs of each layer;

[0142] 5. The resource allocation calculation takes into account the influence of the contribution weight. The video memory allocation adopts a square root relationship to smooth the differences, and the number of threads and the number of batches adopt a linear relationship to highlight the importance.

[0143] The second aspect of the present invention provides a computer-readable storage medium, in which program instructions are stored. When the program instructions run on a computer, they are used to execute the above method for establishing a large model of a mathematical description language.

[0144] The third aspect of the present invention provides a system for establishing a large model of a mathematical description language, including the above computer-readable storage medium. The system can be any one of a computer, a server, and a single-chip microcomputer. The computer-readable storage medium is arranged inside the system, and a microprocessor for executing the program instructions stored inside the computer-readable storage medium is arranged inside the system.

[0145] Specifically, the principle of the present invention is as follows: The technical solution of the present invention is based on deep learning and resource optimization theory. By constructing a hierarchical neural network model and a dynamic resource management mechanism, efficient training of mathematical description language conversion is achieved. In the design of the model structure, a multi-level processing architecture is adopted, and each layer is optimized for a specific task. The semantic understanding layer uses a bidirectional long short-term memory network, which can effectively capture the context semantic information of the text; the mathematical symbol mapping layer realizes the accurate mapping from semantic features to mathematical symbols through an attention mechanism; the logical relationship conversion layer uses a graph neural network to establish the logical relationship between mathematical symbols; the mathematical unit organization layer and the unit relationship construction layer are responsible for constructing a complete mathematical logic system.

[0146] In terms of resource management, the present invention innovatively proposes a video memory optimization allocation strategy based on contribution values. By calculating the contribution degree of each layer to the model performance through the gradient backpropagation algorithm, a quantitative basis for video memory allocation is established. The square root relationship is used to determine the video memory allocation amount, which not only considers the importance difference but also avoids excessive tilting of resource allocation. A linear relationship is adopted in the allocation of computing threads and training batches to highlight the priority of important layers and ensure the execution efficiency of key computing tasks. At the same time, through the unified management of the video memory pool, dynamic scheduling and optimized allocation of video memory resources are realized, effectively solving the problems of video memory fragmentation and resource waste.

[0147] A specific embodiment 1 of the present invention is provided below. The specific implementation manners of each step in this embodiment 1 are described in detail as follows.

[0148] The specific implementation manner of step S01 is to construct a training corpus by establishing a mathematical knowledge graph. First, mathematical concepts, theorems, formulas, and problem-solving methods are extracted from mathematics textbooks, papers, and professional literature to construct a set of mathematical knowledge nodes; then, the association relationships between the mathematical knowledge nodes are established; then, the mathematical knowledge nodes and their association relationships are stored in a structured manner to form a mathematical knowledge graph; based on the mathematical knowledge graph, mathematical problems described in natural language and their standard mathematical expression forms are collected from network resources, teaching materials, and examination question banks to establish an initial corpus; the initial corpus is subjected to data cleaning to remove duplicate samples, incomplete samples, and incorrect samples; finally, the cleaned corpus is divided into a training set and a validation set in a ratio of 8:2. The training set is used for model training, and the validation set is used for model evaluation. The reason for using a knowledge graph is that it can effectively express the hierarchical structure and association relationships of mathematical knowledge, facilitating subsequent model learning and understanding of mathematical concepts. The association relationships between knowledge nodes are represented by a directed graph, and the specific definition is as follows:

[0149] G=(V, E, R);

[0150] In the formula, V is the set of nodes, representing a mathematical concept; E is the set of edges, representing the connection relationship between nodes; R is the set of relationship types, including concept hierarchy relationships, attribute relationships, etc. The purpose of this step is to construct a high-quality training dataset to provide a reliable data basis for subsequent model training.

[0151] The specific implementation of step S02 is to first use a word segmentation tool to perform word segmentation on the natural language description text to obtain a word sequence; then use a pre-trained word vector model to extract the semantic features of the word sequence, and use a semantic feature calculation formula to map each word into a 300-dimensional word vector to construct a semantic vector matrix:

[0152]

[0153] In the formula, s ij is the vector value of the i-th word in the j-th semantic dimension; m is the length of the word sequence; n is the semantic vector dimension, with a value of 300; w k is the weight coefficient of the k-th context word; v k is the word vector of the k-th context word; K is the context window size; α is the semantic adjustment factor. The calculation method of the weight coefficient is:

[0154]

[0155] In the formula, k is the context word position, c is the target word position, ∈ is the smoothing factor, and its value range is from 0.01 to 0.1. Then use regular expressions to extract the numerical information in the text, including integers, decimals, fractions, percentages, etc., and construct a numerical feature matrix after normalizing the numerical information:

[0156]

[0157] In the formula, n ij is the value of the i-th numerical value in the j-th feature dimension; p is the number of numerical values; q is the feature dimension; x t is the t-th numerical sample; μ is the numerical mean; T is the total number of samples; β is the numerical adjustment factor. Finally, use dependency parsing technology to extract the logical relationships in the word sequence, including causal relationships, conditional relationships, parallel relationships, etc., and encode the logical relationships into a relationship feature matrix:

[0158]

[0159] In the formula, r ij is the relationship feature value between the i-th word and the j-th word; u and v are the lengths of the word sequence; c l is the weight coefficient of the l-th type of relationship; d lIt is the feature vector of the l-th type of relationship; L is the total number of relationship types; γ is the relationship adjustment factor. The function of this step is to convert the natural language description text into a feature representation that can be processed by a computer, providing input for the subsequent mathematical description conversion.

[0160] The specific implementation of step S03 is to first perform lexical analysis on the standard mathematical description language text, identify mathematical symbols, operation symbols, keywords, and identifiers, and construct a mathematical symbol matrix:

[0161]

[0162] In the formula, m ij is the value of the i-th symbol in the j-th feature dimension; a is the length of the symbol sequence; b is the feature dimension; f h is the weight coefficient of the h-th type of symbol; g h is the feature vector of the h-th type of symbol; H is the total number of symbol types; δ is the symbol adjustment factor. Then perform syntax analysis and construct an operation symbol matrix:

[0163]

[0164] In the formula, o ij is the value of the i-th operator in the j-th feature dimension; e is the length of the operator sequence; f is the feature dimension; q p is the weight coefficient of the p-th type of operation; z p is the feature vector of the p-th type of operation; P is the total number of operation types; θ is the operation adjustment factor. Then perform semantic analysis, extract the dependency relationship and call relationship between mathematical logic units, and construct a mathematical logic unit matrix:

[0165]

[0166] In the formula, u ij is the relationship value between the i-th logic unit and the j-th logic unit; g and h are the number of logic units; k y is the weight coefficient of the y-th type of relationship; l y is the feature vector of the y-th type of relationship; Y is the total number of relationship types; λ is the unit adjustment factor. The purpose of this step is to convert the standard mathematical description language text into a structured matrix representation, providing the target output for model training.

[0167] The specific implementation of step S04 is to first estimate the video memory requirements during the training process according to the size of the training dataset and the model structure, and calculate the total size of the required video memory blocks; then divide the video memory blocks into basic video memory blocks and dynamic video memory blocks, where the basic video memory blocks account for 60% of the total video memory and are used to store model parameters and fixed data, and the dynamic video memory blocks account for 40% of the total video memory and are used to store intermediate calculation results; the size calculation formula of the basic video memory blocks is:

[0168] M base = M total ×0.6×η;

[0169] In the formula, M total is the total video memory size; η is the video memory utilization factor, and its value range is from 0.8 to 0.9. The size calculation formula of the dynamic video memory block is:

[0170] M dynamic = M total ×0.4×η;

[0171] Then, adopt the video memory pre-allocation strategy to pre-allocate the video memory space required for training in advance to avoid frequent video memory allocation and release during training; finally, establish a video memory management mechanism to realize the dynamic scheduling and recycling of video memory blocks. The function of this step is to optimize the video memory usage efficiency and improve the model training speed.

[0172] The specific implementation method of step S05 is to first construct a semantic understanding layer, and use a bidirectional long short-term memory network to extract the context semantic information of the text. Its state update formula is:

[0173] h t = f(W h × [h t-1 , x t + b h );

[0174] In the formula, h t is the hidden state at time t; W h is the weight matrix; x t is the input vector; b h is the bias term; f is the activation function. Then construct a mathematical symbol mapping layer, and use the attention mechanism to realize the mapping conversion from semantic features to mathematical symbols. The attention weight calculation formula is:

[0175]

[0176] In the formula, a ij is the attention weight; e ij is the energy function value; n is the sequence length. Then construct a logical relationship conversion layer, and use a graph neural network to establish the logical relationship between mathematical symbols. The node update formula is:

[0177]

[0178] In the formula, is the state of node i at time t; W v is the weight matrix; m ij is the edge weight; N(i) is the neighbor set of node i; b vis the bias term; g is the activation function. Then, a mathematical unit organization layer is constructed, and a recurrent neural network is used to organize mathematical symbols and their logical relationships into mathematical logic units. Finally, a unit relationship construction layer is constructed, and a tree-shaped recurrent neural network is used to establish the hierarchical relationship between mathematical logic units. The purpose of this step is to construct a neural network model that can understand natural language and generate mathematical descriptions.

[0179] The specific implementation of step S06 is to first input the imprecise description matrix into the mathematical logic reasoning network model. The processing formula of the input layer is:

[0180] X = W i ×I + b i ;

[0181] In the formula, X is the output of the input layer; W i is the input layer weight matrix; I is the input matrix; b i is the bias term. Then, the deep semantic features of the text are extracted through the semantic understanding layer; then, the semantic features are converted into a mathematical symbol sequence through the mathematical symbol mapping layer, and the mapping accuracy is optimized using the cross-entropy loss function:

[0182]

[0183] In the formula, L ce is the cross-entropy loss; y i is the true label; is the predicted probability; n is the number of samples. Then, the logical relationship between mathematical symbols is constructed through the logical relationship conversion layer; finally, the mathematical logic unit tree structure is generated through the unit relationship construction layer. The role of this step is to realize the conversion process from natural language to mathematical description.

[0184] Among them, the calculation formula for converting the semantic vector matrix into a mathematical concept representation through the semantic understanding layer is:

[0185] C = f(W c ×S + b c );

[0186] In the formula, C is the mathematical concept representation; W c is the conversion weight matrix; S is the semantic vector matrix; b c is the bias term; f is the hyperbolic tangent activation function. The calculation formula for mapping the mathematical concept representation into a mathematical symbol sequence through the mathematical symbol mapping layer is:

[0187]

[0188] In the formula, P ij is the probability that the i-th concept is mapped to the j-th symbol; e ijFor mapping energy; K is the size of the symbol dictionary. The calculation formula for constructing the logical relationship between mathematical symbol sequences through the logical relationship conversion layer is:

[0189] R ij = g(W r × [h i ,h j + b r );

[0190] In the formula, R ij is the relationship vector between symbol i and symbol j; W r is the relationship conversion matrix; h i and h j are the hidden states of the symbol; b r is the bias term; g is the sigmoid activation function. The calculation formula for organizing the mathematical symbol sequence and the logical relationship into a mathematical logic unit through the mathematical unit organization layer is:

[0191] U t = σ(W u × [x t ,h t-1 + b u ) × tanh(W c × [x t ,r t × h t-1 + b c );

[0192] In the formula, U t is the representation of the mathematical logic unit at time t; x t is the input symbol; h t-1 is the hidden state of the previous time; r t is the reset gate; W u and W c are the weight matrices; b u and b c are the bias terms. The calculation formula for establishing the hierarchical relationship and the call relationship between mathematical logic units through the unit relationship construction layer is:

[0193] T ij = softmax(W t × [U i ,U j + b t );

[0194] In the formula, T ij is the relationship probability distribution between unit i and unit j; W t is the tree structure weight matrix; b t is the bias term.

[0195] The specific implementation of step S07 is to first calculate the semantic accuracy score by using cosine similarity to calculate the semantic similarity between the predicted text and the standard text:

[0196]

[0197] In the formula, A s is the semantic accuracy score; x i and y i are the i-th dimensional semantic vectors of the predicted text and the standard text respectively; n is the dimension of the semantic vector; η is the semantic accuracy adjustment factor. Then calculate the symbol accuracy score:

[0198]

[0199] In the formula, A m is the symbol accuracy score; D i is the number of editing operations at the i-th position; L p and L s are the lengths of the predicted sequence and the standard sequence respectively; ξ is the symbol accuracy adjustment factor.

[0200] First, calculate the overall loss function:

[0201] L total = α1L s + α2L m + α3L t + α4L l + λΩ(W);

[0202] In the formula, L s , L m , L t , L l are the semantic loss, symbol loss, structure loss, and logic loss respectively; α1, α2, α3, α4 are the loss weights; λ is the regularization coefficient; Ω(W) is the L2 regularization term. Then use the adaptive moment estimation optimization algorithm for parameter update:

[0203] m t = β1m t-1 + (1 - β1)g t ;

[0204]

[0205] In the formula, m t and v t are the first-order and second-order momenta; β1 and β2 are the momentum decay rates; g t is the gradient; η is the learning rate; ∈ is the numerical stability factor.

[0206] The purpose of this step is to evaluate the accuracy of the model's prediction results and provide guidance for model optimization.

[0207] The specific implementation of step S08 is to first obtain the natural language description text in the validation dataset and process it through the established data processing flow; repeat step S02 to perform word segmentation on the validation text, extract semantic features, and construct a semantic vector matrix S; extract numerical features and construct a numerical feature matrix N; extract logical relationship features and construct a relationship feature matrix R; then repeat step S06, input the constructed imprecise description matrix into the trained mathematical logic reasoning network model, and generate a mathematical logic unit tree structure and predicted mathematical description text through sequential processing of the semantic understanding layer, mathematical symbol mapping layer, logical relationship conversion layer, mathematical unit organization layer, and unit relationship construction layer. This step uses the same feature extraction and model inference process as the training stage, ensuring the consistency of the validation process.

[0208] The specific implementation of step S09 is to first calculate the precision scores of the validation prediction results, including semantic precision A s , symbol precision A m , structure precision A t and logical precision A l ; then compare each precision score with the preset threshold, where the preset semantic precision threshold is 0.85, the preset symbol precision threshold is 0.9, the preset structure precision threshold is 0.8, and the preset logical precision threshold is 0.85; if any precision score is lower than the corresponding threshold, add the validation sample to the training dataset; then retrain the augmented training dataset and optimize the model parameters using the gradient descent algorithm:

[0209]

[0210] where W new is the updated weight; W old is the weight before update; α is the learning rate; L is the loss function; is the gradient of the loss function with respect to the weight. The purpose of this step is to improve the prediction accuracy of the model through incremental training.

[0211] The specific implementation of step S10 is to first construct a unit contribution value evaluation model using a multi-layer perceptron structure, input the intermediate features of each layer, and output the precision score; then use the gradient backpropagation algorithm to calculate the contribution weights of each layer:

[0212]

[0213] where W i is the contribution weight of the i-th layer; L is the total loss function; O iis the output of the i-th layer; ψ is the contribution weight adjustment factor. Among them, the calculation formula of the total loss function is:

[0214] K = λ1L s + λ2L m + λ3L t + λ4L l ;

[0215] In the formula, L s , L m , L t , L l are semantic loss, symbol loss, structure loss and logic loss respectively; λ1, λ2, λ3, λ4 are the weight coefficients of each loss term. The purpose of this step is to quantitatively evaluate the importance of each network layer and provide a basis for resource allocation.

[0216] The specific implementation of step S11 is to first determine the video memory allocation amount of each layer according to the contribution weight:

[0217]

[0218] In the formula, M i is the video memory allocation amount of the i-th layer; M base is the basic video memory; W i is the contribution weight of the i-th layer; σ is the video memory allocation adjustment factor. Then determine the number of computing threads of each layer according to the contribution weight:

[0219] T i = T base × W i + τ;

[0220] In the formula, T i is the number of computing threads of the i-th layer; T base is the basic number of threads; τ is the thread number adjustment factor. Finally, determine the number of training batches of each layer according to the contribution weight:

[0221] B i = B base × W i + ρ;

[0222] In the formula, B i is the number of training batches of the i-th layer; B base is the basic number of batches; ρ is the batch number adjustment factor. The purpose of this step is to reasonably allocate computing resources according to the importance of each layer and improve the training efficiency.

[0223] The specific implementation of step S12 is to first calculate the utilization efficiency index of the video memory block:

[0224]

[0225] Wherein, E i is the utilization efficiency of the i-th video memory block; U i is the effective usage time; T i is the total allocation time; F i is the access frequency; F max is the maximum access frequency; S i is the occupied space; S total is the total video memory space. Then, video memory fragmentation is performed:

[0226]

[0227] Wherein, M compact is the continuous video memory space after reorganization; M i is the size of the i-th video memory block; γ i is the fragmentation rate. Then, idle video memory recycling is performed:

[0228]

[0229] Wherein, M recycle is the size of the recyclable video memory; t i is the idle time; T threshold is the recycling threshold; I is the indicator function. Then, priority reallocation is performed based on the contribution weight:

[0230]

[0231] Wherein, P i is the priority of the video memory block; W i is the contribution weight of the corresponding layer; E i is the utilization efficiency; is the average efficiency; D i is the data dependency; D max is the maximum dependency. Finally, the video memory usage status is updated:

[0232]

[0233] Wherein, and are the new and old states respectively; ΔS i is the state change amount; α is the smoothing factor. Through these optimization operations, the efficient utilization and dynamic adjustment of video memory resources are achieved until the model performance meets the expected requirements.

[0234] To better understand and implement the present invention, Example 2 of a specific application scenario of the present invention is provided below: A research team in a certain university conducts research on an intelligent education project. The project goal is to build an intelligent system that can convert the description of natural language mathematical problems into standard mathematical expressions. The research team adopts the method of the present invention, and the specific implementation process is as follows:

[0235] In the first step, the research team collected 50,000 mathematical problem data from high school mathematics textbooks, test papers, and exercise sets, including application problems, geometry problems, and algebra problems. Each piece of data contains a natural language description and the corresponding standard mathematical expression. After data cleaning and screening, 45,000 pieces of valid data were finally obtained, including 36,000 in the training set and 9,000 in the validation set. An example of the training data is shown in Table 1.

[0236] Table 1: Example of training data

[0237]

[0238] In the second step, word segmentation and feature extraction were performed on the natural language description text. A word segmentation tool was used for word segmentation, with the word vector dimension set to 300 and the context window size set to 5. Numerical information was extracted and normalized, with the feature dimension set to 50. Dependency syntactic analysis was used to extract logical relationships, with the number of relationship types set to 20. In the finally obtained imprecise description matrix, the semantic vector matrix dimension is 200×300, the numerical feature matrix dimension is 50×50, and the relationship feature matrix dimension is 200×200.

[0239] In the third step, the standard mathematical expressions were structurally processed. A mathematical symbol dictionary was constructed, containing 100 basic operation symbols and 200 mathematical function symbols. The symbol feature dimension was set to 100, and the operation feature dimension was set to 50. A mathematical logic unit tree was constructed through syntactic analysis, with an average of 8 logic units in each expression. In the obtained precise description matrix, the mathematical symbol matrix dimension is 300×100, the operation symbol matrix dimension is 100×50, and the mathematical logic unit matrix dimension is 300×300.

[0240] In the fourth step, the research team used an NVIDIA A100 GPU with 32GB of video memory for model training. The basic video memory block was allocated 19.2GB to store model parameters and fixed data; the dynamic video memory block was allocated 12.8GB to store intermediate calculation results. The video memory utilization factor was set to 0.85, and the smoothing factor was set to 0.05.

[0241] In the fifth step, a 5-layer mathematical logic reasoning network model was constructed. The semantic understanding layer uses a bidirectional long short-term memory network with a hidden layer dimension of 512; the mathematical symbol mapping layer uses a multi-head attention mechanism with 8 attention heads; the logical relationship conversion layer uses a graph convolutional network with a convolution kernel size of 3; the mathematical unit organization layer uses a gated recurrent unit network with a hidden layer dimension of 256; the unit relationship construction layer uses a tree-shaped recurrent neural network with an output dimension of 128.

[0242] In the sixth step, the model training adopts an iterative method with a batch size of 64. The semantic accuracy threshold is set to 0.85, the symbol accuracy threshold is set to 0.9, the structure accuracy threshold is 0.8, and the logic accuracy threshold is 0.85. During the training process, by dynamically evaluating the contribution weights of each layer, resource optimization allocation is performed. The contribution weights and resource allocation of each layer in the initial stage are shown in Table 2.

[0243] Table 2: Contribution Weights and Resource Allocation of Each Layer

[0244] Network layer Contribution weight Video memory allocation amount (GB) Number of computing threads Number of training batches Semantic understanding layer 0.3 3.84 12 1000 Symbol mapping layer 0.25 3.20 10 800 Relationship conversion layer 0.2 2.56 8 600 Unit organization layer 0.15 1.92 6 400 Relationship construction layer 0.1 1.28 4 200

[0245] In the seventh step, the video memory usage status is monitored in real time during the training process, and the video memory status is updated every 10 batches. The trigger threshold for video memory fragmentation is set to 30%, and the time threshold for video memory block recycling is set to 100 training batches. By dynamically adjusting the video memory allocation strategy, the video memory utilization rate during the model training process reaches 92%.

[0246] After 50 training cycles, the model reaches the following performance indicators on the validation set: semantic accuracy is 0.87, symbol accuracy is 0.92, structure accuracy is 0.83, and logic accuracy is 0.88. For the newly input natural language mathematical problem description, the model can generate the corresponding standard mathematical expression within 0.5 seconds, and the accuracy rate reaches 85%.

[0247] Traditional methods for training mathematical description language models adopt a fixed video memory allocation strategy, usually evenly distributing the video memory to each network layer, or simply allocating it according to the parameter quantity of the layer. This method does not consider the actual importance and resource demand changes of each layer during the training process, resulting in insufficient video memory for some key layers and affecting the training effect, while there is video memory waste in other layers. At the same time, due to the lack of an effective video memory management mechanism, the problem of video memory fragmentation is serious, and the actual video memory utilization rate is usually only 60% to 70%.

[0248] The present invention adopts a dynamic video memory optimization allocation strategy based on contribution values, and realizes the precise allocation of video memory resources by real-time evaluating the contribution degree of each layer to the model performance. Combined with the video memory pool management mechanism, the problem of video memory fragmentation is effectively solved, and the video memory utilization rate is increased to more than 90%. In this embodiment, by reasonably allocating computing resources, the model training speed is increased by 40% compared with the traditional method, and a higher prediction accuracy is achieved within the same training cycle. This shows that the present invention successfully solves the problem of low computing resource utilization rate caused by unreasonable video memory allocation, and provides an effective solution for the efficient training of mathematical description language models.

[0249] The following provides an embodiment 3 of a specific application scenario of the present invention: A research team conducts research work based on the method of the present invention, and the specific implementation process is as follows:

[0250] In the first step, the research team collected 100,000 English papers in the fields of mathematics, physics, and computer science from databases such as arXiv and Web of Science, and extracted 500,000 corresponding data pairs of natural language descriptions and mathematical formulas from them. Through manual screening and quality assessment, incomplete, incorrect, and duplicate data were removed, and finally 400,000 effective data pairs were obtained. The dataset was divided into a training set of 280,000, a validation set of 80,000, and a test set of 40,000 according to the ratio of 7:2:1. An example of the training data is shown in Table 3:

[0251] Table 3: Example of training data

[0252]

[0253] In the second step, feature extraction and matrix construction were performed on the natural language description text. First, the BERT pre-trained model was used for text encoding to obtain a 768-dimensional text representation vector. Then, a bidirectional long short-term memory network was used to extract context features, and the hidden layer dimension was set to 512. Special marking processing was performed on professional terms and mathematical symbols, and a feature dictionary was constructed, including 5,000 professional terms and 1,000 mathematical symbols. The finally obtained inexact description matrix consists of the following three parts:

[0254] 1. Semantic vector matrix: with a dimension of 1000×768, and feature weighting was performed using the attention mechanism: Among them, the context window size K was set to 7, and the weight coefficient w k was calculated using relative position encoding, and the smoothing factor ∈ was set to 0.05.

[0255] 2. Numerical feature matrix: with a dimension of 200×100, and statistical feature extraction was performed: Among them, the numerical adjustment factor β was determined to be 0.15 through cross-validation.

[0256] 3. Relationship feature matrix: with a dimension of 1000×1000, and dependency syntax analysis was performed: Among them, the total number of relationship types L was set to 50, and the relationship adjustment factor γ was set to 0.2.

[0257] In the third step, structural processing was performed on the standard mathematical expressions. The mathematical symbols and operation symbols were identified through a syntax analyzer, and a mathematical logic unit tree was constructed. The obtained exact description matrix includes:

[0258] 1. Mathematical symbol matrix: with a dimension of 500×200: Among them, the total number H of symbol types is 300, and the symbol adjustment factor δ is set to 0.1.

[0259] 2. Operator symbol matrix: The dimension is 200×100: Among them, the total number P of operation types is 100, and the operation adjustment factor θ is set to 0.15.

[0260] 3. Mathematical logic unit matrix: The dimension is 500×500: Among them, the total number Y of relationship types is 80, and the unit adjustment factor λ is set to 0.25.

[0261] In the fourth step, the research team uses an 8-card NVIDIA A100 GPU cluster for distributed training, and each graphics card is configured with 80GB of video memory. The video memory allocation scheme is as follows:

[0262] 1. Basic video memory configuration: Each graphics card's basic video memory block is allocated 48GB for storing model parameters and fixed data; the dynamic video memory block is allocated 32GB for storing intermediate calculation results. The video memory utilization factor η is set to 0.9, and the basic video memory amount is calculated as follows: M base = M total ×0.6×η = 80×0.6×0.9 = 43.2GB.

[0263] 2. Dynamic video memory configuration: M dynamic = M total ×0.4×η = 80×0.4×0.9 = 28.8GB.

[0264] In the fifth step, a 5-layer mathematical logic reasoning network model is constructed, and the configuration parameters of each layer are as follows:

[0265] 1. Semantic understanding layer: Use a 12-layer Transformer encoder, with 8 attention heads in each layer, a hidden layer dimension of 1024, a feed-forward network dimension of 4096, and a dropout rate of 0.1.

[0266] 2. Mathematical symbol mapping layer: Use a cross-attention mechanism, with 16 attention heads, a key-value dimension of 64, and a mapping hidden layer dimension of 512.

[0267] 3. Logical relationship conversion layer: Use a graph attention network, with 4 layers, 8 attention heads in each layer, a feature dimension of 256, residual connections, and layer normalization.

[0268] 4. Mathematical unit organization layer: Use a recurrent neural network, LSTM units, a hidden layer dimension of 512, 3 layers, and bidirectional processing.

[0269] 5. Unit relationship construction layer: Use a tree-shaped LSTM network, with an input dimension of 256, a hidden layer dimension of 512, and a maximum tree depth of 10.

[0270] Step 6. The following optimization strategy is adopted in the training process:

[0271] 1. Loss function setting: Total loss function: L total = α1L s + α2L m + α3L t + α4L l + λΩ(W); where α1 = 0.3, α2 = 0.25, α3 = 0.25, α4 = 0.2, and the regularization coefficient λ = 0.001.

[0272] 2. Optimizer configuration: The Adam optimizer is adopted, with an initial learning rate of 0.0001, β1 = 0.9, β2 = 0.999, and weight decay of 0.01. The cosine annealing learning rate scheduler is used, with a minimum learning rate of 1e-6.

[0273] 3. Training batch setting: The initial batch size is 128, and the gradient accumulation strategy is adopted, with an accumulation step of 4. The total number of training epochs is 100, and the patience index of the early stopping strategy is set to 10.

[0274] Step 7. Resource dynamic allocation strategy based on contribution weights:

[0275] 1. Calculation of contribution weights: where the contribution weight adjustment factor ψ is set to 0.1.

[0276] 2. Calculation of video memory allocation amount: The video memory allocation adjustment factor σ is set to 0.2.

[0277] 3. Calculation of the number of computing threads allocated: T i = T base × W i + τ; The basic number of threads is set to 16, and the thread number adjustment factor τ is set to 0.15.

[0278] 4. Calculation of the number of training batches allocated: B i = B base × W i + ρ; The basic number of batches is set to 1000, and the batch number adjustment factor ρ is set to 0.1.

[0279] After 100 rounds of training, the model achieves the following performance metrics on the validation set: 1. Semantic accuracy: 0.893 2. Symbol accuracy: 0.921 3. Structural accuracy: 0.856 4. Logical accuracy: 0.878

[0280] In actual application tests, for newly input natural language mathematical descriptions, the model can generate corresponding standard mathematical expressions within 1 second, with an accuracy rate of 88.5%. The performance of the model in processing different types of mathematical expressions is shown in Table 4:

[0281] Table 4: Processing Performance of Different Types of Mathematical Expressions

[0282] Expression type Number of samples Accuracy rate Average processing time (ms) Algebraic expression 10000 91.2% 180 Calculus formula 8000 87.5% 320 Probability and statistics 7000 89.6% 250 Linear algebra 9000 86.8% 420 Geometric formula 6000 90.3% 290

[0283] The traditional method mainly has the following problems: 1. Adopting a fixed ratio of video memory allocation strategy, it is unable to dynamically adjust resource allocation according to the actual needs of each layer during model training; 2. Lacking a quantitative evaluation mechanism for the contribution degree of each layer of the model, resulting in inaccurate resource allocation; 3. The video memory management mechanism is simple, prone to generating video memory fragmentation, which affects the training efficiency. These problems lead to the general video memory utilization rate of the traditional method being lower than 70%, and the model training speed is slower.

[0284] In contrast, the solution of the present invention has the following advantages: 1. Dynamically adjusting video memory allocation by evaluating the contribution value of the model, the video memory utilization rate is increased to 92%; 2. The incremental training strategy based on precision evaluation improves the model prediction accuracy by 5 percentage points; 3. The video memory pool management mechanism effectively reduces video memory fragmentation, and the training speed is increased by 40%; 4. The multi-level model structure design improves the processing ability of complex mathematical expressions. Practice has proved that the present invention has successfully solved the technical problem of low utilization rate of video memory resources, and provides a feasible solution for the efficient training of mathematical description language models.

[0285] It should be noted that the detailed explanations of the variables involved in the present invention are shown in Tables 5 and 6 below.

[0286] Table 5: Variable Explanation Table (First Part)

[0287]

[0288]

[0289] Table 6: Variable Explanation Table (Second Part)

[0290]

[0291] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention.

Claims

1. A method for establishing a large model of a mathematical description language, characterized in that: The following steps are involved: Obtain the natural language description text and its corresponding standard mathematical description language text in the training corpus to establish a training data set; establish a video memory pool and pre-allocate the video memory blocks required for training, divide the video memory blocks into basic video memory blocks and dynamic video memory blocks, the basic video memory blocks are used to store fixed parameters, and the dynamic video memory blocks are used to store intermediate calculation results; construct a mathematical logic reasoning network model including a semantic understanding layer, a mathematical symbol mapping layer, a logical relationship conversion layer, a mathematical unit organization layer and a unit relationship construction layer; establish a unit contribution value evaluation model based on the accuracy score of the verification prediction result, and use the gradient back propagation algorithm to calculate the contribution weight of each layer; determine the video memory allocation amount, the number of computing threads and the number of training batches according to the contribution weight calculated by the unit contribution value evaluation model; The video memory pool is optimized for video memory allocation according to the video memory allocation amount until the accuracy score of the verification prediction result reaches a preset accuracy threshold.

2. The method for establishing a large mathematical description language model according to claim 1, characterized in that: The steps of establishing the training data set are specifically to construct a training corpus by establishing a mathematical knowledge graph, extracting mathematical concepts, theorems, formulas and problem-solving methods from mathematical textbooks, papers and professional literature, constructing a set of mathematical knowledge nodes, establishing inclusion relationships, inheritance relationships, combination relationships and conversion relationships between mathematical knowledge nodes, and performing structured storage of mathematical knowledge nodes and their associations.

3. The method for establishing a large mathematical description language model according to claim 1, characterized in that: The preset accuracy threshold includes a preset semantic accuracy threshold, a preset symbol accuracy threshold, a preset structure accuracy threshold and a preset logic accuracy threshold, wherein the preset semantic accuracy threshold is 0.85, the preset symbol accuracy threshold is 0.9, the preset structure accuracy threshold is 0.8, and the preset logic accuracy threshold is 0.

85.

4. The method for establishing a large mathematical description language model according to claim 1, characterized in that: The semantic understanding layer uses a bidirectional long short-term memory network to extract contextual semantic information of the text, the mathematical symbol mapping layer uses an attention mechanism to realize the mapping conversion of semantic features to mathematical symbols, the logical relationship conversion layer uses a graph neural network to establish logical relationships between mathematical symbols, the mathematical unit organization layer uses a recursive neural network to organize mathematical logical units, and the unit relationship construction layer uses a tree-type recurrent neural network to establish hierarchical relationships between mathematical logical units.

5. The method for establishing a large mathematical description language model according to claim 1, characterized in that: The total size of the video memory block is determined according to the size of the inaccurate description matrix, the size of the accurate description matrix and the preset storage requirements of intermediate calculation results. The basic video memory block accounts for 60% of the total video memory, and the dynamic video memory block accounts for 40% of the total video memory.

6. The method for establishing a large mathematical description language model according to claim 1, characterized in that: The method for determining the contribution weight is that the semantic understanding layer mainly affects the semantic accuracy, the mathematical symbol mapping layer mainly affects the symbol accuracy, the logical relationship conversion layer mainly affects the logical accuracy, and the mathematical unit organization layer and the unit relationship construction layer mainly affect the structural accuracy.

7. The method for establishing a large mathematical description language model according to claim 1, characterized in that: The method for determining the amount of video memory allocation is that the greater the contribution weight, the more video memory is allocated, and the specific ratio is the square root of the contribution weight; the method for determining the number of computing threads is that the greater the contribution weight, the more computing threads are allocated, and the specific number is the number of basic threads multiplied by the contribution weight; the method for determining the number of training batches is that the greater the contribution weight, the more training batches, and the specific number of batches is the number of basic batches multiplied by the contribution weight.

8. The method for establishing a large mathematical description language model according to claim 1, characterized in that: The video memory allocation optimization includes merging the idle video memory in the video memory block, recycling the idle video memory in the video memory block, reallocating the priority of the video memory block according to the contribution weight, and updating the usage status of the video memory block, wherein the usage status includes usage time, access frequency and idle ratio.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are run in a computer, they are used to execute the method for establishing a large model in a mathematical description language according to any one of claims 1 to 8.

10. A mathematical description language large model building system, characterized in that: The system comprises the computer-readable storage medium as claimed in claim 9, wherein the system is any one of a computer, a server, and a single-chip microcomputer, the computer-readable storage medium is arranged in the system, and the system is provided with a microprocessor for executing program instructions stored in the computer-readable storage medium.

Citation Information

Patent Citations

  • Method for optimizing speech recognition process, and device thereof and storage medium

    CN113205818A

  • Method and device for improving throughput of large language model

    CN117349032A

  • Intelligent AI construction and operation method and system and PC host

    CN118485163A

  • Using textures in graphics processing systems

    US20200027260A1

  • Finetuning for multi-task learning

    US20240256860A1