Mathematical formula representing and matching method based on semantic enhancement

Abstract

The invention provides a semantic enhancement-based mathematical formula representation and matching method, which relates to the technical field of mathematical formula representation and matching, and comprises the following steps of: positioning a formula picture of a target formula in a target text, and extracting a formula description text; identifying the formula picture, and converting the formula picture into a LaTeX expression; extracting a formula structure expression; generating a symbol formula tree; according to the formula structure expression, recognizing and extracting formula variable semantics from the formula description text; on the basis of variable proofreading and alignment, formula variable semantics are integrated into a symbol formula tree, and a semantic formula tree is formed; and performing formula matching through a similarity matching strategy, and realizing formula downstream application according to a similarity matching result. According to the method and the device, the technical problem that the formula matching effect is poor due to the fact that original text semantics of the analyzed formula information are lost to a great extent due to the fact that a formula analysis and expression method in the prior art is often insufficient in analysis or formula expression is separated from semantics is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of mathematical formula representation and matching, and in particular to a mathematical formula representation and matching method based on semantic enhancement. Background Art

[0002] Formulas, which are mathematical expressions with clear practical meanings, are usually used to describe mathematical laws and variable relationships in the fields of science or engineering. They are one of the common forms of information expression in scientific and technological research and engineering applications. Documents from the STEM field often contain a large number of mathematical formulas and are mainly stored in the form of pictures or LaTeX expressions.

[0003] Formulas carry extremely rich and highly condensed knowledge, and carry important tasks such as describing natural phenomena or technical systems, simplifying complex problems, conducting derivation and verification, and realizing cross-domain knowledge transfer. Their inherent knowledge value is very prominent. The full analysis and matching application of formula content is conducive to the in-depth mining of scientific and technological literature knowledge content, and can provide a foundation for future advanced intelligent document processing technology and applications. The use of formula knowledge, such as formula matching, formula retrieval, and formula recommendation, all need to be based on the analysis and representation of formulas.

[0004] Current formula parsing and representation methods often have problems with insufficient parsing and formula representation that is out of semantics. Most parsing methods only convert formula images into LaTeX expressions and then do no more processing. A few methods use the Embedding technology strategy to convert formulas into high-dimensional vector representations, and very few methods convert them into symbolic formula tree representations. The formula information parsed by the above methods has lost its original text semantics to a large extent, making the formula relatively isolated and not conducive to the integration of formulas with other parts of the document. For this reason, the formula matching application established on the basis of the above methods has the problem of insufficient matching feature information or serious loss of semantic information, resulting in poor formula matching results, which in turn affects the effects of downstream comprehensive applications such as formula retrieval and formula recommendation. Summary of the invention

[0005] The present application provides a mathematical formula representation and matching method based on semantic enhancement, aiming to solve the technical problem that the formula parsing and representation methods in the prior art often do not parse sufficiently or the formula representation is separated from the semantics, so that the parsed formula information loses its original text semantics to a large extent, resulting in poor formula matching effect.

[0006] The present application discloses a method for representing and matching mathematical formulas based on semantic enhancement, the method comprising: locating a formula image of a target formula in a target text, and extracting a formula description text of the target formula; using OCR technology to recognize the formula image, and converting the recognized formula into a LaTeX expression; using a locally deployed open source large language model to extract a formula structure expression from the LaTeX expression; based on the formula structure expression, generating a symbolic formula tree of the formula, wherein the symbolic formula tree is stored in a json format; according to the formula structure expression, recognizing and extracting formula variable semantics from the formula description text; on the basis of variable proofreading and alignment, integrating the formula variable semantics into the symbolic formula tree to form a formula tree with semantics; based on the formula tree with semantics, performing formula matching through a similarity matching strategy, and realizing formula downstream applications according to the similarity matching results, wherein the formula downstream applications include formula retrieval and formula recommendation, and formula matching includes structure matching and semantic matching.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0008] By locating the formula image of the target formula in the target text and extracting the relevant formula description text, mathematical formula information can be automatically extracted from the text, which provides a basis for subsequent formula parsing and semantic analysis, avoids manual intervention, and improves the degree of automation; by using OCR technology to recognize formula images and convert them into LaTeX expressions, the formula can be converted from image format to operable text format, thereby providing a standardized representation for the structural parsing and further processing of the formula; the locally deployed open source large language model is used to extract the structural expression of the formula from the LaTeX expression, so that the components of the formula and their relationships can be understood, enhancing the accuracy and automation level of formula parsing; based on the formula structural expression, a symbolic formula tree is generated, and the structural information of the formula is stored in a tree structure in json format, which enables the various elements of the formula and their relationships to be clearly and hierarchically represented, which is conducive to subsequent formula processing and matching; through The semantic information of formula variables is identified and extracted from the formula description text, so that each variable in the formula contains not only numeric symbols but also corresponding natural language descriptions, which provides strong support for subsequent semantic alignment and matching; on the basis of variable proofreading and alignment, the semantics of formula variables are integrated into the symbolic formula tree to generate a semantic formula tree. This process makes the formula not only reflect its operation structure but also contain the natural language description of the variables, which significantly enhances the comprehensibility of the formula; based on the semantic formula tree, formula matching is performed through a similarity matching strategy, combining structural matching and semantic matching to determine the similarity of formulas, find the most similar formulas, and provide accurate matching results for formula retrieval and application. By combining the dual matching method of structure and semantics, formulas similar to instance formulas can be identified more accurately. Semantic matching avoids the mismatching that may occur in matching based solely on structure, and improves the accuracy and flexibility of matching.

[0009] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A flowchart of a mathematical formula representation and matching method based on semantic enhancement is provided for an embodiment of the present application.

[0011] Figure 2 A schematic diagram of a variable proofreading and alignment process in a mathematical formula representation and matching method based on semantic enhancement is provided for an embodiment of the present application.

[0012] Figure 3A schematic diagram of the process of node identification for a symbolic formula tree in a mathematical formula representation and matching method based on semantic enhancement is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0013] The embodiments of the present application provide a mathematical formula representation and matching method based on semantic enhancement, thereby solving the technical problem that the formula parsing and representation methods in the prior art often fail to perform sufficient parsing or the formula representation is separated from semantics, causing the parsed formula information to lose its original text semantics to a large extent, resulting in poor formula matching effect.

[0014] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically described below in conjunction with the drawings in the specification.

[0015] like Figure 1 As shown, the embodiment of the present application provides a mathematical formula representation and matching method based on semantic enhancement, the method comprising:

[0016] Step S100: locating a formula image of a target formula in a target text, and extracting a formula description text of the target formula;

[0017] Formula parsing is performed on the basis of structured stored formula basic representations and formula-related texts. For scientific documents stored in PDF format, image processing techniques, such as image-based template matching, image region detection or other image recognition methods, are used to extract the text content of the scientific document, a picture of a specific formula to be parsed, and the position of the formula in the text. This is used as the starting point for formula parsing. With the help of image processing libraries, such as OpenCV, boundary detection can be performed to determine the image area containing the formula.

[0018] Based on the formula location results, extract the text description of the target formula, for example, "Formula (1) describes Newton's second law". These text descriptions help explain the variables, constants and physical meaning of the formula. When extracting the text description of the formula, natural language processing technology can be used to identify the text related to the formula, and text segmentation and keyword extraction technology can be used to extract the descriptive information related to the formula, and finally obtain the formula description text of the target formula. The formula description is the part of the text data such as scientific literature that explains the formula. It is generally located in the context near the formula, but it may also be in other locations in the document.

[0019] Step S200: using OCR technology to recognize the formula image, and converting the recognized formula into LaTeX expression;

[0020] Formula recognition is based on OCR (Optical Character Recognition) technology, which is a process of recognizing formula images and converting them into machine-understandable representations, mainly in LaTeX form. LaTeX is an advanced typesetting tool and language. With the help of LaTeX, precise representation and format control of formulas can be achieved through specific tags and commands.

[0021] Perform OCR recognition on formula images. OCR technology can convert characters and symbols in images into text information. In mathematical formula recognition, OCR technology can recognize not only ordinary text, but also various mathematical symbols, such as Greek letters, integral symbols, fraction lines, etc.

[0022] Formula recognition can be implemented based on a variety of tools and methods. Common methods include: traditional grammar-driven methods, which treat symbol segmentation, symbol recognition, and structural analysis as separate stages, with the latter stage highly dependent on the recognition results of the previous stage; deep learning-based methods, which treat formula recognition as a sequence-to-sequence prediction problem, often using an encoding and decoding framework, and training in an end-to-end manner to obtain a model that can achieve this function.

[0023] Step S300: extracting a formula structure expression from a LaTeX expression using a locally deployed open source large language model;

[0024] Use a locally deployed open source large language model, for example, a model based on the Transformer architecture, to perform formula parsing. This model needs to have the ability to understand LaTeX expressions and extract the formula structure. The model is deployed locally to ensure data security and privacy, and can also improve computing efficiency.

[0025] After receiving the LaTeX expression, the large language model begins to parse the structure. The model needs to understand the hierarchical structure of the formula, such as identifying operators, variables, constants, functions, etc., and identifying the relationship between them. For simple formulas such as a+b, the model recognizes that the plus sign is an operator, and a and b are variables; for complex formulas such as integral formulas, the model needs to recognize the integral symbol, integral interval, integrand, etc., and extract them. Through the processing of the large language model, the structure of the LaTeX expression will be parsed into a structured expression form, which usually contains information such as operators (such as addition, multiplication), operands (such as variables, constants), functions (such as integrals, exponents), etc.

[0026] Step S400: Generate a symbolic formula tree of the formula based on the formula structure expression, wherein the symbolic formula tree is stored in json format;

[0027] According to the extracted formula structure expression, each part of the formula is traversed. Each operator, variable, constant, function, etc. will be mapped to a node of the symbolic formula tree. Each node stores related information. These nodes will be linked through a tree structure to represent the operational relationship between them. The symbolic formula tree of the formula is generated. The generated symbolic formula tree is stored in a standardized format, specifically in json format. json format storage facilitates subsequent processing, query and analysis.

[0028] The formula tree is a representation of the formula through a tree structure, which mainly includes two categories: symbol layout tree and operation tree. In order to adapt to task requirements, the tree structure definitions used in different scenarios may be slightly different. Among them, the symbol layout tree is constructed according to the position of mathematical symbols and their relative position relationship, and the position relationship between formula symbols is saved in 6 position categories (superscript, subscript, right extension, above, below, and inside); the operation tree is constructed according to the operation order and hierarchy of the mathematical formula, with leaf nodes storing operands and non-leaf nodes storing operators, which can better reflect the logical structure of the formula and there is no problem of ambiguous mathematical definition.

[0029] Step S500: identifying and extracting formula variable semantics from the formula description text according to the formula structure expression;

[0030] Based on the formula structure expression, all variables in the formula are identified. Variables are usually represented by letters or symbols. The extraction of variables relies on regular expressions or image processing technology to identify variable names in mathematical expressions. Using a large language model, the meaning of variables is understood through context analysis. For example, in the formula Q = ω*L, the description text indicates that Q is the "total quality factor", ω is the "angular frequency", and L is the "inductance". Through deep semantic analysis of the formula description text, the actual meaning of each variable is extracted, which usually includes the extraction of semantic information such as the definition, alias, abbreviation or explanation of the variable. After semantic analysis, the semantics of the formula variables are extracted. The semantic information corresponding to each variable will provide a basis for subsequent semantic matching and tree structure construction.

[0031] Step S600: Based on the variable proofreading and alignment, the formula variable semantics are integrated into the symbolic formula tree to form a semantic formula tree;

[0032] Variable proofreading and alignment refers to matching the variables extracted from the formula structure expression with the variables identified from the formula description text, and ensuring that the variables in the formula are semantically consistent with the variables extracted from the formula description text through methods such as same-name matching and synonymous variable identification.

[0033] On the basis of variable proofreading and alignment, the semantics of each variable is integrated into the symbolic formula tree. Each node of the symbolic formula tree (representing a variable, constant or operator) will contain the mathematical symbol of the variable and the semantic description of the variable, forming a semantic formula tree after integration.

[0034] The semantic formula tree is an improved formula tree realized by semantic enhancement proposed by the present method. In the present application, the semantic formula tree is obtained by adding and supplementing the semantics of each variable on the basis of the symbolic operation tree of the formula by means of large model semantic extraction. The obtained semantic formula tree not only represents the operation structure information of the formula through the tree structure, but also can represent the variable semantic information of the formula, and the semantics is presented in natural language and can be directly read and understood.

[0035] Step S700: Based on the semantic formula tree, formula matching is performed through a similarity matching strategy, and formula downstream applications are implemented according to the similarity matching results, wherein the formula downstream applications include formula retrieval and formula recommendation, and formula matching includes structural matching and semantic matching.

[0036] Formula matching is a basic application implemented under the premise of formula representation. It achieves a quantitative description of the similarity between formulas through a variety of similarity calculation methods, such as matching formulas containing the same semantic variables, or matching formulas with similar structures. It is the basis for subsequent advanced applications such as formula duplication checking and formula-based document recommendation.

[0037] Formula matching is based on formula representation. Different formula representation methods and different similarity calculation methods constitute different matching methods. Generally speaking, they can be divided into several main strategies: text-based, structure-based, image-based, Embedding-based, and multimodal-based. Specifically, the text-based strategy converts the symbolic language formula into a one-dimensional sequence tag and relies on the traditional text similarity calculation method for retrieval; the structure-based strategy mainly relies on the structural information similarity of the formula for retrieval, and is often implemented with the help of formula trees, such as the root-leaf path of the formula tree, the edit distance between the formula trees to be matched, and the combined features of the symbol layout tree parsing results; the image-based strategy regards formula matching as an image matching task and uses the similarity calculation method between images; the Embedding-based strategy converts one or more of the formula's text, structure, or image information into a vector representation, and achieves comparison by calculating the similarity between vectors. The multimodal-based strategy aims to match and compare formula information of more than one modality. The Embedding method is a common method for implementing multimodal matching.

[0038] Furthermore, if Figure 2 As shown, variable calibration and alignment are performed, including:

[0039] Step S610: extracting a first variable set from the LaTeX expression, and extracting a second variable set from the formula description text;

[0040] Step S620: performing variable matching of the same name on the first variable set and the second variable set to obtain a variable set of the same name;

[0041] Step S630: performing synonymous variable identification on the first variable set and the second variable set, establishing a synonymous variable mapping relationship, and obtaining a synonymous variable set;

[0042] Step S640: Filter, remove duplicates and sort the variable sets with the same name and the variable sets with synonyms as the successfully proofread variables.

[0043] Variables are identified by analyzing symbols in LaTeX expressions. In this process, regular expressions, lexical analyzers, or syntax trees can be used to extract all letters or symbols (usually single characters or combined characters) in the formula as variables, and all variables identified from LaTeX expressions are integrated into a first variable set.

[0044] The formula description text usually follows the formula, or explains the formula in detail in the text. Natural language processing (NLP) technology, especially named entity recognition (NER) algorithm, is used to extract all variables related to the formula from the formula description text. NER can recognize the variable names in the text and distinguish different entities in the formula (for example, physical quantities, constants, etc.). The natural language description of the variables is extracted from the formula description text, and the variable name or symbol is associated with its meaning to form a second variable set.

[0045] Variables with the same name refer to variables with exactly the same names in LaTeX expressions and formula description texts. For example, Q appears in both sets and represents the same physical quantity. Set operations, such as finding the intersection, can be used to find variables with the same names in the first variable set and the second variable set. Specifically, the variable names in the first variable set and the second variable set are compared one by one. If there are variables with exactly the same names in the two sets, then these variables are variables with the same name. All variables with exactly the same names are integrated to form a set of variables with the same name.

[0046] Synonymous variables refer to variables in LaTeX expressions and formula description texts that, despite having different names, actually represent the same physical quantity or mathematical entity. For example, v and velocity may represent the same physical quantity, namely, speed, in different documents. The semantic relationship between variables in LaTeX expressions and variables in formula description texts is analyzed using a large language model or a specialized word sense matching algorithm. If two variables have the same meaning in the context, that is, they have the same semantics, they can be considered synonymous variables. After identifying synonymous variables, a mapping relationship is established between them for subsequent use.

[0047] The variable sets with the same name and synonymous variables identified in the previous steps are processed comprehensively, including screening, deduplication and sorting, to finally obtain a successfully proofread variable set. The variable names and semantics in this set have been proofread and are completely consistent.

[0048] Furthermore, if Figure 3 As shown, before generating the symbolic formula tree, it includes:

[0049] Step S410: traverse the formula structure expression and extract operators, variables, and values;

[0050] Step S420: Mapping operators to operation nodes, mapping variables to index nodes, and mapping values ​​to value nodes;

[0051] Step S430: Identify nodes of the symbolic formula tree according to operation nodes, indicator nodes, and value nodes.

[0052] Traverse the formula structure expression and recursively visit each node. Each node represents an operator, a variable or a value. The traversal order is usually in-order traversal or pre-order traversal to ensure that each element of the formula can be extracted in sequence. In the formula, the operator is usually an operation symbol such as "+" and "-", indicating different mathematical operations; the variable is usually a single letter, such as a, b, c, indicating an unknown quantity or physical quantity; the value is a constant or number in the formula, such as 1, 1.5, π, etc.

[0053] In the symbolic formula tree, all elements are mapped to tree nodes, and different types of elements are mapped to different types of nodes. Specifically, operators are mapped to operation nodes, which represent mathematical operations in the formula; variables are mapped to indicator nodes, which represent unknown quantities or physical quantities in the formula; constants are mapped to numerical nodes, which represent constants in the formula.

[0054] Use the created nodes, including operation nodes, indicator nodes, and value nodes, to organize them into a symbolic formula tree, the structure of which reflects the mathematical operation structure of the formula.

[0055] Furthermore, generating the symbolic formula tree includes:

[0056] Step S440: extracting the preset priority operator of the formula structure expression and generating a root node;

[0057] Step S450: recursively parse the left and right expressions of the root node to generate a left subtree structure and a right subtree structure;

[0058] Step S460: extracting child nodes based on the left subtree structure and the right subtree structure, respectively, to obtain a plurality of left child nodes and a plurality of right child nodes;

[0059] Step S470: Link the multiple left child nodes and the multiple right child nodes to the root node respectively to generate the symbolic formula tree.

[0060] Mathematical formulas usually have clear operator priority rules. The operator priority determines the order of operations. For example, multiplication and division have higher priority than addition and subtraction. Operations in parentheses are performed first. The formula structure expression is traversed, all operators are identified and their priorities are marked, and preset priority operators are generated. In the expression, the operator with the highest priority among the preset priority operators will be used as the root node to build the symbolic formula tree.

[0061] According to the position of the root node, the formula structure expression is divided into a left part and a right part. The left part is the expression on the left side of the root node, that is, the left subtree structure, and the right part is the expression on the right side of the root node, that is, the right subtree structure. The left subtree structure and the right subtree structure are recursively parsed respectively. Each sub-expression may contain multiple operators. The recursive parsing process is similar to the tree construction process. Operators are selected as nodes at each level, and then the remaining parts are parsed. Each sub-expression is recursively processed until all sub-expressions are parsed into leaf nodes. The final left subtree structure and right subtree structure represent the left part and the right part of the formula respectively.

[0062] In recursive parsing, each operator node of the left subtree structure and the right subtree structure will generate child nodes, which can be operators, variables, values, etc. Each operator node corresponds to its sub-expression, and the operator node connects the left and right operands to ensure that the mathematical relationship of the formula is correctly represented.

[0063] The extracted left child node and right child node are connected to the left and right child node positions of the root node respectively. The root node represents the main operator of the formula, and the left and right subtrees represent the two parts of the formula respectively. After completing the construction of the symbolic formula tree, the entire tree can fully express the mathematical structure and operation relationship of the formula. Each node and its child nodes reflect the operation relationship between the elements in the formula.

[0064] Furthermore, if the formula includes a logical relationship symbol, the root node is the logical relationship symbol; if the formula does not include a logical relationship symbol, the root node is a preset priority operator.

[0065] In an expression, the operator with the highest priority among the preset priority operators will be used as the root node to build the symbolic formula tree. For example, multiplication and division have higher priority than addition and subtraction. However, when the formula contains logical relationship symbols such as equal sign, inequality sign, greater than or less than, these logical relationship symbols will be used as root nodes first. This is the core of the tree construction and determines the structure of the entire formula.

[0066] Further, the extracting of child nodes based on the left subtree structure and the right subtree structure to obtain a plurality of left child nodes and a plurality of right child nodes respectively includes:

[0067] Step S461: traverse multiple operators of the left subtree structure, split the sub-expressions according to the mathematical operation priority, and generate multiple left child nodes;

[0068] Step S462: traverse multiple operators of the right subtree structure, split the sub-expressions according to the mathematical operation priority, and generate multiple right child nodes.

[0069] Since the left subtree structure is already a part of the formula structure parsed recursively, in the left subtree, the operator is the key node that determines the formula structure. When traversing multiple operators in the left subtree, for each operator in the formula, the order of operations needs to be determined based on the priority of its mathematical operation. When traversing each operator, the sub-expressions are split from high to low according to the priority. For operators with lower priority, such as addition and subtraction, they are first split into sub-expressions, and then each sub-expression is processed recursively. At each split, a new left child node is generated. The left child node will contain the current operator and its operands, forming a hierarchical structure of the formula. The multiple left child nodes finally generated reflect the structure of the formula and the priority of the operator, ensuring that the expression of the entire symbolic formula tree is accurate.

[0070] The structure of the right subtree is similar to that of the left subtree. It also contains multiple operators. When traversing these operators, they must be split according to the mathematical operation priority to ensure that each operator in the formula can be processed correctly. For the sake of brevity, the extraction process of the right child node is not described here.

[0071] Furthermore, the step of integrating the semantics of formula variables into the symbolic formula tree to form a semantic formula tree includes:

[0072] Step S610: Obtain formula variable semantics of the same-name variable set, synonymous variable set, and unit-aligned variable set as proofreading formula variable semantics;

[0073] Step S620: Integrate the semantics of the proofreading formula variables into the corresponding nodes of the symbolic formula tree to form a semantic formula tree.

[0074] The semantics of the same-name variable sets, synonymous variable sets, and unit-aligned variable sets are merged into a unified proofreading formula variable semantics to ensure that the semantic description of the same variable in the formula is consistent and clear. For example, if v and velocity both represent speed, their semantics will be merged into a unified "speed" concept.

[0075] Traverse each node in the symbolic formula tree, especially each indicator node, that is, the variable node. These nodes are where the variables in the formula are represented. Their semantic information needs to be integrated. For each variable node, find its corresponding item in the proofreading formula variable semantics, and integrate the variable semantics into the corresponding node in the symbolic formula tree. For example, if there is a Q node in the tree, and Q is defined as "total quality factor" in the formula variable semantics, then attach this semantic information to the Q node. After completing the integration of variable semantics, a semantic formula tree is formed, in which each node not only retains mathematical operation information, but also contains a natural language description of the formula variables, making the formula easier to understand.

[0076] Furthermore, the nodes of the semantic formula tree are of three types: operation, indicator or value, the connection line represents the operation relationship, and the node stores the variable semantics of the formula.

[0077] The generated semantic formula tree is composed of different types of nodes, including operation nodes, indicator nodes and value nodes. These nodes establish operation relationships through connections to ensure that the structure and semantics of the formula can be expressed. Among them, operation nodes represent operators in mathematical formulas, such as plus, minus, and division. Operation nodes are the core part of the formula and define the relationship between operands; indicator nodes represent variables or physical quantities in the formula, usually unknown numbers or key elements in the formula. Each variable node not only stores the variable name in the formula, but also contains its corresponding semantic information; value nodes represent constants or fixed values ​​in the formula. Value nodes directly participate in the calculation, but do not involve the change of variables. They are the constant part of the operation. The semantic formula tree is formed by these different types of nodes and the relationships between them. Each node contains the structural information of the formula and the semantic description of the variable, so that the formula is not only a mathematical expression, but also has a natural language interpretation.

[0078] Furthermore, the formula matching is performed based on the semantic formula tree through a similarity matching strategy, and the downstream application of the formula is realized according to the similarity matching result, wherein the downstream application of the formula includes formula retrieval and formula recommendation, including:

[0079] Step S710: Based on the semantic formula tree, compare the same semantic variables of the instance formula and multiple formulas to be matched to obtain the matching numbers of multiple semantic variables;

[0080] Step S720: Arrange the multiple formulas to be matched in descending order according to the matching quantities of the multiple semantic variables to obtain an initial formula arrangement list;

[0081] Step S730: traverse the initial formula arrangement list, calculate the edit distance of the semantic formula tree with the instance formula respectively, and generate the structural similarity according to the edit distance calculation result;

[0082] Step S740: Arrange the to-be-matched formulas with the same number of semantic variables in the initial formula arrangement list in descending order according to their structural similarity to obtain a target formula arrangement list;

[0083] Step S750: performing formula downstream application according to the target formula arrangement list, wherein the formula downstream application includes formula retrieval and formula recommendation.

[0084] The instance formula is the target formula that needs to be matched. All variable nodes and their corresponding semantic information in the instance formula are extracted from the semantic formula tree. These variables usually contain semantic descriptions in natural language. For each formula to be matched, all variable nodes and their corresponding semantic descriptions are also extracted from its semantic formula tree. By comparing the semantic descriptions of the variables in the instance formula and the formula to be matched, it is determined whether they match. The matching criterion is the similarity of the semantic description. Exact matching or matching methods based on semantic similarity can be used. For example, natural language processing tools are used to calculate semantic similarity. For each pair of instance formulas and formulas to be matched, the number of matching semantic variables between them is calculated. Finally, a matching number with the instance formula is generated for each formula to be matched. This number represents the similarity between the instance formula and the formula to be matched.

[0085] All formulas to be matched are sorted from high to low according to the number of matches. The higher the number of matches, the higher the semantic similarity between the formulas. They are sorted in descending order, that is, formulas with more matches are placed at the front. After sorting, an initial formula arrangement list is generated, which contains all formulas to be matched in descending order according to the number of matches. This list helps to prioritize formulas with the most similar semantics to the instance formula.

[0086] The edit distance is the minimum number of operations required to convert one formula into another. For the edit distance calculation of the semantic formula tree, a method similar to the tree edit distance can be used to compare the structural differences between the two trees, including node types, node contents, etc. A value is obtained through the edit distance calculation, which reflects the degree of difference between the semantic formula tree of the formula to be matched and the semantic formula tree of the instance formula. The smaller the edit distance, the more similar the structures of the two formulas.

[0087] Formula structural similarity is the inverse or normalized form of the edit distance. For example, if the edit distance is smaller, the structural similarity is greater, which means that the formula structures are more similar. A structural similarity is calculated for each formula to be matched. This value reflects the structural similarity between the formula and the instance formula, and helps to determine the degree of structural similarity between the formulas.

[0088] For formulas to be matched with the same number of semantic variables, they are sorted in descending order according to their structural similarity. The higher the structural similarity, the more similar the structure is to the instance formula, so it should be ranked in front. When sorting, the numerical value of the structural similarity can be used for direct comparison. The higher the ranking of the formula, the closer the structure is to the instance formula. Finally, a target formula arrangement list is generated, which contains all the formulas to be matched, sorted by priority according to the number of matching semantic variables and structural similarity. This list can be used for further downstream applications of formulas such as subsequent formula retrieval and formula recommendation.

[0089] Furthermore, the method of performing formula matching based on the semantic formula tree by a similarity matching strategy and determining the formula representation according to the similarity matching result also includes:

[0090] Step S750: extracting variable semantic text and formula tree structure based on the semantic formula tree;

[0091] Step S760: performing vector conversion on the variable semantic text and the formula tree structure to generate an instance embedding vector of the instance formula and multiple embedding vectors to be matched of multiple formulas to be matched;

[0092] Step S770: performing similarity calculation on the instance embedding vector and the multiple embedding vectors to be matched to obtain multiple vector similarities;

[0093] Step S780: adjusting the target formula arrangement list based on the multiple vector similarities.

[0094] Each variable node in the semantic formula tree contains a semantic description related to the variable. All variable nodes in the semantic formula tree are traversed to extract the variable semantic text. The structure of the formula tree represents the organization of operators (such as addition, multiplication, equal sign, etc.) and their operands (variables or constants) in the formula. The tree structure of each formula reflects the order of operation and hierarchical relationship of the formula. By traversing the nodes of the semantic formula tree, the hierarchical structure of the formula can be extracted.

[0095] For variable semantic text, natural language processing technology can be used for vectorization, converting each variable semantic text into a vector representation of fixed dimensions; the structure of the formula tree can be converted into an embedding vector of a graph structure or a tree structure, and a graph neural network can be used to map the formula tree structure to a vector space. For the instance formula and each formula to be matched, vector conversion is performed based on its variable semantic text and formula tree structure respectively. The generated instance embedding vector and embedding vector to be matched will represent the overall semantic and structural information of the formula, and these vectors can be used to calculate the similarity between formulas.

[0096] For each formula to be matched, the similarity between its embedding vector and the embedding vector of the instance formula is calculated. For example, the cosine similarity method is used to calculate the cosine values ​​of the two vectors. The closer to 1, the more similar they are. The calculation result will represent the semantic and structural similarity between the instance formula and each formula to be matched. After calculation, multiple vector similarities between the instance formula and multiple formulas to be matched are obtained. These values ​​represent the similarity between the formulas and can be used to select the most similar formula.

[0097] The target formula ranking list is adjusted according to the similarity value to ensure that the formulas with higher similarity are ranked first. The sorting rule is descending order, that is, the formulas with higher similarity are ranked first. Finally, an adjusted target formula ranking list is generated, which includes the formulas most similar to the instance formulas and is prioritized according to similarity.

[0098] In summary, the mathematical formula representation and matching method based on semantic enhancement provided in the embodiments of the present application has the following technical effects:

[0099] By locating the formula image of the target formula in the target text and extracting the relevant formula description text, mathematical formula information can be automatically extracted from the text, which provides a basis for subsequent formula parsing and semantic analysis, avoids manual intervention, and improves the degree of automation; by using OCR technology to recognize formula images and convert them into LaTeX expressions, the formula can be converted from image format to operable text format, thereby providing a standardized representation for the structural parsing and further processing of the formula; by using the locally deployed open source large language model to extract the structural expression of the formula from the LaTeX expression, it is possible to understand the components of the formula and their relationships, thereby enhancing the accuracy and automation level of formula parsing; based on the formula structural expression, a symbolic formula tree is generated, and the structural information of the formula is stored in a tree structure in json format, which enables the various elements of the formula and their relationships to be clearly and hierarchically represented, which is helpful for subsequent formula processing and matching; By identifying and extracting the semantic information of formula variables from the formula description text, each variable in the formula not only contains numeric symbols but also carries the corresponding natural language description, which provides strong support for subsequent semantic alignment and matching; on the basis of variable proofreading and alignment, the semantics of formula variables are integrated into the symbolic formula tree to generate a semantic formula tree. This process makes the formula not only reflect its operation structure but also contain the natural language description of the variables, which significantly enhances the semantics of the formula; based on the semantic formula tree, formula matching is performed through a similarity matching strategy, combining structural matching and semantic matching to determine the similarity of formulas, find the most similar formulas, and provide accurate matching results for formula retrieval and application. By combining the dual matching method of structure and semantics, formulas similar to instance formulas can be identified more accurately. Semantic matching avoids the mismatching that may occur in matching based solely on structure, and improves the accuracy and flexibility of matching.

[0100] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A mathematical formula representation and matching method based on semantic enhancement, characterized in that: The method comprises: Locate the formula image of the target formula in the target text, and extract the formula description text of the target formula; Use OCR technology to recognize formula images and convert the recognized formulas into LaTeX expressions; Use the locally deployed open source large language model to extract formula structure expressions from LaTeX expressions; Based on the formula structure expression, generate a symbolic formula tree of the formula, where the symbolic formula tree is stored in json format; According to the formula structure expression, the formula variable semantics is identified and extracted from the formula description text; On the basis of variable proofreading and alignment, the semantics of formula variables are integrated into the symbolic formula tree to form a semantic formula tree; Based on the semantic formula tree, formula matching is performed through a similarity matching strategy, and the downstream application of the formula is realized according to the similarity matching result, wherein the downstream application of the formula includes formula retrieval and formula recommendation, and the formula matching includes structural matching and semantic matching.

2. A mathematical formula representation and matching method based on semantic enhancement as claimed in claim 1, characterized in that: Perform variable verification and alignment, including: Extract the first variable set from the LaTeX expression and extract the second variable set from the formula description text; Performing variable matching on the first variable set and the second variable set to obtain a variable set with the same name; Perform synonymous variable identification on the first variable set and the second variable set, establish a synonymous variable mapping relationship, and obtain a synonymous variable set; The variable sets with the same name and the variable sets with synonyms are screened, deduplicated and sorted as the proofreading success variables.

3. The method for representing and matching mathematical formulas based on semantic enhancement according to claim 1, characterized in that: Before generating the symbolic formula tree, include: Traverse the formula structure expression and extract operators, variables, and values; Map operators to operation nodes, map variables to indicator nodes, and map values ​​to value nodes; The symbolic formula tree is labeled with nodes according to operation nodes, indicator nodes, and value nodes.

4. A method for representing and matching mathematical formulas based on semantic enhancement as claimed in claim 3, characterized in that: Generating the symbolic formula tree includes: Extract the preset priority operators of the formula structure expression and generate the root node; Recursively parse the left and right expressions of the root node to generate the left subtree structure and the right subtree structure; Based on the left subtree structure and the right subtree structure, respectively extract child nodes to obtain multiple left child nodes and multiple right child nodes; The multiple left child nodes and the multiple right child nodes are linked to the root node respectively to generate the symbolic formula tree.

5. A mathematical formula representation and matching method based on semantic enhancement as claimed in claim 4, characterized in that: If the formula includes a logical relationship symbol, the root node is the logical relationship symbol; if the formula does not include a logical relationship symbol, the root node is a preset priority operator.

6. A mathematical formula representation and matching method based on semantic enhancement as claimed in claim 4, characterized in that: The extracting of child nodes based on the left subtree structure and the right subtree structure to obtain a plurality of left child nodes and a plurality of right child nodes includes: Traversing multiple operators of the left subtree structure, splitting sub-expressions according to mathematical operation priorities, and generating multiple left child nodes; Multiple operators of the right subtree structure are traversed, sub-expressions are split according to mathematical operation priorities, and multiple right child nodes are generated.

7. The method for representing and matching mathematical formulas based on semantic enhancement according to claim 1, characterized in that: The step of integrating the semantics of formula variables into the symbolic formula tree to form a semantic formula tree includes: Obtain formula variable semantics of the same-name variable set, synonymous variable set, and unit-aligned variable set as proofreading formula variable semantics; Integrate the semantics of the proofreading formula variables into the corresponding nodes of the symbolic formula tree to form a semantic formula tree.

8. The method for representing and matching mathematical formulas based on semantic enhancement according to claim 1, characterized in that: The nodes of the semantic formula tree are of three types: operation, indicator or value. The connecting lines represent the operation relationship, and the nodes store the variable semantics of the formula.

9. The method for representing and matching mathematical formulas based on semantic enhancement according to claim 1, characterized in that: The method is based on the semantic formula tree, performs formula matching through a similarity matching strategy, and implements formula downstream applications according to the similarity matching results, wherein the formula downstream applications include formula retrieval and formula recommendation, including: Based on the semantic formula tree, compare the same semantic variables of the instance formula and multiple formulas to be matched to obtain the matching number of multiple semantic variables; Arrange the multiple formulas to be matched in descending order according to the matching numbers of the multiple semantic variables to obtain an initial formula arrangement list; Traversing the initial formula arrangement list, respectively calculating the edit distance of the semantic formula tree with the instance formula, and generating the structural similarity according to the edit distance calculation result; For the formulas to be matched with the same number of semantic variables in the initial formula arrangement list, arrange them in descending order according to their structural similarity to obtain a target formula arrangement list; The formula downstream application is performed according to the target formula arrangement list, wherein the formula downstream application includes formula retrieval and formula recommendation.

10. The method for representing and matching mathematical formulas based on semantic enhancement according to claim 9, characterized in that: The formula matching is performed based on the semantic formula tree by a similarity matching strategy, and further includes: Based on the semantic formula tree, extract the variable semantic text and formula tree structure; Performing vector conversion on the variable semantic text and the formula tree structure to generate an instance embedding vector of the instance formula and multiple embedding vectors to be matched of multiple formulas to be matched; Performing similarity calculation on the instance embedding vector and the multiple embedding vectors to be matched to obtain multiple vector similarities; The target formula arrangement list is adjusted based on the multiple vector similarities.

Images