A method and system for recognizing mathematical formulas in a document

Abstract

The application provides a mathematical formula recognition method and system in a document. Different formulas are verified respectively to determine whether the different formulas have principle errors, so that verified formulas with correct recognition are obtained, and verified formulas with incorrect recognition are obtained. Whether the same parameters exist in the different formulas as in the verified formulas is recognized, whether the parameters are complete is determined, so that adjacent fields of the formulas are recognized, incomplete formulas are corrected, the accuracy of formula recognition is improved, and the document recognition efficiency is improved.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and in particular relates to a method and system for recognizing mathematical formulas in documents. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the development of the times, more and more papers, applications, reports, and other documents appear in our lives, and they are often displayed, communicated, or stored in the form of images. Formulas are often a very important part of these papers and applications. When recognizing or editing formulas, accurately determining which part of the image represents the formula is a crucial step. Many experts and scholars have conducted in-depth research on this issue. Traditional methods often fail to fully consider the characteristics of formula image regions, resulting in non-targeted recognition methods that cannot maintain high accuracy in recognizing formula image regions. Furthermore, many traditional methods employ complex algorithms or require a large number of training samples, significantly increasing computational resource consumption. Therefore, finding a targeted and simple formula image region recognition method that can improve recognition accuracy is an urgent and meaningful task. Summary of the Invention

[0004] To overcome the shortcomings of the prior art, the present invention provides a method and system for recognizing mathematical formulas in documents, which verifies each formula separately, thereby improving the accuracy of formula recognition and increasing document recognition efficiency.

[0005] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solution: a method for recognizing mathematical formulas in documents, comprising:

[0006] Obtain multiple formulas to be identified from the original document and verify them individually to obtain multiple formulas that pass verification and multiple formulas that fail verification;

[0007] Analyze whether the formula for failing the verification contains the same parameters as the formula for passing the verification.

[0008] If it exists, the verification pass formula with the same parameters will be substituted into the current verification fail formula for solving. If the solution is successful, the current verification fail formula will be used as the verification pass formula.

[0009] If the solution fails, check if there are any identical parameters in the current failed formulas. If there are identical parameters, substitute the parameters from other failed formulas into the current failed formula to obtain a new formula.

[0010] The new formula is brought into other verification failed formulas with the same parameters, if the solving is successful, the new formula is taken as a verification passed formula, until all formulas are verified successfully, and all verification successful formulas generate a new document.

[0011] In a second aspect, the embodiments of the present application provide a mathematical formula recognition method and system in a document, comprising:

[0012] An acquisition module: acquiring a plurality of formulas to be recognized in an original document and verifying them respectively, obtaining a plurality of verification passed formulas and a plurality of verification failed formulas;

[0013] A first parameter judgment module: analyzing whether there is a same parameter in the verification passed formulas in the verification failed formulas;

[0014] A first solving module: if there is, the verification passed formulas with the same parameters are brought into the current verification failed formulas for solving, if the solving is successful, the current verification failed formula is taken as a verification passed formula;

[0015] A second parameter judgment module: if the solving is not successful, it is judged whether there is a same parameter in the current verification failed formulas and other verification failed formulas, if there is a same parameter, the current verification failed formula is brought into other verification failed formulas with the same parameters to obtain a new formula;

[0016] A second solving module: the new formula is brought into other verification failed formulas with the same parameters for solving, if the solving is successful, the new formula is taken as a verification passed formula, until all formulas are verified successfully, and all verification successful formulas generate a new document.

[0017] In a third aspect, the embodiments of the present application provide a computer device, comprising a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the computer device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to execute the steps of the above mathematical formula recognition method in a document.

[0018] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by the processor to execute the steps of the above mathematical formula recognition method in a document.

[0019] The above one or more technical solutions have the following beneficial effects:

[0020] In the present application, by identifying a plurality of formulas in the original document, and verifying each formula respectively, it is judged whether different formulas have principle errors, so as to obtain the verified formulas which are correctly identified, and obtain the verified formulas which are not correctly identified; by identifying whether each verified formula has the same parameters as the verified formulas, it is judged whether the parameter identification is complete, so as to correct the incomplete formulas by identifying the adjacent fields of the formulas, and solve the problem of incomplete formula identification caused by segmentation.

[0021] Advantages of the additional aspects of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings, which form a part of this specification, are included to provide a further understanding of the application and are incorporated herein by reference. The illustrations are shown schematically and are not intended to limit the application in any way.

[0023] Figure 1 A flow chart of the mathematical formula identification method in the embodiment one of the present application;

[0024] Figure 2 A flow chart of the adjacent field identification in the embodiment one of the present application;

[0025] Figure 3 A flow chart of the error parameter correction in the embodiment one of the present application;

[0026] Figure 4 A flow chart of the complete parameter correction in the embodiment one of the present application. DETAILED DESCRIPTION

[0027] It should be noted that the following detailed description is merely exemplary and is intended to provide further description of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0028] It should be noted that the terms used herein are merely intended to describe specific embodiments and are not intended to limit the exemplary embodiments according to the present application.

[0029] In the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0030] Embodiment one

[0031] As shown in Figures 1-4 The present embodiment discloses a mathematical formula identification method in a document, comprising:

[0032] Obtaining a plurality of formulas to be identified in the original document and verifying each of the plurality of formulas to obtain a plurality of verified formulas and a plurality of unverified formulas;

[0033] Analyzing whether there is a same parameter in the unverified formula as in the verified formula;

[0034] If there is, the verified formula with the same parameter is brought into the current unverified formula for solving, and if the solving is successful, the current unverified formula is taken as a verified formula;

[0035] If the solving is not successful, it is determined whether there is a same parameter in the current unverified formula as in other unverified formulas, and if there is a same parameter, the current unverified formula is brought into the other unverified formula with the same parameter to obtain a new formula;

[0036] The new formula is brought into the other unverified formula with the same parameter for solving, and if the solving is successful, the new formula is taken as a verified formula, until all formulas are verified successfully, and all verified formulas are generated into a new document.

[0037] In the embodiment, the parameter refers to a plurality of values participating in operation and an operation result in the formula, which can be a variable or a combination of a variable and a constant value. The variable can be represented by any one or more of English letters, characters and symbols. Formula solving is that if there is a same parameter in the current formula as in other formulas, the value representing the same parameter in the other formula is substituted into the formula, and the formula is solved by using the corresponding parameter, so as to verify whether the formula in the document is accurate by using the image recognition technology. For example, in the formula ax+by=cz, a, b and c are all constants, and x, y and z are all variables, wherein the parameter expression is the variables x, y and z, or the combination of the variables and the constants, i.e. ax, by and cz. When there is a same parameter in other formulas, such as x=2a, the formula can be directly substituted and solved. Alternatively, x, y and z can be replaced by any one or more combinations of characters, English characters and symbols.

[0038] In the embodiment, a plurality of formulas in the original document are identified, and each of the formulas is verified to determine whether different formulas have principle errors, so as to obtain verified formulas correctly identified and unverified formulas to be confirmed whether correct.

[0039] Specifically, the formula with the principle error refers to a formula lacking an operator symbol and a mathematical formula not conforming to mathematical axioms and laws. The identification of the operator symbol is performed by using a document repair function, and the mathematical axioms and laws need to be stored in a mechanism in advance. For example, in the identification of a fractional formula, the denominator cannot be 0.

[0040] The two-step method of text detection and text recognition can be used to realize the recognition of scene text, and a large amount of manual knowledge is needed when the results of the two steps are fused. Among them, the principle of extracting text in the picture based on OCR recognition is pre-processing: processing the image containing text to facilitate subsequent feature extraction and learning. The main purpose of this process is to reduce the useless information in the image to facilitate the subsequent processing. In this step, there are usually sub-steps such as graying (if it is a color image), noise reduction, binarization, character segmentation and normalization. Feature extraction and dimension reduction: features are key information for recognizing text, and each different text can be distinguished from other texts through features. Classifier design, training and actual recognition: the classifier is used for recognition, that is, for the second step, a text image is extracted and given to the classifier, and the classifier classifies it. Post-processing: post-processing is used to optimize the classification result and tell you which text the feature should be identified as.

[0041] In the embodiment, specifically includes:

[0042] S1: analyze whether there is a same parameter in the formula as in the formula that passes the verification, and when there is not, identify the adjacent fields to determine whether the parameters of each formula are complete, and when they are not, correct them;

[0043] S2: when there is a same parameter in the formula that does not pass the verification as in the formula that passes the verification, substitute one or more formulas that pass the verification into the formula that does not pass the verification with the corresponding parameters to solve, and if the solution is successful, the current formula that does not pass the verification is used as the formula that passes the verification;

[0044] S3: when the formula solving fails, identify whether there is a same parameter between the formula that does not pass the verification (that is, the formula that does not pass the verification has the same parameter as the formula that passes the verification, but the solving fails) and each formula that does not pass the verification, when there is a same parameter, substitute the formula that fails to solve into each formula that does not pass the verification, and update to obtain a new formula;

[0045] S4: find the common parameters in the parameters of the obtained new formula and the parameters of the formulas that do not pass the verification before updating (that is, all formulas that do not pass the verification before obtaining the new formula), and substitute the obtained new formula into the formulas that do not pass the verification before updating with the same parameters to solve, and when the formulas that do not pass the verification before updating are solved successfully, they are used as formulas that pass the verification;

[0046] S5: Otherwise, determine whether the parameters in the verification failure formula in S3 (i.e., there are the same parameters in the verification success formula, but the verification failure formula fails to solve) exist in other verification success formulas and verification failure formulas, and when it is determined that yes, correct the non-existing parameters; the correction is to replace the parameters that exist in other verification success formulas with the parameters that all exist in the verification success formula. When the number of parameters that all exist is multiple, the parameter with the highest frequency of occurrence is selected for replacement. The basis is that the parameters that repeatedly appear in the document are generally the research objects of the article, such as the parameter "k" that repeatedly appears in the article "Once Measurement Restores the Uniqueness of Segment Constant Conductivity and Kernel".

[0047] S6: The updated verification success formula and / or verification failure formula with the same parameters after correction is substituted into the formula (i.e., the verification failure formula in step S5) to solve, and when the verification failure formula is successfully solved, it is used as the verification success formula; when the solving is unsuccessful, the parameters that do not exist in the verification failure formula are again corrected, i.e., the parameters that all exist in the verification success formula, until all formulas are successfully solved; the original document is regenerated according to all the solved formulas, and the format of each formula is adjusted.

[0048] In S1 of the embodiment, it is determined whether the same parameters exist in the verification failure formula as in the verification success formula, and when they do not exist, the adjacent fields are identified to determine whether the parameters of each formula are complete, and when they are not complete, they are corrected, specifically:

[0049] The adjacent fields of each formula are identified by the one or more fields before the starting character of the parameter and the one or more fields after the ending character of the parameter. The one or more adjacent fields before the starting character of the parameter are identified by the starting character of the parameter, and the one or more adjacent fields after the ending character of the parameter are identified by the ending character of the parameter. The characters are supplemented using the document recognition function. The number of adjacent fields identified can be determined according to whether the adjacent fields are the same or similar, and the character types can include letters, words, character symbols, and punctuation marks.

[0050] For example, the formula 3x+5=90-9y, with 3 as the first starting character and y as the ending character of the parameter, and the characters in the middle are the positions of the multiple fields. The definition of a complete field is that the number part and the symbol part of the formula all exist, and the definition of an incomplete field is that the operator symbol is missing.

[0051] When the number of strokes of the adjacent fields is the same or the stroke value difference is within 1-2, it is determined that the two adjacent character types are the same or similar. Based on the feature extraction and dimension reduction of the OCR recognition extraction of the characters in the picture, the number of strokes of the adjacent field characters corresponds to different feature values, which can be used for judgment.

[0052] Among them, the document recognition repair function is a document repair scheme based on user data flow sector structure to supplement the incomplete places. The user data flow sector generally refers to the case where the data flow sector stored in the document is damaged due to the user's addition, deletion, modification and other operations on the document when generating the file. For this part of data, the sector position of the structure stored in the document needs to be determined in combination with the doc file structure, and then it is judged which structure error or loss causes the file to be damaged (that is, the formula is incomplete), and then the damaged structure is repaired in combination with the fixed information content stored therein.

[0053] When the regenerated parameter after identifying the adjacent field does not exist in the multiple verification passing formulas, manual correction is performed according to the existing parameters in the verification failing formulas to the parameters existing in the verification passing formulas. Among them, the existing parameters can be directly used as the identification result, or part of the existing parameters can be used as the identification result.

[0054] In the embodiment S1, when the same parameter does not exist in the verification passing formula and the verification failing formula, the similar existing parameter is corrected. When the same parameter does not exist in all formulas, the similar existing parameter is corrected to the identification result. Among them, the corresponding identification result can be selected according to the similar existing parameter, or the identification result can be directly replaced by the existing parameter, so as to solve the case that the parameters in the formula exist in the respective abbreviations or aliases.

[0055] Among them, the similar existing parameter character is a character with a character stroke difference within 1-2, and whether two parameters are similar is judged according to this standard.

[0056] In the embodiment S6, after correcting the incomplete parameter, it is analyzed whether the same parameter as the verification passing formula exists in the verification failing formula. If the same parameter does not exist, the original document solving failure formula is identified again, and it is judged whether the non-existing parameter is an identification error and the error is corrected. After correcting the incomplete parameter, it is judged whether the parameter that does not exist in the verification failing formula exists in the first verification formula, so as to judge whether the parameter is an identification error, which causes the formula to be unable to be solved. Thus, when the same parameter is identified for the second time after generating a new formula, the parameter identification error is reduced.

[0057] In the embodiment S6, the format of each formula is adjusted by the label and segmentation. Among them, when multiple formulas lack labels, the labels of each formula are adjusted, or when the label is wrong, the label is adjusted. When the segmentation method in the original document is wrong, such as the arrangement between the formula and the text causing the formula to be segmented, or the formula is disconnected by the space, the formula is adjusted.

[0058] Embodiment Two

[0059] The embodiment discloses a mathematical formula recognition method and system in a document, comprising:

[0060] An acquisition module: acquiring a plurality of formulas to be recognized in an original document and verifying them respectively to obtain a plurality of verified formulas and a plurality of unverified formulas;

[0061] A first parameter judgment module: analyzing whether there is a same parameter in the unverified formulas as in the verified formulas;

[0062] A first solving module: if there is, then the verified formula with the same parameter is brought into the current unverified formula for solving, and if the solving is successful, the current unverified formula is taken as a verified formula;

[0063] A second parameter judgment module: if the solving is not successful, then it is judged whether there is a same parameter in the current unverified formula as in other unverified formulas, if there is a same parameter, then the current unverified formula is brought into the other unverified formula with the same parameter to obtain a new formula;

[0064] A second solving module: the new formula is brought into the other unverified formula with the same parameter for solving, if the solving is successful, the new formula is taken as a verified formula, until all formulas are verified successfully, and all verified formulas are generated into a new document.

[0065] Embodiment Three

[0066] The purpose of the embodiment is to provide a computing device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above method.

[0067] Embodiment Four

[0068] The purpose of the embodiment is to provide a computer-readable storage medium.

[0069] A computer-readable storage medium, having a computer program stored thereon, wherein the program is executed by a processor to implement the steps of the above method.

[0070] The steps and methods involved in the above embodiments two, three and four correspond to embodiment one, and the specific embodiments can be referred to the related description part of embodiment one. The term "computer-readable storage medium" should be understood as including a single medium or multiple media of one or more instruction sets; it should also be understood as including any medium capable of storing, encoding or carrying instruction sets for execution by a processor and causing the processor to perform any method in the present application.

[0071] Those skilled in the art should understand that the modules or steps of the present application described above can be realized by a general computer device, or alternatively, they can be realized by program codes executable by a computing device, so that they can be stored in a storage device and executed by a computing device, or they can be respectively manufactured into individual integrated circuit modules, or a plurality of modules or steps among them can be manufactured into a single integrated circuit module. The present application is not limited to any specific combination of hardware and software.

[0072] The specific embodiments of the present application described above in conjunction with the accompanying drawings are not intended to limit the protection scope of the present application. Those skilled in the art should understand that various modifications or changes made by those skilled in the art on the basis of the technical solutions of the present application without creative labor are still within the protection scope of the present application.

Claims

1. A method for recognizing mathematical formulas in documents, characterized in that, include: Obtain multiple formulas to be identified from the original document and verify them individually to obtain multiple formulas that pass verification and multiple formulas that fail verification; Analyze whether the formula for failing the verification contains the same parameters as the formula for passing the verification. If it exists, the verification pass formula with the same parameters will be substituted into the current verification fail formula for solving. If the solution is successful, the current verification fail formula will be used as the verification pass formula. If the solution fails, check if there are any identical parameters in the current failed formulas. If there are identical parameters, substitute the parameters from other failed formulas into the current failed formula to obtain a new formula. The new formula is substituted into other formulas that fail verification with the same parameters for solution. If the solution is successful, the new formula is used as the verification successful formula. This process continues until all formulas are successfully verified, and then a new document is generated for all successfully verified formulas.

2. The method for recognizing mathematical formulas in documents as described in claim 1, characterized in that, The system retrieves multiple formulas from the original document and verifies them individually. By determining whether there are any fundamental errors, it obtains multiple formulas that pass verification and multiple formulas that fail verification.

3. The method for recognizing mathematical formulas in documents as described in claim 1, characterized in that, If the formula that fails the verification does not contain the same parameter as the formula that passes the verification, then the adjacent fields in the current formula that fails the verification are identified to determine whether the formula has complete parameters. If it is incomplete, it is corrected.

4. The method for recognizing mathematical formulas in documents as described in claim 3, characterized in that, When incomplete, the correction is performed as follows: based on the first one or more adjacent fields before the starting character of the parameter in the formula that failed the validation, and the last one or more adjacent fields after the ending character of the parameter, the incomplete characters are supplemented using the document recognition function.

5. The method for recognizing mathematical formulas in documents as described in claim 1, characterized in that, The new formula is substituted into other failed verification formulas with the same parameters for solution. If the solution fails, it is determined whether the current failed verification formula does not have the same parameters as other passed verification formulas and failed verification formulas. If so, the non-existent parameter is corrected and replaced with a parameter that exists in other passed verification formulas, that is, a parameter that exists in all of the passed verification formulas.

6. The method for recognizing mathematical formulas in documents as described in claim 5, characterized in that, The missing parameter is corrected by replacing it with a parameter that exists in all valid formulas.

7. The method for recognizing mathematical formulas in documents as described in claim 1, characterized in that, The parameters refer to the multiple values ​​and results involved in the calculation in the formula.

8. A method system for recognizing mathematical formulas in documents, characterized in that, include: Acquisition module: Acquires multiple formulas to be identified in the original document and verifies them separately, resulting in multiple formulas that pass verification and multiple formulas that fail verification; First parameter judgment module: Analyzes whether there are any parameters in the verification failure formula that are the same as those in the verification success formula; First solution module: If it exists, the verification pass formula with the same parameters will be substituted into the current verification fail formula for solution. If the solution is successful, the current verification fail formula will be used as the verification pass formula. The second parameter judgment module: If the solution fails, it will determine whether there is a parameter that is the same as that in other formulas that fail verification. If there is a parameter that is the same, it will substitute the other formulas that fail verification with the same parameter into the current formula to obtain a new formula. The second solution module: Substitutes the new formula into other formulas that have failed verification with the same parameters for the solution. If the solution is successful, the new formula is used as the verification formula. This process continues until all formulas are successfully verified, and then a new document is generated for all successfully verified formulas.

9. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of a method for recognizing mathematical formulas in a document as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of a method for recognizing mathematical formulas in a document as described in any one of claims 1 to 7.

Images