Mathematics problem correction method, device, equipment and storage medium

Abstract

The embodiment of the present application discloses a method, device, equipment and storage medium for correcting math problems, which obtains text content recognition results of each answer area of ​​an image of a math problem; the text content recognition results represent the elements contained in the corresponding answer area, and the structural relationship between the elements; for each answer area, if the math problem is a target problem type, the sub-topic of the answer area is converted into a sub-topic of a non-target problem type according to the text content recognition result of the answer area; and the correction result of the answer area is obtained based on the sub-topic of the non-target problem type of the answer area. Automatic correction of math speed calculation problems of various types is achieved, and the intelligence of intelligent review is improved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and more specifically, to a method, device, equipment and storage medium for correcting mathematics problems. Background Art

[0002] Mathematical fast calculation is the foundation of mathematical ability and is also a common test item in primary and secondary school mathematics today. In order to improve the efficiency of reviewers, intelligent review solutions have emerged. For mathematical fast calculation questions, the current intelligent review solutions can only realize the automatic review of simple questions (such as four arithmetic operations, estimation, etc.), but cannot realize the automatic review of more complex questions (such as vertical form, escape form, equation, etc.), and the intelligence is relatively low. Summary of the invention

[0003] In view of this, the present application provides a method, device, equipment and storage medium for correcting mathematics problems to improve the intelligence of intelligent marking.

[0004] In order to achieve the above objectives, the proposed solution is as follows:

[0005] A method for correcting mathematics problems, comprising:

[0006] Obtaining text content recognition results for each answer area of ​​the image of the math problem; each text content recognition result represents each element contained in the corresponding answer area, and the structural relationship between each element;

[0007] For each answer area, if the math question type is a target question type, convert the sub-topic of the answer area into a sub-topic of a non-target question type according to the text content recognition result of the answer area;

[0008] A correction result of the answer area is obtained based on the sub-questions of the non-target question type in the answer area.

[0009] In the above method, preferably, the smallest text unit in the text content recognition result includes at least part of the following units: a single letter, a single Chinese character, a single symbol, and a single unit in the Latteh formula;

[0010] Each of the elements corresponds to at least one minimum text unit.

[0011] In the above method, preferably, the step of converting the sub-topic of the answer area into a sub-topic of a non-target question type according to the text content recognition result of the answer area includes:

[0012] Replacing each text unit in the text content recognition result of the answer area with a preset character to obtain an abstract expression of the answer area;

[0013] Extracting calculation objects and result objects from the abstract expression based on extraction rules corresponding to the target question type;

[0014] Construct a sub-question of a non-target question type in the answer area based on the calculation object and the result object.

[0015] The above method preferably further comprises:

[0016] If the math question type is not the target question type, generating an extended recognition result based on the text content recognition result of the answer area, wherein the extended recognition result is obtained by deleting characters at a target position from the text content recognition result of the answer area;

[0017] The correction result of the answer area is obtained based on the text content recognition result of the answer area and each extended recognition result.

[0018] In the above method, preferably, the target position is determined based on a target criterion, and the target criterion includes:

[0019] On the left side of the first operator;

[0020] On the left side of the first fraction;

[0021] On the left side of the first Chinese character;

[0022] To the left of the second set of consecutive numeric characters.

[0023] In the above method, preferably, obtaining the correction result of the answer area based on the text content recognition result and the extended recognition result of the answer area includes:

[0024] Dividing the target recognition result into a plurality of scopes according to the operator in the target recognition result, wherein there is an operator between two adjacent scopes; the target recognition result is any text content recognition result or extended recognition result;

[0025] Converting the content representing the unconventional characters in the target recognition result after the scope is divided into the content representing the regular characters;

[0026] Mapping the characters representing the units in the target recognition result after character conversion into corresponding numbers;

[0027] Obtaining a legal expression of the target recognition result after unit mapping, and obtaining a correction result of the answer area based on the legal expression;

[0028] If at least one correction result corresponding to the answer area is correct, it is determined that the correction result of the answer area is correct.

[0029] In the above method, preferably, the step of obtaining a legal expression of the target recognition result after unit mapping includes:

[0030] Dividing the target recognition result after unit mapping into a plurality of parts according to the equal sign or approximate sign in the target recognition result after unit mapping, with the equal sign or approximate sign between two adjacent parts;

[0031] For each part, if the part cannot be calculated, with the goal of making the part calculateable, delete the symbols except numbers in the part until the part can be calculated and a legal expression is obtained.

[0032] The above method, preferably, further comprises:

[0033] Obtaining the type of the math problem;

[0034] The type of the math problem is read from a preset storage area, or is obtained based on the stem of the math problem.

[0035] In the above method, preferably, the process of obtaining the type of the math problem based on the stem of the math problem includes:

[0036] Performing printing area recognition on the image of the math problem to determine the printing area;

[0037] Extracting the text content of the question stem in the printed area;

[0038] Extract keywords from the question text content;

[0039] The type of the math problem is determined according to the extracted keywords.

[0040] A mathematical problem correction device, comprising:

[0041] An acquisition module, used to obtain text content recognition results of each answer area of ​​the image of the math problem; the text content recognition results represent each element contained in the corresponding answer area, and the structural relationship between each element;

[0042] A conversion module, for each answer area, if the math question type is a target question type, converting a sub-topic of the answer area into a sub-topic of a non-target question type according to a text content recognition result of the answer area;

[0043] The correction module is used to obtain the correction result of the answer area based on the sub-questions of the non-target question type in the answer area.

[0044] A mathematics problem grading device, comprising a memory and a processor;

[0045] The memory is used to store programs;

[0046] The processor is used to execute the program to implement each step of the math problem correction method as described in any one of the above items.

[0047] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements each step of the method for correcting mathematics problems as described in any one of the above items.

[0048] It can be seen from the above technical solutions that the math problem grading method, device, equipment and storage medium provided in the embodiments of the present application obtain the text content recognition results of each answer area of ​​the image of the math problem; the text content recognition results represent the elements contained in the corresponding answer area, and the structural relationship between the elements; for each answer area, if the math problem is a target problem type, the sub-topic of the answer area is converted into a sub-topic of a non-target problem type according to the text content recognition results of the answer area; the grading result of the answer area is obtained based on the sub-topic of the non-target problem type of the answer area. When the math problem belongs to the target problem type, the present application converts the sub-topic of each answer area into a sub-topic of a non-target problem type based on the text content recognition results of each answer area of ​​the math problem that represent the elements contained in the answer area and the structural relationship between the elements, and then grades the answer area according to the sub-topic of the non-target problem type of the answer area, thereby realizing the automatic grading of math speed calculation problems of various types and improving the intelligence of intelligent grading. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For those skilled in the art, other drawings can be obtained based on the provided drawings without paying any creative work.

[0050] Figure 1 A flowchart of a method for correcting math problems disclosed in an embodiment of the present application;

[0051] Figure 2 An example diagram of a math speed calculation problem that has not yet been answered disclosed in an embodiment of the present application;

[0052] Figure 3 A flowchart for realizing the method of converting a sub-topic of the ith answer area into a sub-topic of a non-target question type according to a text content recognition result of the ith answer area disclosed in an embodiment of the present application;

[0053] Figure 4This is an example diagram of an answering area disclosed in an embodiment of the present application;

[0054] Figure 5 Two example diagrams of math problems with problem numbers disclosed in the embodiments of the present application;

[0055] Figure 6 A flowchart for realizing obtaining the correction result of the answer area based on the text content recognition result of the i-th answer area and each extended recognition result disclosed in the embodiment of the present application;

[0056] Figure 7 A schematic diagram of the structure of a math problem correction device disclosed in an embodiment of the present application;

[0057] Figure 8 This is a hardware structure block diagram of the math problem grading device disclosed in the embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0059] In order to improve the intelligence of intelligent review, embodiments of the present application are proposed.

[0060] like Figure 1 As shown, a flowchart of an implementation of the method for correcting math problems provided in an embodiment of the present application may include:

[0061] Step S101: obtaining text content recognition results of each answer area of ​​the image of the math problem; each text content recognition result represents the elements contained in the content of the corresponding answer area, and the structural relationship between the elements.

[0062] The content in the answer area may contain elements including letters, Chinese characters, symbols, numbers (including integers, decimals and fractions, etc.), etc.

[0063] The math problems here refer to the math problems to be corrected, that is, math problems with answer content.

[0064] In actual scenarios, a math problem may have multiple small stems (each small stem is recorded as a sub-stem), and the problem solver answers in the area where each sub-stem is located. Each sub-stem and the corresponding answer content is an answer area. Figure 2As shown, it is an example diagram of a math speed calculation problem that has not been answered (i.e., the answer content has not been given) provided in an embodiment of the present application. In this example, the type of problem is an equation. The problem has three sub-stems, i.e., three small stems. When answering each sub-stem, the solver usually writes the answer content below the sub-stem. Therefore, each sub-stem and the answer content below it constitute a sub-topic, and the area where the sub-topic is located is the answer area.

[0065] As an example, the image of the math problem to be corrected can be processed by the region detection model to obtain each answer region of the image of the math problem and each text line region of each answer region.

[0066] For each answer area (for the convenience of description and distinction, it is recorded as the i-th answer area, i=1, 2,..., N, N is the number of answer areas included in the math problem), when performing text content recognition on the i-th answer area, each text line area of ​​the i-th answer area can be recognized separately to obtain the text recognition result of the i-th answer area.

[0067] As an example, the text recognition model may be used to perform text recognition on text lines belonging to the same answer area respectively to obtain text content recognition results for each answer area.

[0068] As an example, the text recognition model can be an encoder-decoder network, where the encoder side can use VGG or ResNet + bidirectional LSTM or bidirectional GRU to encode and extract text formula features, and the decoder side uses attention and bidirectional LSTM or bidirectional GRU for decoding. The modeling unit of the text recognition model is a single word, a single Chinese character, a single symbol, a single unit of the LaTeX formula, etc. For example, The latex formula is \frac{1}{2}, so it contains 5 single units of latex formulas, namely: "\frac", "{", "}", "1", "2", among which "\frac", "{", "}" are virtual, and "1" and "2" are real.

[0069] Based on this, the minimum text unit in the text content recognition result includes at least part of the following units: a single letter, a single Chinese character, a single symbol, and a single unit in the Latteh formula. Each element in the answer area corresponds to at least one minimum text unit. Different elements correspond to different minimum text units.

[0070] Step S102: For each answer area (denoted as the i-th answer area), if the math question type is the target question type, the sub-topic of the i-th answer area is converted into a sub-topic of a non-target question type according to the text content recognition result of the i-th answer area.

[0071] The type of the math problem can be input into the math problem grading device by the grading user. Or,

[0072] The type of math problem can be obtained by processing the image of the math problem to be corrected through the problem type recognition model. Or,

[0073] The type of the math problem can be obtained by the math problem correction device based on the text content of the math problem stem. As an example, the image of the math problem can be identified by a text recognition model to identify the printing area, and the printed area can be identified to obtain the text content of the problem stem.

[0074] Keyword extraction is performed on the text content of the question stem, and the question type of the math question is determined according to the extracted keywords. As an example, the question type corresponding to the extracted keyword can be determined as the question type of the math question to be corrected according to the preset correspondence between the keyword and the question type.

[0075] As an example, the image of the math problem to be corrected can be processed by the separation model to determine the mask corresponding to the handwritten area and the mask corresponding to the printed area; the handwritten area image can be obtained according to the mask corresponding to the handwritten area, and the printed area image can be obtained according to the mask corresponding to the printed area. Specifically, the mask corresponding to the handwritten area can be multiplied with the image of the math problem to be corrected to obtain the handwritten area image, and the mask corresponding to the printed area can be multiplied with the image of the math problem to be corrected to obtain the printed area image.

[0076] The first region detection model can be used to perform region detection on the image of the math problem to be corrected to determine each answer region in the image.

[0077] The second region detection model performs region detection on the handwriting region image and the printing region image respectively to determine each handwritten text line in the handwriting region image and each printed text line in the printing region image.

[0078] The text recognition model can be used to identify printed text lines that do not belong to the answer area (for example, Figure 2 The text line "2. Find the unknown number x, (2 points for each question, 6 points in total)" in the above text is used for text recognition to obtain a text content recognition result. The text content recognition result only represents the main theme of the math problem to be corrected (i.e., "find the unknown number x"), and the text content recognition result is used as the text content of the question.

[0079] As an example, all printed text lines (e.g., Figure 2 The text line "2. Find the unknown number x, (2 points for each question, 6 points in total)" in the text, as well as the three text lines where the three small questions are located, are respectively recognized to obtain text content recognition results, which represent the main topic and sub-topics (i.e., "6x-2.5=9.5", "3.5x+x=16.2", ), and use the text recognition result as the text content of the question.

[0080] As an example, the target question type may be any of the following: vertical form, equation, escape formula, etc.; the non-target question type may include four arithmetic operations, and the non-target question type may further include operations such as estimation.

[0081] Step S103: obtaining the correction result of the i-th answer area based on the sub-questions of the non-target question type in the i-th answer area.

[0082] As an example, it can be determined whether the sub-question of the non-target question type in the i-th answer area is correct. If correct, the correction result of the i-th answer area is determined to be correct; otherwise, the correction result of the i-th answer area is determined to be incorrect.

[0083] The math problem grading method provided in the embodiment of the present application, when the math problem belongs to the target question type, converts the sub-topics of each answer area into sub-topics of a non-target question type based on the text content recognition results of each answer area of ​​the math problem that characterizes each element contained in the answer area and the structural relationship between each element, and then corrects the answer area according to the sub-topics of the non-target question type of the answer area, thereby realizing automatic grading of math speed calculation problems of various question types and improving the intelligence of intelligent grading.

[0084] In an optional embodiment, a flowchart for implementing the above-mentioned conversion of the sub-topic of the i-th answer area into a sub-topic of a non-target question type according to the text content recognition result of the i-th answer area is as follows: Figure 3 As shown, it may include:

[0085] Step S301: Replace each text unit in the text content recognition result of the i-th answer area with a preset character to obtain an abstract expression of the i-th answer area.

[0086] Each text unit in the text content recognition result of the i-th answer area can be replaced with a predefined character to obtain an abstract expression corresponding to the i-th answer area.

[0087] Step S302: Based on the extraction rules corresponding to the target question type, the calculation objects and result objects are extracted from the abstract expression.

[0088] Based on different target question types, the calculation objects extracted from the abstract expression are different. For example, for vertical formulas, the calculation objects extracted are the two numbers involved in the calculation. For paraphrases or equations, the calculation objects extracted can be only the first line of content, or the calculation objects extracted can be every line except the last line.

[0089] The result object is the answer result. In some cases, the result object will appear at the end of the text content recognition result, such as the vertical form, equation, addition vertical form, multiplication vertical form and subtraction vertical form. In some cases, the result object will appear at the beginning of the text content recognition result. For example, in the vertical form of division, when the result only needs the quotient, since the quotient is in the first line of the answer area, the result object will appear at the beginning of the text content recognition result. In other cases, the result object will appear at the beginning and end of the text content recognition result. For example, in the vertical form of division, when the answer result contains the quotient and remainder, when the text content recognition is performed on it, the quotient will appear at the beginning of the text content recognition, and the remainder will appear at the end of the text content recognition result.

[0090] Step S303: construct a sub-topic of the non-target question type in the answer area based on the calculation object and the result object. The sub-topic of the non-target question type is an equation based on four arithmetic operations, which is recorded as a four arithmetic operation equation.

[0091] As an example, for vertical calculation, the calculation formula on one side of the equal sign of the four arithmetic equation (recorded as the first four arithmetic equation) can be constructed based on the two numbers involved in the vertical calculation, and the other side of the equal sign of the first four arithmetic equation is the result object. Alternatively, the calculation formula on one side of the equal sign of the four arithmetic equation (recorded as the second four arithmetic equation) can be constructed with one of the two numbers involved in the vertical calculation and the result object, and the other side of the equal sign of the second four arithmetic equation is the other number of the two numbers involved in the vertical calculation; wherein, the first four arithmetic equation and the second four arithmetic equation can be converted to each other, that is, the second four arithmetic equation can be obtained from the first four arithmetic equation, and the first four arithmetic equation can also be obtained from the second four arithmetic equation.

[0092] As an example, corresponding to the deletion, the content of each line can be converted into an arithmetic calculation formula on one side of the equal sign of the arithmetic equation, and the other side of the equal sign of the arithmetic equation is the result object.

[0093] As an example, for an equation, the unknown number in the content (equation) of each row can be replaced with a result object, and the equation after the unknown number is replaced can be converted into an arithmetic equation.

[0094] Specifically, if there are characters that do not belong to the four arithmetic operations (recorded as unconventional characters) in the calculation formula, the unconventional characters can be converted into regular characters (that is, characters that belong to the four arithmetic operations). The specific conversion process can be found in the subsequent embodiments.

[0095] like Figure 4 As shown, it is an example diagram of an answer area provided in an embodiment of the present application. In this example, the question type is vertical division, and the text content recognition result obtained by performing text recognition on the answer area is:

[0096] 5\\\hline 48)252\\240\\\hline 12

[0097] Among them, "\\" represents a line break, and "\hline" represents a horizontal line.

[0098] In this application, "\hline" is defined as h, "\\" is defined as l, and numbers are defined as N. Numbers in different positions are distinguished by subscripts. Based on this, the above text content recognition result can be simplified into the following abstract expression:

[0099] N 1 n 2 )N 3 oeLh 4 n 5

[0100] Based on the question type, the calculation objects extracted from the abstract expression can be determined to include: N 2 、N 3 , the result object includes N 1 、N 5 .

[0101] The four arithmetic equations constructed based on the calculation object and the result object are:

[0102] N 3 ÷N 2 =N 1 ......N 5

[0103] That is, 252÷48=5......12

[0104] Based on this, we can determine whether the quotient of 252 divided by 48 is equal to 5 and whether the remainder is 12. Figure 4 Specifically, if the quotient of 252 divided by 48 is 5 and the remainder is 12, then determine Figure 4 The correct answer in the answer area is correct. If the quotient of 252 divided by 48 is not 5, and / or the remainder is not 12, then confirm Figure 4The answer area shown has been graded as incorrect.

[0105] Or, the four arithmetic equations constructed based on the calculation object and calculation result are:

[0106] N 1 ×N 2 +N 5 =N 3

[0107] That is, 5×48+12=252

[0108] Based on this, we can determine whether the product of 5 and 48 plus 12 equals 252. Figure 4 Specifically, if the product of 5 and 48 plus 12 equals 252, then Figure 4 The correct answer in the answer area is correct. If the product of 5 and 48 plus 12 is not equal to 252, then confirm Figure 4 The answer area shown has been graded as incorrect.

[0109] In an optional embodiment, if the math question is not a target question type, for example, the math question is an estimation question, or it can be a simple arithmetic operation. In some cases, there may be a question number in front of the question, such as Figure 5 As shown, there are two example diagrams of math problems with problem numbers provided in an embodiment of the present application. Figure 5 -The question number in a is ②, Figure 5 -b is 4. When performing text recognition on these math problems with problem numbers, the circle (or triangle or square, etc.) outside the problem number may be missed due to the problem number being too small or the color being too light. Figure 5 -a, the recognition result may be There may also be situations where the true meaning of some symbols cannot be distinguished due to detection reasons, such as Figure 5 -b, the "." to the right of 4 may not be determined as a question number separator or a decimal point, and may be recognized as 4.27. In order to overcome this type of problem, this application proposes the following solution:

[0110] If the math question type is not the target question type, an extended recognition result is generated based on the text content recognition result of the i-th answer area, and the extended recognition result is obtained by deleting characters at the target position of the text content recognition result of the i-th answer area.

[0111] The purpose of character deletion is to remove the interference of the question number. Therefore, the target position is usually located at the left part of the text content recognition result that meets the conditions.

[0112] When there are multiple consecutive characters at the target position, different numbers of characters may be deleted at the target position to obtain multiple extended recognition results.

[0113] The correction result of the i-th answer area is obtained based on the text content recognition result of the i-th answer area and each extended recognition result.

[0114] Optionally, if the correction result of the i-th answer area is determined to be correct based on the text content recognition result of the i-th answer area, or if the correction result of the i-th answer area is determined to be correct based on any extended recognition result, then the correction result of the i-th answer area is determined to be correct.

[0115] If the correction result of the answer area is determined to be wrong based on the text content recognition result of the i-th answer area, and the correction results of the i-th answer area are all determined to be wrong based on each extended recognition result, the text content recognition result of the i-th answer area is determined to be wrong.

[0116] By extending the recognition results and automatically correcting math problems in combination with the extended recognition results, the accuracy of correction is improved.

[0117] In an optional embodiment, the target location may be determined based on target criteria, which may include:

[0118] On the left side of the first operator;

[0119] On the left side of the first fraction;

[0120] On the left side of the first Chinese character;

[0121] To the left of the second set of consecutive numeric characters.

[0122] The above four criteria must be met at the same time, that is, when there is at least one operator in the text content recognition result, the target position must be on the left side of the first operator; when there is at least one operator and at least one fraction in the text content recognition result, the target position must be on the left side of the first operator and must be on the left side of the first fraction; when there is at least one operator, at least one fraction, and at least one Chinese character in the text content recognition result, the target position must be on the left side of the first operator and must be on the left side of the first fraction and on the left side of the first Chinese character; when there is at least one operator, at least one fraction, at least one Chinese character, and at least one group of consecutive numeric characters in the text content recognition result, the target position must be on the left side of the first operator, on the left side of the first fraction, on the left side of the first Chinese character, and on the left side of the second group of consecutive numeric characters.

[0123] For example, suppose the text content recognition result is:

[0124] 1.34\times 0.5 2\approx 0.1 8

[0125] The text content recognition result contains operators and consecutive numeric characters, but no fractions or Chinese characters. The first operator is \times, the first group of consecutive numeric characters is 1, and the second group of consecutive numeric characters is 34. Based on "to the left of the first operator" and "to the left of the second group of consecutive numeric characters", it can be determined that the target position is to the left of 34. Based on this, the extended recognition results obtained by expansion include:

[0126] .34\times 0.5 2\approx 0.1 8

[0127] 34\times 0.5 2\approx 0.1 8

[0128] For another example, suppose the text content recognition result is:

[0129] \textcircled{3}\frac{5}{2}\div 2\approx 0.1 8

[0130] The text content recognition result contains operators, fractions, and consecutive numeric characters, but no Chinese characters. The first operator is \div, the fraction is \frac{5}{2}, the first group of consecutive numeric characters is 3, and the second group of consecutive numeric characters is 5. Based on "on the left side of the first operator", "on the left side of the first fraction", and "on the left side of the second group of consecutive numeric characters", it can be determined that the target position is the left side of \frac. Based on this, the extended recognition results obtained by expansion include:

[0131] \frac{5}{2}\div 2\approx 0.1 8

[0132] For another example, suppose the text content recognition result is:

[0133] \textcircled{3 2}4\UC_75ED\div 0.5 2\approx 0.1 8

[0134] The text content recognition result contains operators, Chinese characters and continuous numeric characters, but no fractions. The first operator is \div, the Chinese character is UC_75ED, the first group of continuous numeric characters is 32, and the second group of continuous numeric characters is 4. Based on "on the left side of the first operator", "on the left side of the first Chinese character" and "on the left side of the second group of continuous numeric characters", it can be determined that the target position is on the left side of 4. Based on this, the extended recognition results obtained by expansion include:

[0135] 4\UC_75ED\div 0.5 2\approx 0.1 8

[0136] For the extended recognition results, there may be cases where illegal formulas are represented. For example, “.34\times0.5 2\approx 0.1 8” represents the formula: .34×0.52≈0.18. Obviously, this is not a legal formula. Therefore, the extended recognition result “.34\times 0.5 2\approx 0.1 8” is deleted.

[0137] Optionally, a flowchart for implementing the above-mentioned obtaining the correction result of the answer area based on the text content recognition result of the i-th answer area and each extended recognition result is as follows: Figure 6 As shown, this may include:

[0138] Step S601: Divide the target recognition result into multiple scopes according to the operator in the target recognition result, wherein there is an operator between two adjacent scopes; the target recognition result is any text content recognition result or any extended recognition result.

[0139] In some cases, when correcting text content, there may be a process of unit conversion. For example, the target recognition result is: 2 km 500 m - 1 km 100 m = 1400 m. To correct it, it is necessary to convert the units, such as converting kilometers to meters, etc. In some cases, unit conversion may cause semantic changes. In order to avoid semantic changes when unit conversion is carried out, this application first divides the scope of the target recognition results.

[0140] That is to say, for the text content recognition result of the ith answer area and any extended recognition result of the ith answer area, scope division is performed. When performing scope division, the operator is used as the dividing line to divide it into multiple scopes, and each scope is enclosed in brackets. For example, the result of scope division for the target recognition result "2 kilometers 500 meters - 1 kilometer 100 meters = 1400 meters" is: "(2 kilometers 500 meters) - (1 kilometer 100 meters) = (1400 meters)".

[0141] Step S602: converting the content representing the non-conventional characters in the target recognition result after the scope is divided into the content representing the regular characters, so as to convert the operations in the target recognition result into four arithmetic operations.

[0142] The regular characters mentioned here refer to characters related to arithmetic operations, such as +, -, ×, ÷, as well as =, ≈, etc.

[0143] In some cases, there may be unconventional characters in the text recognition results, such as:%,^,{,:,\frac, etc. As shown in Table 1, an example of converting some unconventional characters into regular characters provided in the embodiment of the present application is as follows:

[0144] Table 1

[0145] Unconventional characters Before converting to regular characters After converting to regular characters ％ 2\times 20%=0.4 2\times(20\times 0.01)=0.4 ^ 2+5^3＝0.4 2+(5\times 5\times5)=0.4 { \boxed{3}34\div 5 \boxed(3)34\div 5 \frac \frac{\boxed{3}}{64} ({\boxed{3}}\div{64}) : 3:2＝6:4 3\div 2=6\div 4 \boxed \boxed{3}34\div 5 {3}34\div 5

[0146] Each row in Table 1 only describes how to convert a non-conventional character into a regular character. In practical applications, it is necessary to convert all non-conventional characters in a calculation formula into regular characters. For example, "\boxed{3}34\div 5" includes two non-conventional characters, namely "\boxed" and "{". The result after converting these two non-conventional characters into regular characters is: (3)34\div 5.

[0147] Step S603: Mapping the characters representing the units in the target recognition result after character conversion (ie, the target recognition result obtained in step S602) into corresponding numbers.

[0148] As shown in the example above, the result after unit mapping of "(2 kilometers 500 meters) - (1 kilometer 100 meters) = (1400 meters)" is: (2*1000+500*1) - (1*1000+100*1) = (1400*1).

[0149] Step S604: Obtain a legal expression of the target recognition result after unit mapping (that is, the target recognition result obtained in step S603), and obtain a correction result for the i-th answer area based on the legal expression.

[0150] In addition to the problems of multiple recognition and missed recognition of question numbers, the text recognition model may also have the problem of multiple recognition of operation symbols, which will lead to the appearance of illegal expressions. For example, the text content recognition result is "23+23=23)*2≈60". Due to the recognition problem of the model, the symbol ")" is added, making it an illegal expression, which needs to be converted into a legal expression.

[0151] As an example, the target recognition result after unit mapping may be divided into a plurality of parts according to an equal sign or an approximate sign in the target recognition result after unit mapping, and an equal sign or an approximate sign is between two adjacent parts.

[0152] As shown in the above example, according to the equal sign and the approximately equal sign, the target recognition result is divided into three parts, namely: "23+23", "23)*2", and "60".

[0153] For each part, if the part cannot be calculated, with the goal of making the part calculateable, delete the symbols except numbers in the part until the part can be calculated and a legal expression is obtained.

[0154] Continuing with the above example, both "23+23" and "60" can be calculated, but "23)*2" cannot be calculated. Then, the symbols other than numbers in "23)*2" (i.e. ")" and "*") are deleted. Obviously, if "*" is deleted, the resulting "23)2" still cannot be calculated, while the "23*2" obtained after deleting ")" can be calculated. Therefore, delete ")" in "23+23=23)*2≈60" and the legal expression is "23+23=23*2≈60".

[0155] The method of obtaining a correction result for the i-th answer area based on the legal expression is: for any legal expression, if the judgment results of 23+23=23*2 and 23*2≈60 are both correct, the correction result of the i-th answer area is determined to be correct; otherwise, the correction result of the i-th answer area is determined to be wrong.

[0156] Step S605: If at least one correction result corresponding to the i-th answer area is correct, determine that the correction result of the i-th answer area is correct.

[0157] In the case where multiple legal expressions are obtained in the i-th answer area, as long as the correction result of the i-th answer area is determined to be correct based on one legal expression, the correction result of the i-th answer area is determined to be correct. If the correction result of the i-th answer area is determined to be wrong based on each legal expression, the correction result of the i-th answer area is determined to be wrong.

[0158] In addition, since the math questions of the target question type usually do not have question numbers, after obtaining the sub-questions of the non-target question type in the i-th answer area, it is not necessary to expand the recognition results of the sub-questions of the non-target question type and try to process legal expressions.

[0159] As an example, if the math questions of the target question type are also set with question numbers, after obtaining the sub-questions of the non-target question type in the i-th answer area, the sub-questions of the non-target question type can be expanded to obtain the expanded recognition results, character conversion is performed on each expanded recognition result (i.e., unconventional characters are converted into regular characters), and the legal expressions of each expanded recognition result are obtained, and then the correction results of the math questions of the target question type are obtained based on the sub-questions of the non-target question type in the i-th answer area and the various legal expressions obtained by expansion. The specific implementation process can refer to the aforementioned embodiment of processing the questions of the non-target question type, which will not be repeated here.

[0160] Corresponding to the method embodiment, the embodiment of the present application also provides a mathematical problem correction device. A structural schematic diagram of the mathematical problem correction device provided by the embodiment of the present application is shown in FIG. Figure 7 As shown, this may include:

[0161] Acquisition module 701, conversion module 702 and correction module 703; wherein,

[0162] The acquisition module 701 is used to obtain the text content recognition results of each answer area of ​​the image of the math problem; the text content recognition results represent the elements contained in the corresponding answer area and the structural relationship between the elements;

[0163] The conversion module 702 is used for, for each answer area, if the question type of the math question is a target question type, converting the sub-topic of the answer area into a sub-topic of a non-target question type according to the text content recognition result of the answer area;

[0164] The correction module 703 is used to obtain the correction result of the answer area based on the sub-questions of the non-target question type in the answer area.

[0165] The math problem grading device provided in the embodiment of the present application, when the math problem belongs to the target question type, converts the sub-topics of each answer area into sub-topics of a non-target question type based on the text content recognition results of the elements contained in the answer area corresponding to the representation of each answer area of ​​the math problem and the structural relationship between the elements, and then corrects the answer area according to the sub-topics of the non-target question type of the answer area, thereby realizing automatic grading of math speed calculation problems of various types and improving the intelligence of intelligent grading.

[0166] In an optional embodiment, the minimum text unit in the text content recognition result includes at least part of the following units: a single letter, a single Chinese character, a single symbol, and a single unit in the Latteh formula;

[0167] Each of the elements corresponds to at least one minimum text unit.

[0168] In an optional embodiment, the conversion module 702 includes:

[0169] A replacement module, used to replace each text unit in the text content recognition result of the answer area with a preset character to obtain an abstract expression of the answer area;

[0170] An extraction module, used for extracting calculation objects and result objects from the abstract expression based on extraction rules corresponding to the target question type;

[0171] A construction module is used to construct a sub-question of a non-target question type in the answer area based on the calculation object and the result object.

[0172] In an optional embodiment, the mathematics problem correction device may further include:

[0173] An expansion module, for generating an extended recognition result based on the text content recognition result of the answer area if the question type of the math question is not the target question type, wherein the extended recognition result is obtained by deleting characters at a target position from the text content recognition result of the answer area;

[0174] The correction module 703 is further used to obtain a correction result of the answer area based on the text content recognition result of the answer area and each extension recognition result.

[0175] In an optional embodiment, the target position is determined based on a target criterion, wherein the target criterion includes:

[0176] On the left side of the first operator;

[0177] On the left side of the first fraction;

[0178] On the left side of the first Chinese character;

[0179] To the left of the second set of consecutive numeric characters.

[0180] In an optional embodiment, when the correction module 703 obtains the correction result of the answer area based on the text content recognition result and the extended recognition result of the answer area, it is used to:

[0181] Dividing the target recognition result into a plurality of scopes according to the operator in the target recognition result, wherein there is an operator between two adjacent scopes; the target recognition result is any text content recognition result or extended recognition result;

[0182] Converting the content representing the unconventional characters in the target recognition result after the scope is divided into the content representing the regular characters;

[0183] Mapping the characters representing the units in the target recognition result after character conversion into corresponding numbers;

[0184] Obtaining a legal expression of the target recognition result after unit mapping, and obtaining a correction result of the answer area based on the legal expression;

[0185] If at least one correction result corresponding to the answer area is correct, it is determined that the correction result of the answer area is correct.

[0186] In an optional embodiment, when the correction module 703 obtains the legal expression of the target recognition result after unit mapping, it is used to:

[0187] Dividing the target recognition result after unit mapping into a plurality of parts according to the equal sign or approximate sign in the target recognition result after unit mapping, with the equal sign or approximate sign between two adjacent parts;

[0188] For each part, if the part cannot be calculated, with the goal of making the part calculateable, delete the symbols except numbers in the part until the part can be calculated and a legal expression is obtained.

[0189] In an optional embodiment, the mathematics problem correction device may further include:

[0190] A question type determination module, used for obtaining the question type of the math question;

[0191] The type of the math problem is read from a preset storage area, or is obtained based on the stem of the math problem.

[0192] In an optional embodiment, when the question type determination module obtains the question type of the math question, it is used to:

[0193] Performing printing area recognition on the image of the math problem to determine the printing area;

[0194] Extracting the text content of the question stem in the printed area;

[0195] Extract keywords from the question text content;

[0196] The type of the math problem is determined according to the extracted keywords.

[0197] The math problem grading apparatus provided in the embodiment of the present application can be applied to math problem grading equipment, such as PC terminals, cloud platforms, servers and server clusters. Figure 8 The hardware structure diagram of the mathematics problem correction device is shown in FIG. Figure 8 ,The hardware structure of the mathematics problem grading device may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0198] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0199] The processor 1 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0200] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0201] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0202] Obtaining text content recognition results for each answer area of ​​the image of the math problem; each text content recognition result represents each element contained in the corresponding answer area, and the structural relationship between each element;

[0203] For each answer area, if the math question type is a target question type, convert the sub-topic of the answer area into a sub-topic of a non-target question type according to the text content recognition result of the answer area;

[0204] A correction result of the answer area is obtained based on the sub-questions of the non-target question type in the answer area.

[0205] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0206] The embodiment of the present application further provides a storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0207] Obtaining text content recognition results for each answer area of ​​the image of the math problem; each text content recognition result represents each element contained in the corresponding answer area, and the structural relationship between each element;

[0208] For each answer area, if the math question type is a target question type, convert the sub-topic of the answer area into a sub-topic of a non-target question type according to the text content recognition result of the answer area;

[0209] A correction result of the answer area is obtained based on the sub-questions of the non-target question type in the answer area.

[0210] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0211] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0212] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0213] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0214] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0215] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0216] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0217] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0218] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for correcting mathematics problems, characterized in that: The method comprises: Obtaining text content recognition results for each answer area of ​​the image of the math problem; each text content recognition result represents each element contained in the corresponding answer area, and the structural relationship between each element; For each answer area, if the math question type is a target question type, convert the sub-question of the answer area into a sub-question of a non-target question type according to the text content recognition result of the answer area; the target question type is any one of the following: vertical form, equation, and escape form; the non-target question type includes four arithmetic operations; A correction result of the answer area is obtained based on the sub-questions of the non-target question type in the answer area.

2. The method according to claim 1, characterized in that The minimum text unit in the text content recognition result includes at least some of the following types of units: a single letter, a single Chinese character, a single symbol, and a single unit in the Lathe formula; Each of the elements corresponds to at least one minimum text unit.

3. The method according to claim 1, characterized in that The step of converting the sub-topic of the answer area into a sub-topic of a non-target question type according to the text content recognition result of the answer area includes: Replacing each text unit in the text content recognition result of the answer area with a preset character to obtain an abstract expression of the answer area; Extracting calculation objects and result objects from the abstract expression based on extraction rules corresponding to the target question type; Construct a sub-question of a non-target question type in the answer area based on the calculation object and the result object.

4. The method according to claim 1, characterized in that Also includes: If the math question type is not the target question type, generating an extended recognition result based on the text content recognition result of the answer area, wherein the extended recognition result is obtained by deleting characters at a target position from the text content recognition result of the answer area; The correction result of the answer area is obtained based on the text content recognition result of the answer area and each extended recognition result.

5. The method according to claim 4, characterized in that The target position is determined based on target criteria, the target criteria comprising: On the left side of the first operator; On the left side of the first fraction; On the left side of the first Chinese character; To the left of the second set of consecutive numeric characters.

6. The method according to claim 4, characterized in that The step of obtaining the correction result of the answer area based on the text content recognition result and the extended recognition result of the answer area includes: Dividing the target recognition result into a plurality of scopes according to the operator in the target recognition result, wherein there is an operator between two adjacent scopes; the target recognition result is any text content recognition result or extended recognition result; Converting the content representing the unconventional characters in the target recognition result after the scope is divided into the content representing the regular characters; Mapping the characters representing the units in the target recognition result after character conversion into corresponding numbers; Obtaining a legal expression of the target recognition result after unit mapping, and obtaining a correction result of the answer area based on the legal expression; If at least one correction result corresponding to the answer area is correct, it is determined that the correction result of the answer area is correct.

7. The method according to claim 6, characterized in that The legal expression for obtaining the target recognition result after unit mapping includes: Dividing the target recognition result after unit mapping into a plurality of parts according to the equal sign or approximate sign in the target recognition result after unit mapping, with the equal sign or approximate sign between two adjacent parts; For each part, if the part cannot be calculated, with the goal of making the part calculateable, delete the symbols except numbers in the part until the part can be calculated and a legal expression is obtained.

8. The method according to claim 1, characterized in that The method further comprises: Obtaining the type of the math problem; The type of the math problem is read from a preset storage area, or is obtained based on the stem of the math problem.

9. The method according to claim 8, characterized in that The process of obtaining the type of the math problem based on the stem of the math problem includes: Performing printing area recognition on the image of the math problem to determine the printing area; Extracting the text content of the question stem in the printed area; Extract keywords from the question text content; The type of the math problem is determined according to the extracted keywords.

10. A mathematical problem correction device, characterized in that: The device comprises: An acquisition module, used to obtain text content recognition results of each answer area of ​​the image of the math problem; the text content recognition results represent each element contained in the corresponding answer area, and the structural relationship between each element; A conversion module is used for, for each answer area, if the question type of the math question is a target question type, converting the sub-question of the answer area into a sub-question of a non-target question type according to the text content recognition result of the answer area; the target question type is any one of the following: vertical form, equation, and escape form; the non-target question type includes four arithmetic operations; The correction module is used to obtain the correction result of the answer area based on the sub-questions of the non-target question type in the answer area.

11. A mathematical problem grading device, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the method for correcting mathematics problems as described in any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the method for correcting mathematics problems as described in any one of claims 1 to 9 is implemented.

Images