Method, device, readable storage medium and electronic device for correcting calculation problems

Abstract

The present disclosure relates to a method, device, readable storage medium and electronic device for correcting calculation questions. The method comprises: performing text recognition on the answer image to be corrected, obtaining K text contents with the highest recognition probability as candidate text contents; respectively determining whether the formula represented by each candidate text content is established; determining the target text content from the K candidate text contents according to the determination result and the recognition probability corresponding to each candidate text content; and judging whether the answer content is right or wrong according to the determination result of whether the formula represented by the target text content is established. In this way, the accuracy of the target text content can be guaranteed, the correction accuracy can be improved, and any answer image can be corrected without relying on the question bank. In addition, according to the determination result of whether the formula represented by the target text content is established, the right or wrong judgment of the answer content can be realized, without distinguishing between the question stem and the answer content, and without distinguishing between handwritten and printed characters, thereby improving the correction efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular, to a method, device, readable storage medium and electronic device for correcting calculation problems. Background Art

[0002] Teachers and parents have always had an urgent need for automated homework grading. The use of artificial intelligence technology, especially text recognition technology, to achieve automated homework grading can greatly improve teachers' work efficiency, freeing teachers from the tedious task of homework grading and allowing them to better devote themselves to teaching.

[0003] Among them, math calculation homework in the basic education stage often has the characteristics of a large number of questions and high repetition of daily routines. At present, the correction of calculation questions is mainly based on the text recognition results. Among them, the correction results are directly related to the accuracy of text recognition. When the text recognition accuracy is not high, the accuracy of the correction of calculation questions cannot be guaranteed. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a method, device, readable storage medium and electronic device for correcting calculation problems.

[0005] In order to achieve the above objectives, in a first aspect, the present disclosure provides a method for correcting a calculation problem, comprising:

[0006] Perform text recognition on the answer images to be corrected, and obtain the K text contents with the highest recognition probability as candidate text contents;

[0007] Determine whether the formula represented by each candidate text content is valid;

[0008] Determine the target text content from the K candidate text contents according to the determination result of whether the formula represented by each candidate text content is established and the recognition probability corresponding to each candidate text content;

[0009] According to the determination result of whether the formula represented by the target text content is established, the answer content contained in the answer image to be corrected is judged to be correct or not.

[0010] Optionally, performing text recognition on the answer image to be corrected to obtain K text contents with the highest recognition probability as candidate text contents includes:

[0011] Determine the binary image of the answer image to be corrected and the type of the calculation question contained in the answer image to be corrected;

[0012] Performing minimum circumscribed rectangle fitting on the text area in the binary image of the answer image to be corrected to obtain a target rectangular area;

[0013] According to the type of the calculation problem, text recognition is performed on the target rectangular area to obtain K text contents with the highest recognition probability as candidate text contents.

[0014] Optionally, determining the binary image of the answer image to be corrected and the type of the calculation question contained in the answer image to be corrected includes:

[0015] Inputting the answer image to be corrected into a pre-trained text detection model to obtain a binary image of the answer image to be corrected and the type of calculation question contained in the answer image to be corrected;

[0016] Wherein, the text detection model includes:

[0017] A first feature extraction module, used for generating a first feature map according to the answer image to be corrected;

[0018] A semantic segmentation module, connected to the first feature extraction module, for generating a pixel probability map and a pixel threshold map of the answer image to be corrected according to the first feature map;

[0019] A first processing module is connected to the semantic segmentation module and is used to generate N binary images corresponding to N preset calculation problem types according to the pixel probability map and the pixel threshold map, wherein the binary image with the largest text area among the N binary images is used as the target binary image;

[0020] N output channels corresponding to the N preset calculation problem types are respectively connected to the first processing module and are used to output binary images corresponding to the corresponding preset calculation problem types; and

[0021] The second processing module is connected to each of the output channels respectively, and is used to superimpose the binary images output by each of the output channels to obtain the binary image of the answer image to be corrected, and determine the preset calculation question type corresponding to the output channel that outputs the target binary image as the type of the calculation question.

[0022] Optionally, the text detection model is trained in the following way:

[0023] Acquire a historical question-answering image, a labeled binary image of the historical question-answering image, and a labeled type of a calculation question contained in the historical question-answering image;

[0024] The text detection model is obtained by taking the history answer image as the input of the first feature extraction module, taking the annotated binary image as the target output of the output channel corresponding to the annotation type among the N output channels, taking the preset binary image as the target output of each other output channel among the N output channels except the output channel corresponding to the annotation type, and taking the annotated binary image and the annotation type as the target outputs of the second processing module to perform model training, wherein the preset binary image only contains the background area.

[0025] Optionally, performing text recognition on the target rectangular area according to the type of the calculation problem to obtain K text contents with the highest recognition probability as candidate text contents includes:

[0026] According to the type of the calculation problem, determining whether the target rectangular area contains multiple text lines;

[0027] If the target rectangular area includes multiple text lines, the target rectangular area is segmented into multiple text line areas according to a segmentation rule corresponding to the type of the calculation problem;

[0028] Determining a target text row area from the plurality of text row areas according to the type of the calculation problem;

[0029] For each target text line region, perform text recognition on the target text line region to obtain M text contents with the highest recognition probability corresponding to the target text line region as candidate sub-text contents;

[0030] For each other text line region among the multiple text line regions, except the target text line region, performing text recognition on the other text line region to obtain target sub-text content corresponding to the other text line region;

[0031] Determine P character combinations, wherein each element in the character combination comes from any candidate sub-text content corresponding to different target character line regions, and the number of candidate sub-text content contained in the character combination is equal to the number of the target character line regions;

[0032] For each of the character combinations, according to the position information of the target character line area corresponding to the candidate sub-text content in the character combination and the position information of the other character line areas corresponding to each of the target sub-text content in the target rectangular area, each candidate sub-text content and each of the target sub-text content in the character combination are spliced ​​to obtain a spliced ​​text content, wherein the recognition probability of the spliced ​​text content is equal to the sum of the recognition probabilities of each candidate sub-text content constituting the spliced ​​text content;

[0033] The K concatenated text contents with the highest recognition probabilities among the P concatenated text contents are determined as the K candidate text contents with the highest recognition probabilities, wherein P≥K.

[0034] Optionally, determining a target text row area from the plurality of text row areas according to the type of the calculation problem includes:

[0035] If the type of the calculation problem is a non-formula calculation or a calculation for solving an equation, the text row area located at the top of the target rectangular area and the text row area located at the bottom of the target rectangular area among the multiple text row areas are determined as target text row areas;

[0036] If the type of the calculation problem is vertical calculation, the two text row areas located at the top of the target rectangular area among the multiple text row areas and the text row area located at the bottom of the target rectangular area among the multiple text row areas are determined as target text row areas.

[0037] Optionally, judging whether the target rectangular area includes a plurality of text lines according to the type of the calculation problem includes:

[0038] If the type of the calculation problem is one of vertical calculation, non-vertical calculation, and equation solving, it is determined that the target rectangular area includes a plurality of text lines.

[0039] Optionally, performing text recognition on the target text line region to obtain M text contents with the highest recognition probability corresponding to the target text line region as candidate sub-text contents includes:

[0040] Input the target text line region into a pre-trained recognition model to obtain M text contents with the highest recognition probability corresponding to the target text line region as candidate sub-text contents;

[0041] Wherein, the recognition model includes:

[0042] A second feature extraction module, used for generating a second feature map according to the target text line area;

[0043] A sequence encoding module, connected to the second feature extraction module, for performing sequence encoding on the second feature graph to obtain a feature sequence;

[0044] The prediction module is connected to the sequence encoding module and is used to decode the feature sequence through a beam search decoding method to obtain M text contents with the highest recognition probability corresponding to the target text line area as candidate sub-text contents, wherein the beam width of the beam search is M.

[0045] Optionally, the type of the calculation problem is one of vertical calculation, free-form calculation, equation solving calculation, fractional single-line calculation and non-fractional single-line calculation.

[0046] Optionally, judging whether the answer content contained in the answer image to be corrected is correct or not according to the judgment result of whether the formula represented by the target text content is valid, comprises:

[0047] If the equation represented by the target text content holds true, it is determined that the answer content contained in the answer image to be corrected is correct.

[0048] Optionally, determining the target text content from the K candidate text contents according to the determination result of whether the formula represented by each candidate text content is established and the recognition probability corresponding to each candidate text content includes:

[0049] Determine whether there is a candidate text content for which the represented formula is established among the K candidate text contents;

[0050] If there is a candidate text content for which the represented formula is satisfied among the K candidate text contents, then the candidate text content with the highest recognition probability among the candidate text contents for which the represented formula is satisfied among the K candidate text contents is determined as the target text content.

[0051] Optionally, the determining the target text content from the K candidate text contents according to the determination result of whether the formula represented by each candidate text content is established and the recognition probability corresponding to each candidate text content, further includes:

[0052] If there is no candidate text content for which the represented formula is satisfied among the K candidate text contents, then the candidate text content with the highest recognition probability among the K candidate text contents is determined as the target text content.

[0053] In a second aspect, the present disclosure provides a computing problem grading device, comprising:

[0054] The recognition module is used to perform text recognition on the answer images to be corrected, and obtain the K text contents with the highest recognition probability as candidate text contents;

[0055] A first determination module, used to determine whether the equation represented by each candidate text content obtained by the recognition module is established;

[0056] A determination module, configured to determine a target text content from K candidate text contents according to a determination result of whether a formula represented by each candidate text content obtained by the first determination module is established and a recognition probability corresponding to each candidate text content obtained by the recognition module;

[0057] The second determination module determines whether the answer content contained in the answer image to be corrected is correct or not based on the determination result of whether the formula represented by the target text content is established determined by the determination module.

[0058] In a third aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect of the present disclosure.

[0059] In a fourth aspect, the present disclosure provides an electronic device, including:

[0060] a memory having a computer program stored thereon;

[0061] A processor is used to execute the computer program in the memory to implement the steps of the method provided in the first aspect of the present disclosure.

[0062] In the above technical scheme, firstly, the answer image to be corrected is subjected to text recognition, and the K text contents with the highest recognition probability are obtained as candidate text contents; then, it is determined whether the formula represented by each candidate text content is valid; next, the target text content is determined from the K candidate text contents according to the determination result of whether the formula represented by each candidate text content is valid and the recognition probability corresponding to each candidate text content; finally, according to the determination result of whether the formula represented by the target text content is valid, the answer content contained in the answer image to be corrected is judged to be right or wrong. Among them, the target text content is determined from multiple text recognition results based on multiple text recognition results and the right or wrong judgment results corresponding to the multiple text recognition results, so that the accuracy of the target text content used for judging the right or wrong of the answer content can be guaranteed, the accuracy of the calculation question correction can be improved, and any answer image to be corrected can be corrected without relying on the question bank. In addition, based on the judgment result of whether the formula represented by the target text content is valid, the correctness of the answer content contained in the answer image to be corrected can be judged without distinguishing between the question stem and the answer content, and there is no need to distinguish between handwritten characters and printed characters, thereby improving the efficiency of correcting calculation questions.

[0063] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0065] Figure 1 The present invention is a flowchart of a method for correcting calculation problems according to an exemplary embodiment.

[0066] Figure 2 The present invention is a flowchart showing a method for performing text recognition on an answer image to be corrected according to an exemplary embodiment.

[0067] Figure 3 It is a structural diagram of a text detection model according to an exemplary embodiment.

[0068] Figure 4A The diagram is a schematic diagram showing a method of dividing a target rectangular area into a plurality of text line areas according to an exemplary embodiment.

[0069] Figure 4B It is a schematic diagram showing a method of dividing a target rectangular area into a plurality of text line areas according to another exemplary embodiment.

[0070] Figure 4C It is a schematic diagram showing a method of dividing a target rectangular area into a plurality of text line areas according to another exemplary embodiment.

[0071] Figure 4D It is a schematic diagram showing a method of dividing a target rectangular area into a plurality of text line areas according to another exemplary embodiment.

[0072] Figure 5 It is a structural diagram of a recognition model according to an exemplary embodiment.

[0073] Figure 6 The present invention is a block diagram of a computing problem correction device according to an exemplary embodiment.

[0074] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment.

[0075] Figure 8 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0076] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0077] Figure 1 FIG. 1 is a flow chart of a method for correcting a calculation problem according to an exemplary embodiment. Figure 1 As shown, the method includes S101 to S104.

[0078] In S101, text recognition is performed on the answer image to be corrected, and K text contents with the highest recognition probability are obtained as candidate text contents.

[0079] In the present disclosure, the above-mentioned calculation question grading method can be applied to electronic devices, such as terminal devices or servers connected to the terminal devices. The terminal devices can be, for example, smart phones, tablet computers, personal computers, etc. The answer image to be graded includes the calculation question and the answer corresponding to the calculation question (i.e., the answer content).

[0080] In S102, it is determined whether the equation represented by each candidate text content is established.

[0081] In the present disclosure, an operation may be performed on the algebraic expression represented by each candidate text content, and whether the algebraic expression represented by the candidate text content is established may be determined based on the operation result.

[0082] In S103, the target text content is determined from the K candidate text contents according to the determination result of whether the formula represented by each candidate text content is established and the recognition probability corresponding to each candidate text content.

[0083] In S104, based on the determination result of whether the formula represented by the target text content is established, the answer content contained in the answer image to be corrected is judged to be correct or not.

[0084] In the present disclosure, if the formula represented by the target text content is established, the answer content contained in the answer image to be correct is determined to be correct; if the formula represented by the target text content is not established, the answer content contained in the answer image to be correct is determined to be wrong.

[0085] In the above technical scheme, firstly, the answer image to be corrected is subjected to text recognition, and the K text contents with the highest recognition probability are obtained as candidate text contents; then, it is determined whether the formula represented by each candidate text content is valid; next, the target text content is determined from the K candidate text contents according to the determination result of whether the formula represented by each candidate text content is valid and the recognition probability corresponding to each candidate text content; finally, according to the determination result of whether the formula represented by the target text content is valid, the answer content contained in the answer image to be corrected is judged to be right or wrong. Among them, the target text content is determined from multiple text recognition results based on multiple text recognition results and the right or wrong judgment results corresponding to the multiple text recognition results, so that the accuracy of the target text content used for judging the right or wrong of the answer content can be guaranteed, the accuracy of the calculation question correction can be improved, and any answer image to be corrected can be corrected without relying on the question bank. In addition, based on the judgment result of whether the formula represented by the target text content is valid, the correctness of the answer content contained in the answer image to be corrected can be judged without distinguishing between the question stem and the answer content, and there is no need to distinguish between handwritten characters and printed characters, thereby improving the efficiency of correcting calculation questions.

[0086] The following is a detailed description of a specific implementation method for performing text recognition on the answer image to be corrected in S101 above to obtain K text contents with the highest recognition probability as candidate text contents. Figure 2 It is implemented by S1011 to S1013 shown in FIG.

[0087] In S1011 , the binary image of the answer image to be corrected and the type of the calculation question included in the answer image to be corrected are determined.

[0088] In the present disclosure, the binary image of the answer image of the homework to be corrected includes a text area and a background area. The type of calculation question is one of vertical calculation, escape calculation, equation solving calculation, fraction single-line calculation and non-fraction single-line calculation. Among them, non-fraction single-line calculation includes four arithmetic mixed operations, comparison of size, unit conversion, estimation, remainder calculation, etc.; fraction single-line calculation is a single-line calculation with fractions, for example, Single-row calculation for fractions.

[0089] In addition, vertical calculations, skipped-form calculations, and equation solving are all multi-line calculations, while fractional single-line calculations and non-fractional single-line calculations are all single-line calculations, that is, the calculation question and the answer are on the same line.

[0090] In S1012, a minimum circumscribed rectangle is fitted to the text area in the binary image of the answer image to be corrected to obtain a target rectangular area.

[0091] In S1013, text recognition is performed on the target rectangular area according to the type of the calculation problem, and K text contents with the highest recognition probability are obtained as candidate text contents.

[0092] The following is a detailed description of the specific implementation method for determining the binary image of the answer image to be corrected and the type of calculation questions contained in the answer image to be corrected in the above S1011. Specifically, the answer image to be corrected can be input into a pre-trained text detection model to obtain the binary image of the answer image to be corrected and the type of calculation questions contained in the answer image to be corrected. Among them, the above-mentioned text detection module can be, for example, a real-time scene text detection model based on differentiable binarization (Real-time Scene Text Detection with DifferentiableBinarization, DBNet), mask-rcnn, Faster-rcnn, RetinaNet and other models.

[0093] Specifically, the above-mentioned text detection model may include: a first feature extraction module, used to generate a first feature map according to the answer image to be corrected; a semantic segmentation module, connected to the first feature extraction module, used to generate a pixel probability map and a pixel threshold map of the answer image to be corrected according to the first feature map; a first processing module, connected to the semantic segmentation module, used to generate N binary images corresponding to N preset calculation question types one by one according to the pixel probability map and the pixel threshold map, wherein the binary image with the largest text area in the N binary images is used as the target binary image; N output channels corresponding to the N preset calculation question types one by one, respectively connected to the first processing module, used to output binary images corresponding to the corresponding preset calculation question types; and a second processing module, respectively connected to each output channel, used to superimpose the binary images output by each output channel to obtain a binary image of the answer image to be corrected, and determine the preset calculation question type corresponding to the output channel that outputs the target binary image as the type of the calculation question.

[0094] In the present disclosure, the binary image output by the output channel among the above-mentioned N channels corresponding to the type of calculation question contained in the answer image to be corrected will contain more text content, while the binary images output by other output channels only contain background areas. Therefore, the preset calculation question type corresponding to the output channel that outputs the binary image with the largest text area among the N binary images (i.e., the target binary image) is determined as the type of calculation question contained in the answer image to be corrected.

[0095] For example, the N preset calculation problem types include: vertical calculation, escaped calculation, equation solving calculation, fractional single-line calculation and non-fractional single-line calculation, that is, N=5. Figure 3 As shown, the above text detection model includes 5 output channels corresponding to vertical calculation, stripped calculation, equation solution calculation, fraction single-line calculation and non-fraction single-line calculation, namely, the output channel corresponding to vertical calculation, the output channel corresponding to stripped calculation, the output channel corresponding to equation solution calculation, the output channel corresponding to fraction single-line calculation and the output channel corresponding to non-fraction single-line calculation. Among them, the first processing module is used to output binary images (i.e. 5 binary images) corresponding to vertical calculation, stripped calculation, equation solution calculation, fraction single-line calculation and non-fraction single-line calculation respectively; and the output channel corresponding to vertical calculation is used to output the binary image corresponding to vertical calculation, the output channel corresponding to stripped calculation is used to output the binary image corresponding to stripped calculation, the output channel corresponding to equation solution calculation is used to output the binary image corresponding to equation solution calculation, the output channel corresponding to fraction single-line calculation is used to output the binary image corresponding to fraction single-line calculation, and the output channel corresponding to non-fraction single-line calculation is used to output the binary image corresponding to non-fraction single-line calculation.

[0096] In the above-mentioned text detection model, the preset calculation question type corresponding to the output channel of the output target binary image is determined as the type of calculation question contained in the answer image to be corrected, that is, the output channel can characterize the type of calculation question. In this way, the type of calculation question contained in the answer image to be corrected can be quickly determined through the output channel without setting a classifier in the text detection model, thereby simplifying the structure of the text detection model and quickly determining the type of calculation question contained in the answer image to be corrected.

[0097] In addition, the above text detection model is trained in the following way:

[0098] First, obtain a history answer image, an annotated binary image of the history answer image, and an annotation type of the calculation question contained in the history answer image, wherein the annotation type can be one of vertical calculation, escaped calculation, equation solving calculation, fractional single-line calculation, and non-fractional single-line calculation; then, take the history answer image as the input of the first feature extraction module, take the annotated binary image as the target output of the output channel corresponding to the annotation type among N output channels, take the preset binary image as the target output of each other output channel among N output channels except the output channel corresponding to the annotation type, take the annotated binary image and the annotation type as the target output of the second processing module to perform model training, so as to obtain a text detection model, wherein the preset binary image only contains the background area.

[0099] The following is a detailed description of the specific implementation method of performing text recognition on the target rectangular area according to the type of the calculation problem in the above S1013 to obtain K text contents with the highest recognition probability as candidate text contents. Specifically, it can be achieved by the following steps (1) to (8):

[0100] (1) Based on the type of calculation problem, determine whether the target rectangular area contains multiple lines of text.

[0101] In the present disclosure, if the type of the calculation problem is one of vertical calculation, escape calculation, and equation solving, it is determined that the target rectangular area contains multiple text lines. If the type of the calculation problem is fraction single-line calculation or non-fraction single-line calculation, it is determined that the target rectangular area does not contain multiple characters.

[0102] (2) If the target rectangular area contains multiple text lines, the target rectangular area is segmented into multiple text line areas according to the segmentation rules corresponding to the type of the calculation problem.

[0103] For example, Figure 4A As shown, the type of calculation problem is vertical calculation, which can be done according to Figure 4A The target rectangular area is divided into three text line areas in the manner shown in .

[0104] For example, Figure 4B As shown, the type of calculation problem is vertical calculation, which can be done according to Figure 4B The target rectangular area is divided into six text line areas in the manner shown in .

[0105] Among them, the horizontal line in the vertical calculation can be divided into a text line area with the previous text adjacent to it (such as Figure 4A ), or the next line adjacent to it can be divided into a text line area (as shown in Figure 4B ).

[0106] For example, Figure 4C As shown in the figure, the type of calculation problem is free calculation, which can be done according to Figure 4C The target rectangular area is divided into four text line areas in the manner shown in .

[0107] For example, Figure 4D As shown, the type of calculation problem is to solve the equation. Figure 4D In the manner shown, the target rectangular area is divided into three text line areas.

[0108] (3) According to the type of the calculation problem, a target text line area is determined from multiple text line areas.

[0109] In the present disclosure, the stem of the calculation of the out-of-form calculation or the solution of the equation is usually located in the first line, and the final answer is located in the last line, which is the key to the correction of the calculation problem. Therefore, it is necessary to focus on the text line area at the top of the corresponding target rectangular area and the text line area at the bottom of the target rectangular area (that is, to obtain multiple recognition results). That is, if the type of the calculation problem is the out-of-form calculation or the solution of the equation, the text line area at the top of the target rectangular area and the text line area at the bottom of the target rectangular area among the multiple text line areas are determined as the target text line area.

[0110] The stem of the vertical calculation is usually located in the first two lines, and the final answer is located in the last line, which is the key to correcting the calculation questions. Therefore, it is necessary to focus on identifying the two text line areas at the top of the corresponding target rectangular area and the text line area at the bottom of the target rectangular area. That is, if the type of the calculation question is vertical calculation, the two text line areas at the top of the target rectangular area and the text line area at the bottom of the target rectangular area among the multiple text line areas are determined as the target text line area.

[0111] For example, Figure 4A The type of calculation problem contained in is vertical calculation, so Figure 4A The three text line areas shown in are all determined as target text line areas.

[0112] As another example, Figure 4B The type of calculation problem contained in is vertical calculation, so the text line area where "25" is located, The text row area where "0" is located and the text row area where "0" is located are determined as the target text row area.

[0113] As another example, Figure 4C The type of calculation problem included in is free calculation, so the text line area where "4×7×25" is located and the text line area where "=700" is located can be determined as the target text line area.

[0114] As another example, Figure 4D The type of calculation problem included in is the calculation of solving equations, so the text line area where "4×x+5=25" is located and the text line area where "x=5" is located can be determined as the target text line area.

[0115] (4) For each target text line region, perform text recognition on the target text line region to obtain M text contents with the highest recognition probability corresponding to the target text line region as candidate sub-text contents.

[0116] (5) For each of the multiple text line regions except the target text line region, text recognition is performed on the other text line region to obtain the target sub-text content corresponding to the other text line region.

[0117] (6) Determine P text combinations, where each element in the text combination comes from any candidate sub-text content corresponding to different target text line regions, and the number of candidate sub-text contents contained in the text combination is equal to the number of target text line regions.

[0118] In the present disclosure, P may be less than or equal to M n , where n is the number of target text line areas.

[0119] For example, M=3, P=6, such as Figure 4CAs shown in , two target text line areas are obtained through the above step (3) (i.e., the text line area where "4×7×25" is located, and the text line area where "=700" is located"), wherein the three candidate sub-text contents corresponding to the text line area where "4×7×25" is located are "4×7×25", "4×7×55", and "4×7×35", respectively, and the three candidate sub-text contents corresponding to the text line area where "=700" is located are "=700", "=708", and "=788", respectively. Then, the number of candidate sub-text contents contained in each text combination is 2, wherein one element in each text combination comes from any one of "4×7×25", "4×7×55", and "4×7×35", and the other element comes from any one of "=700", "=708", and "=788". Therefore, at most 3 can be obtained. 2 =9 character combinations, namely "4×7×25", "=700", "4×7×55", "=700", "4×7×35", "=700", "4×7×25", "=708", "4×7×55", "=708", "4×7×35", "=708", "4×7×25", "=788", "4×7×55", "=788", "4×7×35", "=788", and 6 character combinations can be randomly selected from the 9 character combinations.

[0120] (7) For each character combination, based on the position information of the target character line area corresponding to the candidate sub-text content in the character combination and the position information of other character line areas corresponding to each target sub-text content in the target rectangular area, each candidate sub-text content and each target sub-text content in the character combination are spliced ​​to obtain a spliced ​​text content, wherein the recognition probability of the spliced ​​text content is equal to the sum of the recognition probabilities of each candidate sub-text content constituting the spliced ​​text content.

[0121] (8) Determine the K concatenated text contents with the highest recognition probabilities among the P concatenated text contents as the K candidate text contents with the highest recognition probabilities, where P ≥ K.

[0122] In the above embodiment, if the target rectangular area contains multiple text lines, the target rectangular area is divided into multiple text line areas, and text recognition is performed on each text line area respectively. After that, the text recognition results corresponding to each text line area are merged to obtain the text recognition result corresponding to the target rectangular area. In this way, multiple lines of formulas can be recognized, and text recognition can be performed in parallel on each text line area, thereby improving the efficiency of text recognition in the target rectangular area.

[0123] The following describes in detail the specific implementation method of performing text recognition on the target text line area in the above step (6) to obtain M text contents with the highest recognition probability corresponding to the target text line area as candidate sub-text contents.

[0124] Specifically, the target text line region can be input into a pre-trained recognition model to obtain M text contents with the highest recognition probability corresponding to the target text line region as candidate sub-text contents. The recognition model can be, for example, a convolutional recurrent neural network (CRNN), a sequence-to-sequence prediction model based on an attention mechanism, etc.

[0125] For example, Figure 5 As shown, the above-mentioned recognition model may include: a second feature extraction module, used to generate a second feature map according to the target text line area; a sequence encoding module, connected to the second feature extraction module, used to sequence encode the second feature map to obtain a feature sequence; a prediction module, connected to the sequence encoding module, used to decode the feature sequence through a beam search decoding method to obtain the M text contents with the highest recognition probability corresponding to the target text line area as candidate sub-text contents, wherein the beam width of the beam search is M.

[0126] The following is a detailed description of the specific implementation method of determining the target text content from the K candidate text contents according to the determination result of whether the formula represented by each candidate text content is established and the recognition probability corresponding to each candidate text content in the above S103. Specifically, it can be achieved by the following methods:

[0127] First, determine whether there is candidate text content for which the represented formula holds among the K candidate text contents; if there is candidate text content for which the represented formula holds among the K candidate text contents, then determine the candidate text content with the highest recognition probability among the candidate text contents for which the represented formula holds among the K candidate text contents as the target text content; if there is no candidate text content for which the represented formula holds among the K candidate text contents, then directly determine the candidate text content with the highest recognition probability among the K candidate text contents as the target text content.

[0128] For example, for the answer image to be corrected containing "2+3=5", the five candidate text contents with the highest recognition probability, the recognition probability corresponding to each candidate text content, and the judgment result of whether the formula represented by each candidate text content is valid are obtained as shown in Table 1 below.

[0129] Table 1 Multiple candidate text contents with the highest recognition probability and their corresponding recognition probabilities and judgment results

[0130]

[0131] It can be seen from Table 1 above that among the 5 candidate text contents, there are candidate text contents for which the represented formula holds true, namely, the candidate text "2+3=5" and the candidate text "5+3=8", among which the recognition probability corresponding to the candidate text "2+3=5" is greater than the recognition probability corresponding to the candidate text "5+3=8", therefore, the candidate text "2+3=5" is determined as the target text content.

[0132] As another example, for the answer image to be corrected containing "2+3=5", the five candidate text contents with the highest recognition probability, the recognition probability corresponding to each candidate text content, and the judgment result of whether the formula represented by each candidate text content is valid are obtained as shown in Table 2 below.

[0133] Table 2 Multiple candidate text contents with the highest recognition probability and their corresponding recognition probabilities and judgment results

[0134]

[0135] It can be seen from Table 2 above that there is no candidate text content for which the represented formula is true among the 5 candidate text contents. Therefore, the candidate text content "2+2=5" with the highest recognition probability among the 5 candidate text contents is directly determined as the target text content.

[0136] Figure 6 FIG. 1 is a block diagram of a computing problem correction device according to an exemplary embodiment. Figure 6 As shown, the calculation question correction device 600 includes: a recognition module 601, which is used to perform text recognition on the answer image to be corrected, and obtain K text contents with the highest recognition probability as candidate text contents; a first judgment module 602, which is used to respectively judge whether the formula represented by each of the candidate text contents obtained by the recognition module 601 is valid; a determination module 603, which is used to determine the target text content from the K candidate text contents according to the judgment result of whether the formula represented by each of the candidate text contents obtained by the first judgment module 602 is valid and the recognition probability corresponding to each of the candidate text contents obtained by the recognition module 601; a second judgment module 604, which uses the judgment result of whether the formula represented by the target text content determined by the determination module 603 is valid to judge whether the answer content contained in the answer image to be corrected is correct.

[0137] In the above technical scheme, firstly, the answer image to be corrected is subjected to text recognition, and the K text contents with the highest recognition probability are obtained as candidate text contents; then, it is determined whether the formula represented by each candidate text content is valid; next, the target text content is determined from the K candidate text contents according to the determination result of whether the formula represented by each candidate text content is valid and the recognition probability corresponding to each candidate text content; finally, according to the determination result of whether the formula represented by the target text content is valid, the answer content contained in the answer image to be corrected is judged to be right or wrong. Among them, the target text content is determined from multiple text recognition results based on multiple text recognition results and the right or wrong judgment results corresponding to the multiple text recognition results, so that the accuracy of the target text content used for judging the right or wrong of the answer content can be guaranteed, the accuracy of the calculation question correction can be improved, and any answer image to be corrected can be corrected without relying on the question bank. In addition, based on the judgment result of whether the formula represented by the target text content is valid, the correctness of the answer content contained in the answer image to be corrected can be judged without distinguishing between the question stem and the answer content, and there is no need to distinguish between handwritten characters and printed characters, thereby improving the efficiency of correcting calculation questions.

[0138] Optionally, the recognition module 601 includes: a first determination submodule, used to determine the binary image of the answer image to be corrected and the type of calculation questions contained in the answer image to be corrected; a fitting submodule, used to perform minimum circumscribed rectangle fitting on the text area in the binary image of the answer image to be corrected to obtain a target rectangular area; a target rectangular area recognition submodule, used to perform text recognition on the target rectangular area according to the type of the calculation question, and obtain K text contents with the highest recognition probability as candidate text contents.

[0139] Optionally, the first determination submodule is used to input the answer image to be corrected into a pre-trained text detection model to obtain a binary image of the answer image to be corrected and the type of calculation question contained in the answer image to be corrected; wherein the text detection model includes: a first feature extraction module, used to generate a first feature map according to the answer image to be corrected; a semantic segmentation module, connected to the first feature extraction module, used to generate a pixel probability map and a pixel threshold map of the answer image to be corrected according to the first feature map; a first processing module, connected to the semantic segmentation module, used to generate a pixel probability map and a pixel threshold map according to the pixel probability map and the pixel threshold map , generating N binary images corresponding to N preset calculation question types one by one, wherein the binary image with the largest text area among the N binary images is used as the target binary image; N output channels corresponding to the N preset calculation question types one by one are respectively connected to the first processing module for outputting the binary images corresponding to the corresponding preset calculation question types; and a second processing module is respectively connected to each of the output channels for superimposing the binary images output by each of the output channels to obtain the binary image of the answer image to be corrected, and determining the preset calculation question type corresponding to the output channel that outputs the target binary image as the type of the calculation question.

[0140] Optionally, the text detection model is obtained by training through a first model training device, wherein the first model training device includes: a first acquisition module, used to obtain a historical answer image, an annotated binary image of the historical answer image, and an annotated type of the calculation question contained in the historical answer image; a first training module, used to perform model training by taking the historical answer image as the input of the first feature extraction module, taking the annotated binary image as the target output of the output channel among the N output channels corresponding to the annotation type, taking the preset binary image as the target output of each other output channel among the N output channels except the output channel corresponding to the annotation type, and taking the annotated binary image and the annotation type as the target output of the second processing module to obtain the text detection model, wherein the preset binary image only contains the background area.

[0141] Optionally, the target rectangular area identification submodule includes: a first judgment submodule, used to judge whether the target rectangular area contains multiple text lines according to the type of the calculation problem; a segmentation submodule, used to divide the target rectangular area into multiple text line areas according to a segmentation rule corresponding to the type of the calculation problem if the target rectangular area contains multiple text lines; a second determination submodule, used to determine the target text line area from the multiple text line areas according to the type of the calculation problem; a target text line area identification submodule, used to perform text recognition on the target text line area for each of the target text line areas, and obtain M text contents with the highest recognition probability corresponding to the target text line area as candidate sub-text contents; an other text line area identification submodule, used to perform text recognition on each other text line area among the multiple text line areas except the target text line area, and obtain the target sub-text content corresponding to the other text line area; a third determination submodule A block for determining P character combinations, wherein each element in the character combination comes from any of the candidate sub-text contents corresponding to different target character line areas, and the number of candidate sub-text contents contained in the character combination is equal to the number of the target character line areas; a splicing submodule for, for each of the character combinations, splicing each candidate sub-text content and each target sub-text content in the character combination according to the position information of the target character line area corresponding to the candidate sub-text content in the character combination in the target rectangular area and the position information of other character line areas corresponding to each target sub-text content in the target rectangular area to obtain a spliced ​​text content, wherein the recognition probability of the spliced ​​text content is equal to the sum of the recognition probabilities of each candidate sub-text content constituting the spliced ​​text content; a fourth determination submodule for determining the K spliced ​​text contents with the highest recognition probability among the P spliced ​​text contents as the K candidate text contents with the highest recognition probability, wherein P≥K.

[0142] Optionally, the second determination submodule includes: a fifth determination submodule, which is used to determine the text row area at the top of the target rectangular area and the text row area at the bottom of the target rectangular area among the multiple text row areas as target text row areas if the type of the calculation problem is free-form calculation or equation solving calculation; a sixth determination submodule, which is used to determine the two text row areas at the top of the target rectangular area and the text row area at the bottom of the target rectangular area among the multiple text row areas as target text row areas if the type of the calculation problem is vertical calculation.

[0143] Optionally, the first judgment submodule is used to determine that the target rectangular area includes multiple text lines if the type of the calculation problem is one of vertical calculation, free form calculation, and equation solving calculation.

[0144] Optionally, the target text row area recognition submodule is used to input the target text row area into a pre-trained recognition model to obtain M text contents with the highest recognition probability corresponding to the target text row area as candidate sub-text contents; wherein the recognition model includes: a second feature extraction module, used to generate a second feature map according to the target text row area; a sequence encoding module, connected to the second feature extraction module, used to sequence encode the second feature map to obtain a feature sequence; a prediction module, connected to the sequence encoding module, used to decode the feature sequence by a beam search decoding method, to obtain M text contents with the highest recognition probability corresponding to the target text row area as candidate sub-text contents, wherein the beam width of the beam search is M.

[0145] Optionally, the type of the calculation problem is one of vertical calculation, free-form calculation, equation solving calculation, fractional single-line calculation and non-fractional single-line calculation.

[0146] Optionally, the second determination module 604 is used to determine that the answer content contained in the answer image to be correct is correct if the equation represented by the target text content is established.

[0147] Optionally, the determination module 603 includes: a second judgment submodule, used to determine whether there is candidate text content for which the represented formula is established among the K candidate text contents; and a seventh determination submodule, used to determine the candidate text content with the highest recognition probability among the candidate text contents for which the represented formula is established among the K candidate text contents as the target text content if there is candidate text content for which the represented formula is established among the K candidate text contents.

[0148] Optionally, the determination module 603 also includes: an eighth determination submodule, which is used to determine the candidate text content with the highest recognition probability among the K candidate text contents as the target text content if there is no candidate text content for which the represented formula is satisfied among the K candidate text contents.

[0149] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0150] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the above-mentioned calculation problem correction method provided by the present disclosure are implemented.

[0151] Figure 7 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 7 As shown, the electronic device 700 may include: a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0152] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned method for correcting calculation problems. The memory 702 is used to store various types of data to support the operation of the electronic device 700. For example, these data may include instructions for any application or method used to operate on the electronic device 700, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 705 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0153] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned calculation problem correction method.

[0154] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned calculation problem correction method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions can be executed by the processor 701 of the electronic device 700 to complete the above-mentioned calculation problem correction method.

[0155] Figure 8 8 is a block diagram of an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be provided as a server. Figure 8 The electronic device 800 includes a processor 822, which may be one or more, and a memory 832 for storing a computer program executable by the processor 822. The computer program stored in the memory 832 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 822 may be configured to execute the computer program to perform the above-mentioned calculation problem correction method.

[0156] In addition, the electronic device 800 may further include a power supply component 826 and a communication component 850. The power supply component 826 may be configured to perform power management of the electronic device 800, and the communication component 850 may be configured to implement communication of the electronic device 800, for example, wired or wireless communication. In addition, the electronic device 800 may further include an input / output (I / O) interface 858. The electronic device 800 may operate based on an operating system stored in the memory 832, such as Windows Server 2000. TM , Mac OSX TM , Unix TM , Linux TM etc.

[0157] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned calculation problem correction method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 832 including program instructions, and the above-mentioned program instructions can be executed by the processor 822 of the electronic device 800 to complete the above-mentioned calculation problem correction method.

[0158] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned method for correcting calculation problems when executed by the programmable device.

[0159] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0160] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0161] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for correcting calculation problems, characterized in that: include: Perform text recognition on the answer images to be corrected, and obtain the K text contents with the highest recognition probability as candidate text contents; Determine whether the formula represented by each candidate text content is valid; Determine the target text content from the K candidate text contents according to the determination result of whether the formula represented by each candidate text content is established and the recognition probability corresponding to each candidate text content; According to the determination result of whether the formula represented by the target text content is established, determining whether the answer content contained in the answer image to be corrected is correct or not; The step of performing text recognition on the answer image to be corrected and obtaining K text contents with the highest recognition probability as candidate text contents includes: Determine the binary image of the answer image to be corrected and the type of the calculation question contained in the answer image to be corrected; Performing minimum circumscribed rectangle fitting on the text area in the binary image of the answer image to be corrected to obtain a target rectangular area; According to the type of the calculation problem, text recognition is performed on the target rectangular area to obtain K text contents with the highest recognition probability as candidate text contents; The step of performing text recognition on the target rectangular area according to the type of the calculation problem to obtain K text contents with the highest recognition probability as candidate text contents includes: According to the type of the calculation problem, determining whether the target rectangular area contains multiple text lines; If the target rectangular area includes multiple text lines, the target rectangular area is segmented into multiple text line areas according to a segmentation rule corresponding to the type of the calculation problem; Determining a target text row area from the plurality of text row areas according to the type of the calculation problem; For each target text line region, perform text recognition on the target text line region to obtain M text contents with the highest recognition probability corresponding to the target text line region as candidate sub-text contents; For each other text line region among the multiple text line regions, except the target text line region, performing text recognition on the other text line region to obtain target sub-text content corresponding to the other text line region; Determine P character combinations, wherein each element in the character combination comes from any candidate sub-text content corresponding to different target character line regions, and the number of candidate sub-text content contained in the character combination is equal to the number of the target character line regions; For each of the character combinations, according to the position information of the target character line area corresponding to the candidate sub-text content in the character combination and the position information of the other character line areas corresponding to each of the target sub-text content in the target rectangular area, each candidate sub-text content and each of the target sub-text content in the character combination are spliced ​​to obtain a spliced ​​text content, wherein the recognition probability of the spliced ​​text content is equal to the sum of the recognition probabilities of each candidate sub-text content constituting the spliced ​​text content; The K concatenated text contents with the highest recognition probabilities among the P concatenated text contents are determined as the K candidate text contents with the highest recognition probabilities, wherein P≥K.

2. The method according to claim 1, characterized in that The determining of the binary image of the answer image to be corrected and the type of the calculation question contained in the answer image to be corrected includes: Inputting the answer image to be corrected into a pre-trained text detection model to obtain a binary image of the answer image to be corrected and the type of calculation question contained in the answer image to be corrected; Wherein, the text detection model includes: A first feature extraction module, used for generating a first feature map according to the answer image to be corrected; A semantic segmentation module, connected to the first feature extraction module, for generating a pixel probability map and a pixel threshold map of the answer image to be corrected according to the first feature map; A first processing module is connected to the semantic segmentation module and is used to generate N binary images corresponding to N preset calculation problem types according to the pixel probability map and the pixel threshold map, wherein the binary image with the largest text area among the N binary images is used as the target binary image; N output channels corresponding to the N preset calculation problem types are respectively connected to the first processing module and are used to output binary images corresponding to the corresponding preset calculation problem types; and The second processing module is connected to each of the output channels respectively, and is used to superimpose the binary images output by each of the output channels to obtain the binary image of the answer image to be corrected, and determine the preset calculation question type corresponding to the output channel that outputs the target binary image as the type of the calculation question.

3. The method according to claim 2, characterized in that The text detection model is trained in the following way: Acquire a historical question-answering image, a labeled binary image of the historical question-answering image, and a labeled type of a calculation question contained in the historical question-answering image; The text detection model is obtained by taking the history answer image as the input of the first feature extraction module, taking the annotated binary image as the target output of the output channel corresponding to the annotation type among the N output channels, taking the preset binary image as the target output of each other output channel among the N output channels except the output channel corresponding to the annotation type, and taking the annotated binary image and the annotation type as the target outputs of the second processing module to perform model training, wherein the preset binary image only contains the background area.

4. The method according to claim 1, characterized in that The step of determining a target text row area from the plurality of text row areas according to the type of the calculation problem comprises: If the type of the calculation problem is a non-formula calculation or a calculation for solving an equation, the text row area located at the top of the target rectangular area among the multiple text row areas and the text row area located at the bottom of the target rectangular area among the multiple text row areas are determined as target text row areas; If the type of the calculation problem is vertical calculation, the two text row areas located at the top of the target rectangular area among the multiple text row areas and the text row area located at the bottom of the target rectangular area among the multiple text row areas are determined as target text row areas.

5. The method according to claim 1, characterized in that The step of judging whether the target rectangular area includes a plurality of text lines according to the type of the calculation problem includes: If the type of the calculation problem is one of vertical calculation, non-vertical calculation, and equation solving, it is determined that the target rectangular area includes a plurality of text lines.

6. The method according to claim 1, characterized in that The performing text recognition on the target text line region to obtain M text contents with the highest recognition probability corresponding to the target text line region as candidate sub-text contents includes: Input the target text line area into a pre-trained recognition model to obtain M text contents with the highest recognition probability corresponding to the target text line area as candidate sub-text contents; Wherein, the recognition model includes: A second feature extraction module, used for generating a second feature map according to the target text line area; A sequence encoding module, connected to the second feature extraction module, for performing sequence encoding on the second feature graph to obtain a feature sequence; The prediction module is connected to the sequence encoding module and is used to decode the feature sequence through a beam search decoding method to obtain M text contents with the highest recognition probability corresponding to the target text line area as candidate sub-text contents, wherein the beam width of the beam search is M.

7. The method according to any one of claims 1 to 6, characterized in that The type of the calculation problem is one of vertical calculation, skipped-form calculation, equation solving calculation, fractional single-line calculation and non-fractional single-line calculation.

8. The method according to any one of claims 1 to 6, characterized in that The step of judging whether the answer content contained in the answer image to be corrected is correct or not based on the judgment result of whether the formula represented by the target text content is valid includes: If the equation represented by the target text content holds true, it is determined that the answer content contained in the answer image to be corrected is correct.

9. The method according to any one of claims 1 to 6, characterized in that: The step of determining the target text content from the K candidate text contents according to the determination result of whether the formula represented by each candidate text content is established and the recognition probability corresponding to each candidate text content comprises: Determine whether there is a candidate text content for which the represented formula is established among the K candidate text contents; If there is a candidate text content for which the represented formula is satisfied among the K candidate text contents, then the candidate text content with the highest recognition probability among the candidate text contents for which the represented formula is satisfied among the K candidate text contents is determined as the target text content.

10. The method according to claim 9, characterized in that The step of determining the target text content from the K candidate text contents according to the determination result of whether the formula represented by each candidate text content is established and the recognition probability corresponding to each candidate text content further includes: If there is no candidate text content for which the represented formula is satisfied among the K candidate text contents, then the candidate text content with the highest recognition probability among the K candidate text contents is determined as the target text content.

11. A device for correcting calculation problems, characterized in that: include: The recognition module is used to perform text recognition on the answer images to be corrected, and obtain the K text contents with the highest recognition probability as candidate text contents; A first determination module, used to determine whether the equation represented by each candidate text content obtained by the recognition module is established; A determination module, configured to determine a target text content from K candidate text contents according to a determination result of whether a formula represented by each candidate text content obtained by the first determination module is established and a recognition probability corresponding to each candidate text content obtained by the recognition module; A second determination module is used to determine whether the answer content contained in the answer image to be corrected is correct or not according to the determination result of whether the formula represented by the target text content is established determined by the determination module; Wherein, the identification module includes: A first determination submodule is used to determine the binary image of the answer image to be corrected and the type of the calculation question contained in the answer image to be corrected; A fitting submodule, used for performing minimum circumscribed rectangle fitting on the text area in the binary image of the answer image to be corrected to obtain a target rectangular area; A target rectangular area recognition submodule is used to perform text recognition on the target rectangular area according to the type of the calculation problem, and obtain K text contents with the highest recognition probability as candidate text contents; The target rectangular area recognition submodule includes: A first judgment submodule is used to judge whether the target rectangular area contains multiple text lines according to the type of the calculation problem; a segmentation submodule, for segmenting the target rectangular area into a plurality of text line areas according to a segmentation rule corresponding to the type of the calculation problem if the target rectangular area includes a plurality of text lines; A second determination submodule is used to determine a target text row area from the plurality of text row areas according to the type of the calculation problem; A target text line region recognition submodule is used to perform text recognition on each target text line region, and obtain M text contents with the highest recognition probability corresponding to the target text line region as candidate subtext contents; Other text line area recognition submodule, for performing text recognition on each other text line area in the plurality of text line areas except the target text line area, to obtain target subtext content corresponding to the other text line area; A third determination submodule is used to determine P character combinations, wherein each element in the character combination is from any candidate sub-text content corresponding to different target character line regions, and the number of candidate sub-text content contained in the character combination is equal to the number of target character line regions; a splicing submodule, for splicing each candidate subtext content and each target subtext content in the character combination according to the position information of the target text line area corresponding to the candidate subtext content in the character combination and the position information of other text line areas corresponding to each target subtext content in the target rectangular area, so as to obtain a spliced ​​text content, wherein the recognition probability of the spliced ​​text content is equal to the sum of the recognition probabilities of each candidate subtext content constituting the spliced ​​text content; The fourth determination submodule is used to determine K concatenated text contents with the highest recognition probabilities among the P concatenated text contents as the K candidate text contents with the highest recognition probabilities, wherein P≥K.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 10 are implemented.

13. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 10.

Images