Homework correction method, device, storage medium and electronic device
By performing text line recognition and region weight correction on single-question images, and combining this with the reference answer string, grading information is generated, which solves the problem of high grading error rate in existing technologies and improves the accuracy of homework grading.
Patent Information
- Application Number
- CN202210179832.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-25
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-02-25
AI Technical Summary
Existing AI-based homework grading methods suffer from high error rates, hindering their widespread adoption.
By acquiring individual question images of assignments to be graded, text line recognition is performed. Based on regional distribution and preset regional weight correction probabilities, character recognition is performed to generate grading information. Difference analysis is then conducted in conjunction with the reference answer string.
It improved the accuracy of recognizing lines of answer text, reduced the error rate in grading, and enhanced the accuracy of homework grading.
Smart Images

Figure CN114581649B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular, to a homework grading method, device, storage medium and electronic device. Background Art
[0002] For a long time, students' homework has been graded manually by teachers. Manual grading has the problems of heavy workload, unstable grading results and low efficiency. In recent years, with the rapid development of artificial intelligence, automatic grading methods based on artificial intelligence technology have gradually emerged to improve the problems of manual grading by teachers.
[0003] However, the artificial intelligence-based homework grading method in the related art has the problem of high grading error rate, which makes the artificial intelligence-based homework grading method not yet widely popularized. Summary of the invention
[0004] The purpose of the present disclosure is to provide a homework grading method, device, storage medium and electronic device, which solve the problem of high correction error rate in homework grading.
[0005] In order to achieve the above objectives, in a first aspect, the present disclosure provides a homework marking method, the method comprising:
[0006] Get the single-question image corresponding to the homework to be graded;
[0007] Performing text line recognition on the single-question image to obtain a first probability that each text line included in the single-question image is an answer text line;
[0008] Based on the regional distribution of each text line in the single-question image and the preset regional weight, the first probability is corrected to obtain a second probability that each text line is an answer text line, wherein the preset regional weight represents the probability of the answer text line being distributed in each region;
[0009] Performing text recognition on the target text line whose second probability meets the probability threshold, to obtain an answer character string corresponding to the target text line;
[0010] Based on the difference between the answer character string corresponding to the single-question image and the reference answer character string corresponding to the single-question image, correction information of the single-question image is generated.
[0011] Optionally, the performing text line recognition on the single-question image to obtain a first probability that each text line included in the single-question image is an answer text line includes:
[0012] The single-question image is input into a pre-trained text line recognition model to obtain a first recognition result output by the text line recognition model, wherein the first recognition result includes each text line included in the single-question image and a first probability that each text line is an answer text line.
[0013] Optionally, the method further comprises:
[0014] Acquire a first sample data set consisting of a plurality of sample single-question images, each sample single-question image including an answer area distribution label;
[0015] The preset area weight is determined based on the answer area distribution label included in each sample single-question image in the first sample data set.
[0016] Optionally, performing text recognition on the target text line whose second probability satisfies the probability threshold to obtain the answer character string corresponding to the target text line includes:
[0017] Inputting the target text line into a pre-trained text recognition model to obtain a second recognition result output by the text recognition model, wherein the second recognition result includes each predicted character corresponding to each text in the target text line and the probability corresponding to each predicted character;
[0018] Based on the probabilities corresponding to the predicted characters, an answer character string corresponding to the target text line is obtained.
[0019] Optionally, obtaining the answer character string corresponding to the target text line based on the probabilities corresponding to the predicted characters includes:
[0020] Based on the word frequency weight relationship, the probabilities corresponding to the predicted characters corresponding to the same text are corrected to obtain the target probabilities corresponding to the predicted characters corresponding to the text, and the texts in the text lines included in the same question share a word frequency weight relationship, and the word frequency weight relationship represents the weighted weights corresponding to the characters;
[0021] Get the predicted character with the highest target probability as the target character corresponding to the text;
[0022] Based on the target characters corresponding to the respective texts in the target text row, an answer character string corresponding to the target text row is obtained.
[0023] Optionally, the method further comprises:
[0024] Acquire a second sample data set consisting of a plurality of sample single-question images of the same question;
[0025] Based on the frequency of occurrence of each character in the second sample data set, a character frequency weight relationship corresponding to the question is determined.
[0026] Optionally, the pre-trained text recognition model is trained by the following steps:
[0027] Acquire multiple sample text lines, each sample text line corresponds to a real character string, each character in the real character string corresponds to a loss weight, and the loss weight corresponding to a character represents the possibility of the character being predicted as other characters;
[0028] Inputting the sample text line into the character recognition model to be trained to obtain a predicted character string output by the character recognition model to be trained;
[0029] Based on the difference between the predicted character string and the true character string, obtaining a first loss corresponding to each character;
[0030] Based on the loss weights, weighted summing is performed on the first losses of the corresponding characters to obtain a total loss;
[0031] Based on the total loss, the text recognition model to be trained is back-propagated to complete one training of the text recognition model to be trained. After multiple trainings, the text recognition model to be trained obtains the pre-trained text recognition model.
[0032] Optionally, obtaining a single-question image corresponding to the homework to be marked includes:
[0033] Performing feature point recognition on the image of the homework to be corrected to obtain first feature information;
[0034] Determining a target standard operation image based on a degree of matching between the second feature information of each candidate standard operation image and the first feature information;
[0035] Based on the first characteristic information and the second characteristic information corresponding to the target standard homework image, performing image correction on the image of the homework to be corrected to obtain a corrected image;
[0036] The corrected image is segmented into questions to obtain a single-question image corresponding to the homework to be graded.
[0037] In a second aspect, the present disclosure further provides a homework correction device, the device comprising:
[0038] An acquisition module, used to acquire a single-question image corresponding to the homework to be graded;
[0039] A text line recognition module, used to perform text line recognition on the single-question image to obtain a first probability that each text line included in the single-question image is an answer text line;
[0040] A correction module, configured to correct the first probability based on the regional distribution of each text line in the single-question image and a preset regional weight, so as to obtain a second probability that each text line is an answer text line, wherein the preset regional weight represents the probability of the answer text line being distributed in each region;
[0041] A text recognition module, used for performing text recognition on the target text line whose second probability meets the probability threshold, to obtain an answer character string corresponding to the target text line;
[0042] The correction module is used to generate correction information of the single question image based on the difference between the answer character string corresponding to the single question image and the reference answer character string corresponding to the single question image.
[0043] In a third aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method in the first aspect when executed by a processor.
[0044] In a fourth aspect, the present disclosure further provides an electronic device, including:
[0045] a memory having a computer program stored thereon;
[0046] A processor is used to execute the computer program in the memory to implement the steps of the method in the first aspect.
[0047] Through the above technical scheme, the single-question image corresponding to the homework to be corrected is first obtained, and then the text line recognition is performed on the single-question image to obtain the first probability that each text line included in the single-question image is an answer text line, and then based on the regional distribution of each text line in the single-question image and the preset regional weight, the first probability is corrected to obtain the second probability that each text line is an answer text line, and then the target text line whose second probability meets the probability threshold is subjected to text recognition to obtain the answer character string corresponding to the target text line, and finally based on the difference between the answer character string corresponding to the single-question image and the reference answer character string corresponding to the single-question image, the correction information of the single-question image can be generated. Since the preset regional weights represent the possibility of the distribution of answer text lines in various regions, after obtaining the first probability that each text line included in the single-question image is an answer text line, the first probability is corrected based on the regional distribution of each text line in the single-question image and the preset regional weights. This can take into account the actual distribution of the answer text lines, so that the recognition results of the text lines are no longer completely dependent on the results output by the text line recognition model, thereby reducing the impact of the insufficient recognition ability of the text line recognition model, improving the accuracy of answer text line recognition, and thereby improving the accuracy of subsequent homework grading.
[0048] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:
[0050] Figure 1 It is a flowchart of a homework correction method provided by an embodiment of the present disclosure.
[0051] Figure 2 It is a schematic diagram of a single-question image recognition result provided by an embodiment of the present disclosure.
[0052] Figure 3 It is a schematic diagram of rendering and displaying correction information provided by an embodiment of the present disclosure.
[0053] Figure 4 It is a schematic diagram of a partial result output of a text recognition model provided by an embodiment of the present disclosure.
[0054] Figure 5 It is a flowchart of another homework correction method provided by an embodiment of the present disclosure.
[0055] Figure 6 It is a structural schematic diagram of a homework correction device provided by an embodiment of the present disclosure.
[0056] Figure 7 It is a structural schematic diagram of another homework correction device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0057] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.
[0058] In the related art, when automatically grading homework, in order to achieve a better automatic grading effect, the reference answer is usually relatively standardized. For example, in the reference answer, the order between each problem-solving step and the order of each reference answer string in the problem-solving step are usually fixed. Therefore, by comparing the recognized answer string with the reference answer string of each step one by one, the correctness of the homework can be judged, thereby generating grading information. However, due to the insufficient recognition ability of the text recognition model in the related art, the content in the question stem may be recognized as the answer string in the problem-solving step, resulting in a deviation from the actual answer string, which in turn causes the problem of high grading error rate, and also makes the homework grading method based on artificial intelligence not yet widely popularized.
[0059] In order to solve the above problems, the inventors proposed the homework grading method, device, storage medium and electronic device provided by the present invention. First, a single-question image corresponding to the homework to be graded is obtained, and then text line recognition is performed on the single-question image to obtain a first probability that each text line included in the single-question image is an answer text line. Then, based on the regional distribution of each text line in the single-question image and the preset regional weight, the first probability is corrected to obtain a second probability that each text line is an answer text line. Then, text recognition is performed on the target text line whose second probability meets the probability threshold to obtain the answer character string corresponding to the target text line. Finally, based on the difference between the answer character string corresponding to the single-question image and the reference answer character string corresponding to the single-question image, the grading information of the single-question image can be generated. Since the preset regional weights represent the possibility of the distribution of answer text lines in various regions, after obtaining the first probability that each text line included in the single-question image is an answer text line, the first probability is corrected based on the regional distribution of each text line in the single-question image and the preset regional weights. This can take into account the actual distribution of the answer text lines, so that the recognition results of the text lines are no longer completely dependent on the results output by the text line recognition model, reducing the impact of the insufficient recognition ability of the text line recognition model, improving the accuracy of answer text line recognition, and thereby improving the accuracy of subsequent homework grading.
[0060] See also Figure 1 , Figure 1 This is a flowchart of a homework grading method according to an exemplary embodiment of the present disclosure, which is applied to an electronic device, such as a mobile terminal or a server. The mobile terminal includes: a mobile phone, a notebook, a tablet computer, a desktop computer, a smart TV, etc. The server includes: a local server and a cloud server. The homework grading method includes steps S110 to S150.
[0061] Specific:
[0062] S110, obtaining a single-question image corresponding to the homework to be corrected.
[0063] The single-question image can be understood as an image corresponding to each question in the homework to be graded.
[0064] It is understandable that the homework to be corrected can have different forms. Optionally, the homework to be corrected can have only one question, so there is only one single-question image corresponding to the homework to be corrected. Optionally, the homework to be corrected can have multiple questions, so there can be multiple single-question images corresponding to the homework to be corrected.
[0065] It should be noted that in the embodiments of the present disclosure, the processing process for any single-question image is similar. Therefore, for ease of understanding, the subsequent embodiments are described in detail using a single-question image as an example.
[0066] S120, performing text line recognition on the single-question image to obtain a first probability that each text line included in the single-question image is an answer text line.
[0067] In the disclosed embodiment, the text lines in a single-question image may include text lines corresponding to the question itself and text lines corresponding to the student's answer content, wherein the answer text lines may be understood as text lines corresponding to the student's answer content.
[0068] S130, based on the regional distribution of each text line in the single-question image and the preset regional weight, the first probability is corrected to obtain a second probability that each text line is an answer text line, and the preset regional weight represents the possibility of the answer text line being distributed in each region.
[0069] Among them, the regional distribution can be understood as the regional distribution of questions and the regional distribution of answers.
[0070] In the disclosed embodiment, considering that in actual answering, students usually do not answer in the question area under normal circumstances, therefore, there is a certain regularity in the area where the answer text line appears in the single-question image. For example, the probability of the answer text line appearing in the question area is low, and the probability of appearing in the answer area is high. Therefore, the answer area weight can be preset, and different weights can be set for the text line in the question area and the text line in the answer area. Thus, after obtaining the first probability that each text line is an answer text line, the first probability corresponding to each text line can be further modified according to the regional distribution of each text line and the preset regional weight, to obtain the second probability of each text line being an answer text line.
[0071] S140, performing text recognition on the target text line whose second probability meets the probability threshold, and obtaining an answer character string corresponding to the target text line.
[0072] The second probability threshold is the standard for considering a text line as an answer text line. If the second probability threshold corresponding to a text line is greater than or equal to the probability threshold, the text line can be determined as an answer text line. Since the answer text line corresponds to the answer content, the answer text line can be further used as the target text line for subsequent text recognition to obtain the corresponding answer character string.
[0073] It should be noted that in the embodiments of the present disclosure, the characters in the character string may be Chinese characters, pinyin, letters of other countries, Arabic numerals, various operation symbols, etc.
[0074] For example, see Figure 2 , assuming that after text recognition is performed on the single-question image 20, text line 1 is obtained, the corresponding first probability is 0.54, text line 2 is obtained, the corresponding first probability is 0.90, and text line 3 is obtained, the corresponding first probability is 0.49. And assuming that the corresponding area weight of the question area 21 is 0.3, the corresponding area weight of the answer area 22 is 1.2, and the probability threshold is set to 0.5, that is, text lines greater than or equal to 0.5 are determined as target text lines.
[0075] Well, before correction, since text line 1 is actually the question, it is still determined as the target text line for subsequent text recognition, so text line 1 is misdetected, and text line 3 is actually the answer text line, but it is not determined as the target text line for subsequent text recognition, so text line 3 is missed detection.
[0076] After correction, the second probability corresponding to text line 1 is 0.54*0.3=0.162, the second probability corresponding to text line 2 is 0.9*1.2=1.08, and the second probability corresponding to text line 3 is 0.49*1.2=0.588. If the probability threshold is still set to 0.5, text line 2 and text line 3 will be accurately identified as target text lines for subsequent text recognition, and text line 1 will no longer be identified as the target text line for subsequent text recognition. Therefore, the possibility of false detection and missed detection is reduced, and the accuracy of identifying the target text is improved.
[0077] S150, generating correction information of the single-question image based on the difference between the answer character string corresponding to the single-question image and the reference answer character string corresponding to the single-question image.
[0078] In the disclosed embodiment, after obtaining the answer string, the answer string can be compared with the reference answer string corresponding to the same question, so as to automatically generate the correction information of the single question image according to the comparison result.
[0079] In addition, taking into account the diversity of the math problem-solving process, in some embodiments, when comparing the difference between the answer string and the reference answer string, only the digital string after the equal sign "=" can be compared to see if they are the same. If they are the same, correction information representing the correct solution is generated, such as a check, that is, a check is placed in the single-question image. If they are different, correction information representing the incorrect solution is generated, such as a cross, that is, a cross is placed in the single-question image.
[0080] In addition, considering the existence of synonyms in Chinese and other question types, in some embodiments, when comparing the differences between the answer string and the reference answer string, a semantic design rule can be set. For example, the answer string "cannot complete this project" and the reference answer string "cannot complete this project" express the same meaning, and correction information representing the correct solution can be generated, such as a check mark, that is, a check mark in the single question image.
[0081] In addition, in order to make the correction information more detailed, in some implementations, the reference answer string also corresponds to an error analysis and knowledge focus points. At this time, when comparing the difference between the answer string and the reference answer string, if there is a difference between a certain string and the corresponding reference answer string, the corresponding error analysis and knowledge focus points can be output accordingly to prompt the students doing the homework.
[0082] For example, see Figure 3 , the answer string "32+29=60 (people)" is calculated incorrectly. At this time, the correction information "The calculation here is wrong, you need more practice to add two numbers!" can be output near the wrong answer text line.
[0083] In combination with the foregoing content, it can be known that the electronic device can be a mobile terminal or a server.
[0084] In some embodiments, when the electronic device is a mobile terminal, the student user can use the electronic device to take a photo of the homework he or she has completed, so that the mobile terminal can obtain the homework to be corrected. After the mobile terminal processes and generates the correction information of the single-question image, it can be rendered and displayed directly on the mobile terminal.
[0085] In other embodiments, when the electronic device is a server, the student user can first use the mobile terminal to take a photo of the homework he or she has completed, so that the mobile terminal can obtain the homework to be corrected, and then upload the homework to be corrected to the server through the mobile terminal, so that the server can obtain the homework to be corrected. After the server processes and generates the correction information of the single-question image, the data corresponding to the correction information can be sent to the mobile terminal, which then renders and displays it.
[0086] By adopting the above technical scheme, the single-question image corresponding to the homework to be corrected is first obtained, and then text line recognition is performed on the single-question image to obtain a first probability that each text line included in the single-question image is an answer text line. Then, based on the regional distribution of each text line in the single-question image and the preset regional weight, the first probability is corrected to obtain a second probability that each text line is an answer text line. Then, text recognition is performed on the target text line whose second probability meets the probability threshold to obtain the answer character string corresponding to the target text line. Finally, based on the difference between the answer character string corresponding to the single-question image and the reference answer character string corresponding to the single-question image, the correction information of the single-question image can be generated. Since the preset regional weights represent the possibility of the distribution of answer text lines in various regions, after obtaining the first probability that each text line included in the single-question image is an answer text line, the first probability is corrected based on the regional distribution of each text line in the single-question image and the preset regional weights. This can take into account the actual distribution of the answer text lines, so that the recognition results of the text lines are no longer completely dependent on the results output by the text line recognition model, reducing the impact of the insufficient recognition ability of the text line recognition model, improving the accuracy of answer text line recognition, and thereby improving the accuracy of subsequent homework grading.
[0087] In some implementations, text line recognition may be performed on a single-question image using a pre-trained neural network model. In this case, performing text line recognition on the single-question image and obtaining a first probability that each text line included in the single-question image is an answer text line may include the following steps:
[0088] The single-question image is input into a pre-trained text line recognition model to obtain a first recognition result output by the text line recognition model. The first recognition result includes each text line included in the single-question image and a first probability that each text line is an answer text line.
[0089] In the disclosed embodiment, the pre-trained text line recognition model may be a network model such as DB-Net, CTPN, EAST, Seglink, Textbox, Textbox++ or PSEnet.
[0090] For example, continue to refer to Figure 2 , in Figure 2 After the single-question image 20 shown is input into the pre-trained text line recognition model, text line 1 can be obtained, the corresponding first probability is 0.54, text line 2, the corresponding first probability is 0.90, and text line 3, the corresponding first probability is 0.49. It should be noted that Figure 2 Only some of the text lines and the corresponding probabilities are shown.
[0091] In some implementations, the method of the embodiment of the present disclosure further includes the step of determining a preset area weight, that is, the method of the embodiment of the present disclosure further includes the following steps:
[0092] A first sample data set consisting of a plurality of sample single-question images is obtained, each sample single-question image including an answer area distribution label; and a preset area weight is determined based on the answer area distribution label included in each sample single-question image in the first sample data set.
[0093] Among them, the answer area distribution label can be understood as the answer area marked in the single question image.
[0094] In the disclosed embodiment, the homework of the student's historical answers can be obtained, and then the answer area distribution label can be marked for each single question image included in each homework, so that the first sample data set can be obtained. When the number of sample single question images in the first sample data set is large enough, the handwritten answer area can be counted to generate a statistical result similar to a heat map. The heat map can represent the area with a higher probability of the answer text line appearing, that is, the preset area weight is obtained. Therefore, in the subsequent application stage, the preset area weight can be used to correct the first probability and improve the confidence of the answer text line appearing in the high probability area.
[0095] The preset area weights are obtained based on the single-question images marked with answer area distribution labels, which can better reflect the students' answer areas during homework, and then reflect the actual distribution of answer text lines, so as to improve the accuracy of subsequent determination of target text lines.
[0096] In some implementations, the target text line may be recognized by a pre-trained neural network model. In this case, the target text line whose second probability satisfies the probability threshold may be recognized to obtain the answer character string corresponding to the target text line, which may include the following steps:
[0097] The target text line is input into a pre-trained text recognition model to obtain a second recognition result output by the text recognition model. The second recognition result includes the predicted characters corresponding to the texts in the target text line and the probabilities corresponding to the predicted characters. Based on the probabilities corresponding to the predicted characters, the answer character string corresponding to the target text line is obtained.
[0098] In the disclosed embodiment, the pre-trained text line recognition model may be a CRNN or Seq2Seq+Attention scheme, or an optical character recognition (OCR) model based on deep learning or a model based on traditional OCR scheme.
[0099] In the disclosed embodiment, when the target text line is input into the pre-trained text recognition model, the predicted characters corresponding to the texts in the target text line and the probabilities corresponding to the predicted characters can be obtained. Thus, in one embodiment, the predicted character with the highest probability corresponding to each text can be further determined as the predicted character corresponding to the text, and then the answer characters corresponding to each text in the target text line can be finally obtained, and the answer characters can be combined in order to obtain the answer character string.
[0100] In some embodiments, the pre-trained text recognition model is trained by the following steps:
[0101] A plurality of sample text lines are obtained, each of which corresponds to a real character string. Each character in the real character string has a corresponding loss weight, and the loss weight corresponding to a character represents the possibility that the character is predicted to be other characters; the sample text lines are input into the text recognition model to be trained to obtain the predicted character string output by the text recognition model to be trained; based on the difference between the predicted character string and the real character string, the first loss corresponding to each character is obtained; based on the loss weight, the first loss of the corresponding character is weightedly summed to obtain the total loss; based on the total loss, the text recognition model to be trained is back-propagated to complete one training of the text recognition model to be trained, and after multiple trainings of the text recognition model to be trained, a pre-trained text recognition model is obtained.
[0102] In the disclosed embodiment, each character in the sample text line corresponds to a loss weight, and the loss weight corresponding to each character can be determined according to the probability of each character being predicted as other characters.
[0103] Exemplarily, within a limited character set, it is possible to count which similar characters each character is more likely to be recognized as (for example, 1 is more likely to be recognized as 7 and 2, so when 1 is predicted to be 7 or 2, a relatively large loss weight is set for character 1, such as 1.2, but it is almost never recognized as 6 and 9, so when 1 is predicted to be 6 or 9, a relatively small loss weight is set for character 1, such as 1.05), so that the loss weight can be set according to the possibility of being predicted as other characters.
[0104] Therefore, in the training stage, when the true annotation is known, the corresponding loss weight is determined according to the model prediction result. Thus, after obtaining the first loss value corresponding to each character based on the difference between the predicted string and the true string, the loss weight is added on the basis of the first loss value, and the weighted summation is performed to obtain the total loss. Then, based on the total loss, the text recognition model to be trained is back-propagated to complete one training of the text recognition model to be trained. Therefore, after the text recognition model to be trained has been trained for multiple times, a pre-trained text recognition model can be obtained.
[0105] For example, assuming that the true character string corresponding to a sample text line is "4+8=12", after it is input into the text recognition model to be trained, the output predicted character string is "4+3=12". Since character 8 is predicted to be 3, in addition to the first loss value corresponding to character 8, the table shows that the misidentification of 8 as 3 corresponds to a loss weight, for example, it can be 1.2. At this time, the total loss of the character string can be the weighted sum of the losses of each character, that is, 0+0+0.3*1.2+0+0+0=0.36, and then based on the total loss of 0.36, the text recognition model to be trained is back-propagated to complete one training of the text recognition model to be trained.
[0106] By adding corresponding loss weights to each character, it is equivalent to establishing a semantic error correction process, so that the model can adjust the model parameters according to the possibility of predicting other characters, thereby improving the model recognition accuracy.
[0107] In addition, considering that the prediction performance of the pre-trained text recognition model may also be insufficient, so that the prediction result may not be completely accurate, therefore, in order to improve the accuracy of character prediction, in some embodiments, based on the probability corresponding to each predicted character, obtaining the answer string corresponding to the target text line may include the following steps:
[0108] Based on the word frequency weight relationship, the probabilities corresponding to each predicted character corresponding to the same text are corrected to obtain the target probabilities corresponding to each predicted character corresponding to the text. The texts in each text line included in the same question share a word frequency weight relationship, and the word frequency weight relationship represents the weighted weights corresponding to each character; the predicted character with the largest target probability is obtained as the target character corresponding to the text; based on the target characters corresponding to each text in the target text line, the answer string corresponding to the target text line is obtained.
[0109] Among them, the word frequency weight relationship can characterize the likelihood of each answered character appearing in a single question. In the embodiments of the present disclosure, for each single question image, a corresponding word frequency weight relationship can be established, and then based on the word frequency weight relationship, the probabilities corresponding to each predicted character in the single question image are corrected to obtain the target probabilities corresponding to each predicted character in the text.
[0110] Exemplarily, assume that the target text line is "3 + 6 = 9 (pieces)". Inputting this target text line into a pre-trained character recognition model, the second recognition result output by the character recognition model can be obtained. The second recognition result can be output in one-hot form. Refer to Figure 4 , Figure 4 shows the one-hot output result corresponding to the 6th text. Among them, the original predicted probability of being predicted as the character "pieces" is 0.43, and the original predicted probability of being predicted as the character "not" is 0.46. If the 6th text is determined according to the result output by the character recognition model, the character corresponding to the maximum probability 0.46 is taken, that is, it is misrecognized as "not", and the word frequency weight relationship is used to weight each character. Assume that the word frequency weight relationship corresponding to the character "pieces" is 0.85, and the word frequency weight relationship corresponding to the character "not" is 0.64. Then the probability of being predicted as the character "pieces" is 0.43 * 0.85 = 0.36, and the probability of being predicted as the character "not" is 0.46 * 0.64 = 0.29. Thus, the 6th text is correctly recognized as "pieces".
[0111] In some embodiments, the word frequency weight relationship can be obtained according to the second sample data set. That is to say, the method of the embodiments of the present disclosure further includes the following steps:
[0112] Obtain a second sample data set composed of multiple sample single question images of the same question; based on the frequencies of each character appearing in the second sample data set, determine the word frequency weight relationship corresponding to the question.
[0113] In the embodiments of the present disclosure, for each question, the answering results of multiple students in history can be counted, that is, a second sample data set composed of multiple sample single question images is obtained. By counting the frequencies of each character written by the students when answering from the second sample data set, the word frequency weight relationship corresponding to each question can be obtained.
[0114] It should be noted that one question corresponds to one second sample data set. Thus, the same word frequency weight relationship is shared by all texts in each text line included in the same question.
[0115] By counting the frequency of occurrence of each character in the second sample data set to determine the word frequency weight relationship, the students' actual answering habits can be taken into account, thereby reflecting the actual writing situation of the characters, so as to improve the accuracy of subsequent determination of the target characters.
[0116] In addition, in some implementations, obtaining a single-question image corresponding to the homework to be corrected may include the following steps:
[0117] Feature point recognition is performed on the image of the homework to be corrected to obtain first feature information; based on the matching degree between the second feature information of each candidate standard homework image and the first feature information, a target standard homework image is determined; based on the first feature information and the second feature information corresponding to the target standard homework image, image correction is performed on the image of the homework to be corrected to obtain a corrected image; and question segmentation is performed on the corrected image to obtain a single question image corresponding to the homework to be corrected.
[0118] In the disclosed embodiment, feature point recognition can be performed on the image of the homework to be corrected to obtain first feature information. In addition, second feature information of each candidate standard homework image can be obtained, and then the first feature information is matched with each second feature information, and the standard homework image with the highest matching degree is selected to be determined as the target standard homework image. Then, the homography matrix is estimated for the first feature information and the second feature information using the RANSAC algorithm, and the image of the homework to be corrected is transformed into the standard homework image coordinate system using this matrix. According to the boundary position of the standard homework image, the image background of the homework to be corrected after transformation is removed to obtain a corrected image.
[0119] For example, when a homework to be corrected is placed on a sofa or a table and a photo is uploaded, the background such as the sofa or the table can be removed through image correction methods to retain the homework part.
[0120] After obtaining the rectified image, the question segmentation can be performed on the rectified image to obtain a single question image corresponding to the homework to be corrected.
[0121] In some embodiments, a pre-trained question number detection model can be used to detect question numbers in the corrected image to obtain the position of each question number, and then the corrected image is segmented according to the position of the question number with a preset number of pixels (for example, 3 pixels) above the question number as the dividing line, thereby obtaining a single question image corresponding to the homework to be corrected.
[0122] In some implementations, the feature points may be SURF feature points.
[0123] In some implementations, the pre-trained question number detection model may be a model such as YOLO, SSD, Mask-RCNN, Cascade-RCNN, or RetinaNet.
[0124] Combine the following Figure 5 , a complete process is used to introduce the homework grading method of the embodiment of the present disclosure.
[0125] First, students use their mobile phones to take photos of their completed test papers, so that the electronic device can obtain the image of the homework to be corrected, thus completing the image input process.
[0126] Next, the SURF feature points of the input image of the homework to be corrected are calculated to obtain the first SURF feature information. At the same time, the second SURF feature information of each candidate standard job image is obtained. Then, the first SURF feature information is matched with each second SURF feature information to obtain the target standard job image. Then, the second SURF feature information and the first SURF feature information of the target standard job image are used to correct the image of the homework to be corrected to obtain an image with the background removed.
[0127] Next, the image with the background removed is subjected to question number detection and question segmentation to obtain individual question images.
[0128] Next, the single-question image is processed in sequence through text line detection, probability correction, character recognition and word frequency weighting to obtain the answer character string.
[0129] Next, the single question character string is compared with the reference answer character string corresponding to the corresponding single question image to generate the correction information of the single question image, thereby completing the correction of the homework. Finally, it can be rendered and displayed on the electronic device.
[0130] It should be noted that the above process description is relatively simple, and the detailed description of the relevant processes can refer to the aforementioned embodiments, which will not be repeated here.
[0131] See also Figure 6 An exemplary embodiment of the present disclosure further provides a homework correction device 600, which is applied to an electronic device. The device 600 includes:
[0132] An acquisition module 610 is used to acquire a single-question image corresponding to the homework to be graded;
[0133] A text line recognition module 620 is used to perform text line recognition on the single-question image to obtain a first probability that each text line included in the single-question image is an answer text line;
[0134] A correction module 630, configured to correct the first probability based on the regional distribution of each text line in the single-question image and a preset regional weight, so as to obtain a second probability that each text line is an answer text line, wherein the preset regional weight represents the probability of the answer text line being distributed in each region;
[0135] A text recognition module 640 is used to perform text recognition on the target text line whose second probability meets the probability threshold, and obtain an answer character string corresponding to the target text line;
[0136] The correction module 650 is used to generate correction information of the single question image based on the difference between the answer string corresponding to the single question image and the reference answer string corresponding to the single question image.
[0137] In some embodiments, the text line recognition module 620 is also used to input the single-question image into a pre-trained text line recognition model to obtain a first recognition result output by the text line recognition model, wherein the first recognition result includes each text line included in the single-question image and a first probability that each text line is an answer text line.
[0138] In some embodiments, the apparatus 600 further comprises:
[0139] A preset area weight determination module is used to obtain a first sample data set consisting of multiple sample single-question images, each sample single-question image including an answer area distribution label; based on the answer area distribution labels included in each sample single-question image in the first sample data set, determine the preset area weight.
[0140] In some implementations, the text recognition module 640 includes:
[0141] The model recognition submodule is used to input the target text line into a pre-trained text recognition model to obtain a second recognition result output by the text recognition model, wherein the second recognition result includes each predicted character corresponding to each text in the target text line and the probability corresponding to each predicted character.
[0142] The answer character string determination submodule is used to obtain the answer character string corresponding to the target text line based on the probabilities corresponding to the various predicted characters.
[0143] In some embodiments, the answer string determination submodule is also used to correct the probabilities corresponding to the various predicted characters corresponding to the same text based on the word frequency weight relationship, and obtain the target probabilities corresponding to the various predicted characters corresponding to the text. The texts in the various text lines included in the same question share a word frequency weight relationship, and the word frequency weight relationship represents the weighted weights corresponding to the various characters; obtain the predicted character with the largest target probability as the target character corresponding to the text; and obtain the answer string corresponding to the target text line based on the target characters corresponding to the various texts in the target text line.
[0144] In some embodiments, the apparatus 600 further comprises:
[0145] The character frequency weight relationship determination module is used to obtain a second sample data set consisting of multiple sample single-question images of the same question; based on the frequency of occurrence of each character in the second sample data set, determine the character frequency weight relationship corresponding to the question.
[0146] In some embodiments, the apparatus 600 further comprises:
[0147] A text recognition model training module is used to obtain multiple sample text lines, each sample text line corresponds to a real character string, each character in the real character string corresponds to a loss weight, and the loss weight corresponding to a character represents the possibility of the character being predicted as other characters; the sample text line is input into the text recognition model to be trained to obtain the predicted character string output by the text recognition model to be trained; based on the difference between the predicted character string and the real character string, the first loss corresponding to each character is obtained; based on the loss weight, the first loss of the corresponding character is weighted summed to obtain the total loss; based on the total loss, the text recognition model to be trained is back-propagated to complete one training of the text recognition model to be trained, and after multiple trainings, the text recognition model to be trained obtains the pre-trained text recognition model.
[0148] In some embodiments, the acquisition module 610 is also used to perform feature point recognition on the image of the homework to be corrected to obtain first feature information; determine the target standard homework image based on the matching degree between the second feature information of each candidate standard homework image and the first feature information; perform image correction on the image of the homework to be corrected based on the first feature information and the second feature information corresponding to the target standard homework image to obtain a corrected image; perform question segmentation on the corrected image to obtain a single question image corresponding to the homework to be corrected.
[0149] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0150] Figure 7 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 7 As shown, the electronic device 700 may include: a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.
[0151] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned homework correction method. The memory 702 is used to store various types of data to support the operation of the electronic device 700. For example, these data may include instructions for any application or method used to operate on the electronic device 700, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 705 may include: Wi-Fi module, Bluetooth module, NFC module, etc.
[0152] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned homework grading method.
[0153] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned homework correction method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions can be executed by the processor 701 of the electronic device 700 to complete the above-mentioned homework correction method.
[0154] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned homework marking method when executed by the programmable device.
[0155] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.
[0156] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.
[0157] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.
Claims
1. A homework marking method, characterized in that: include: Get the single-question image corresponding to the homework to be graded; Performing text line recognition on the single-question image to obtain a first probability that each text line included in the single-question image is an answer text line; Acquire a first sample data set consisting of a plurality of sample single-question images, each sample single-question image including an answer area distribution label; Determining a preset area weight based on the answer area distribution label included in each sample single-question image in the first sample data set; Based on the regional distribution of each text line in the single-question image and the preset regional weight, the first probability is corrected to obtain a second probability that each text line is an answer text line, wherein the preset regional weight represents the probability of the answer text line being distributed in each region; Performing text recognition on the target text line whose second probability meets the probability threshold, to obtain an answer character string corresponding to the target text line; generating correction information of the single-question image based on a difference between the answer character string corresponding to the single-question image and the reference answer character string corresponding to the single-question image; The performing text line recognition on the single-question image to obtain a first probability that each text line included in the single-question image is an answer text line includes: The single-question image is input into a pre-trained text line recognition model to obtain a first recognition result output by the text line recognition model, wherein the first recognition result includes each text line included in the single-question image and a first probability that each text line is an answer text line.
2. The homework marking method according to claim 1, characterized in that: The performing text recognition on the target text line whose second probability meets the probability threshold to obtain the answer character string corresponding to the target text line includes: Inputting the target text line into a pre-trained text recognition model to obtain a second recognition result output by the text recognition model, wherein the second recognition result includes each predicted character corresponding to each text in the target text line and the probability corresponding to each predicted character; Based on the probabilities corresponding to the predicted characters, an answer character string corresponding to the target text line is obtained.
3. The homework marking method according to claim 2, characterized in that: The step of obtaining the answer character string corresponding to the target text line based on the probabilities corresponding to the predicted characters includes: Based on the word frequency weight relationship, the probabilities corresponding to the predicted characters corresponding to the same text are corrected to obtain the target probabilities corresponding to the predicted characters corresponding to the text, and the texts in the text lines included in the same question share a word frequency weight relationship, and the word frequency weight relationship represents the weighted weights corresponding to the characters; Get the predicted character with the highest target probability as the target character corresponding to the text; Based on the target characters corresponding to the respective texts in the target text row, an answer character string corresponding to the target text row is obtained.
4. The homework marking method according to claim 3, characterized in that: The method further comprises: Acquire a second sample data set consisting of a plurality of sample single-question images of the same question; Based on the frequency of occurrence of each character in the second sample data set, a character frequency weight relationship corresponding to the question is determined.
5. The homework marking method according to claim 2, characterized in that: The pre-trained text recognition model is trained by the following steps: Acquire multiple sample text lines, each sample text line corresponds to a real character string, each character in the real character string corresponds to a loss weight, and the loss weight corresponding to a character represents the possibility of the character being predicted as other characters; Inputting the sample text line into the character recognition model to be trained to obtain a predicted character string output by the character recognition model to be trained; Based on the difference between the predicted character string and the true character string, obtaining a first loss corresponding to each character; Based on the loss weights, weighted summing is performed on the first losses of the corresponding characters to obtain a total loss; Based on the total loss, the text recognition model to be trained is back-propagated to complete one training of the text recognition model to be trained. After multiple trainings, the text recognition model to be trained obtains the pre-trained text recognition model.
6. The homework marking method according to claim 1, characterized in that: The step of obtaining a single-question image corresponding to the homework to be graded includes: Performing feature point recognition on the image of the homework to be corrected to obtain first feature information; Determining a target standard operation image based on a degree of matching between the second feature information of each candidate standard operation image and the first feature information; Based on the first characteristic information and the second characteristic information corresponding to the target standard homework image, performing image correction on the image of the homework to be corrected to obtain a corrected image; The corrected image is segmented into questions to obtain a single-question image corresponding to the homework to be graded.
7. A homework correction device, characterized in that: include: An acquisition module, used to acquire a single-question image corresponding to the homework to be graded; A text line recognition module, used to perform text line recognition on the single-question image to obtain a first probability that each text line included in the single-question image is an answer text line; A correction module, configured to correct the first probability based on the regional distribution of each text line in the single-question image and a preset regional weight, so as to obtain a second probability that each text line is an answer text line, wherein the preset regional weight represents the probability of the answer text line being distributed in each region; A text recognition module, used for performing text recognition on the target text line whose second probability meets the probability threshold, to obtain an answer character string corresponding to the target text line; A correction module, used for generating correction information of the single-question image based on the difference between the answer character string corresponding to the single-question image and the reference answer character string corresponding to the single-question image; A preset area weight determination module is used to obtain a first sample data set consisting of a plurality of sample single-question images, each sample single-question image including an answer area distribution label; based on the answer area distribution label included in each sample single-question image in the first sample data set, determine the preset area weight; The text line recognition module is also used to input the single-question image into a pre-trained text line recognition model to obtain a first recognition result output by the text line recognition model. The first recognition result includes each text line included in the single-question image and a first probability that each text line is an answer text line.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.
9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 6.
