Homework correction method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a homework correction method and device, equipment and a storage medium, and the method comprises the steps: recognizing a pinyin region and a writing region of each question of a pinyin-written Chinese character question type in a test paper image through a preset model when correcting the question of the pinyin-written Chinese character question type, and detecting whether the writing region is written or not, and the homework correcting speed and the homework correcting efficiency are greatly improved. In addition, by predicting the at least one target word corresponding to the at least one predicted pinyin combination, the embodiment of the invention can adapt to scenes with various correct answers, so that the application scene is improved, and the situation that when students write words with the same pronunciation but meaningless words, mistaken batch is changed to be correct is prevented. According to the embodiment of the invention, at least one predicted character corresponding to each character is identified, so that the condition of correction errors caused by the fact that the characters are identified as characters with similar shapes can be reduced, the correction accuracy is further improved, and the misjudgment rate is reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and in particular to a method, apparatus, device, and storage medium for grading homework. Background Art

[0002] Writing Chinese characters based on pinyin is a common question type in Chinese language teaching. This question type requires students to write the corresponding Chinese character based on pinyin, which can effectively assess students' pronunciation and literacy. This question type often appears in teaching aids, quizzes, and exams. Due to its high frequency, the number of questions of this type is large when the number of students is large, which puts a lot of pressure on teachers to correct them. Although existing homework grading systems can also grade questions based on pinyin, the grading efficiency is low and the error rate is high.

[0003] In summary, the existing homework grading system has the technical problems of low grading efficiency and high error rate when grading questions that require writing Chinese characters by looking at pinyin. Summary of the Invention

[0004] The embodiments of the present invention provide a homework grading method, apparatus, device and storage medium, which solve the technical problems in the prior art that the homework grading system has low grading efficiency and high error rate when grading questions that require writing Chinese characters by looking at pinyin.

[0005] In a first aspect, an embodiment of the present invention provides a method for grading homework, comprising:

[0006] receiving a test paper image, wherein the test paper image includes test paper content;

[0007] Inputting the test paper image into a preset model, so that the preset model detects the pinyin area and the writing area of ​​each question in the "Look at the Pinyin and Write Chinese Characters" question type from the test paper image, and detects whether there is writing in the writing area;

[0008] For a target topic written in the writing area, identifying the pinyin area of ​​the target topic to obtain at least one predicted pinyin combination corresponding to the target topic;

[0009] Determining at least one target word corresponding to each of the predicted pinyin combinations;

[0010] Identifying the writing area of ​​the target question and obtaining predicted characters corresponding to each character written in the writing area, wherein the number of the predicted characters is at least one;

[0011] Whether each target question is answered correctly is determined based on the target words corresponding to each target question and the corresponding predicted text.

[0012] The embodiment of the present invention provides a method for correcting homework. When correcting questions of the type of writing Chinese characters based on pinyin, the embodiment of the present invention can use a preset model to identify the pinyin area and writing area of ​​each question of the type of writing Chinese characters based on pinyin in the test paper image, and at the same time detect whether there is writing in the writing area. While not needing to identify the text of the entire test paper image, it is also not necessary to determine the pinyin print and handwritten text of the same question based on the positional relationship between the pinyin print and the handwritten text. At the same time, it can skip questions that have no answers, greatly improving the speed and efficiency of correcting homework. In addition, the embodiment of the present invention can adapt to scenarios where there are multiple correct answers by predicting at least one target word corresponding to at least one predicted pinyin combination, improving the applicable scenarios, and preventing students from mistakenly correcting words that have the same pronunciation but are meaningless as correct. Secondly, the embodiment of the present invention can reduce the situation of incorrect correction due to recognition of similar characters by identifying at least one predicted text corresponding to each character, further improving the accuracy of correction and reducing the error rate.

[0013] Preferably, the preset model includes a feature extraction module, a feature integration module and a result output module;

[0014] The step of inputting the test paper image into a preset model so that the preset model detects the pinyin area and the writing area of ​​each question in the question type of writing Chinese characters by looking at the pinyin from the test paper image, and detecting whether there is writing in the writing area, includes:

[0015] Inputting the test paper image into the feature extraction module so that the feature extraction module extracts image features of multiple scales in the test paper image;

[0016] Inputting the image features into the feature integration module so that the feature integration module is used to integrate the image features to generate a feature matrix;

[0017] The feature matrix is ​​input into the result output module so that the result output module processes the feature matrix and outputs the target information of the test paper image. The target information includes the probability that each point in the test paper image is the question, the center position offset of the question, the size and offset angle of the question, the center position offset of the pinyin area of ​​the question, the size and offset angle of the pinyin area of ​​the question, the center position offset of the writing area of ​​the question, the size and offset angle of the writing area of ​​the question, and the answer probability of the writing area.

[0018] The preset model provided by the embodiment of the present invention, compared with the target detection algorithm or text detection algorithm in the prior art, does not need to judge whether it belongs to a question based on the positional relationship between the pinyin print and the handwritten text. The preset model can directly predict the relationship between the question, the pinyin area and the writing area (because it is output at the same time). When the training data is diverse enough, the preset model can automatically adapt to various special scenarios such as rotation and print offset. Secondly, compared with the traditional algorithm, the preset model can also output the predicted offset angle, which is more conducive to recognition for rotation scenarios, and the preset model can also predict whether there is an answer in the writing area. For questions without writing, the recognition step can be omitted, thereby improving the efficiency of correction. In addition, the preset module in the embodiment of the present invention can also predict the position of the pinyin area and the position of the writing area, which provides important priors for subsequent recognition. When the preset module is subsequently trained, the model for recognizing the pinyin area and the model for recognizing the writing area can be trained separately, so that the model for recognizing the pinyin area does not need to support Chinese characters, outputs fewer categories, is faster and more accurate, and the model for recognizing the writing area can be trained with a higher proportion of writing data, while removing the category of pinyin characters, which is faster and more accurate.

[0019] Preferably, the identifying of the pinyin region of the target question to obtain at least one predicted pinyin combination corresponding to the target question includes:

[0020] Identify the pinyin region of the target question and obtain a first probability that each identified first character belongs to a different letter;

[0021] At least one predicted pinyin combination corresponding to the target topic is determined based on the first probability prediction.

[0022] Preferably, the step of determining the predicted pinyin combination corresponding to the target question based on the first probability prediction includes:

[0023] Determine a first letter with the greatest first probability corresponding to each of the first characters, and determine a first predicted letter corresponding to each of the first characters based on the first letters, and determine a first predicted pinyin combination based on the first predicted letters;

[0024] updating the first predicted pinyin combination according to the first probability to obtain at least one second predicted pinyin combination;

[0025] According to the first predicted pinyin combination and the second predicted pinyin combination, a predicted pinyin combination corresponding to the target question is determined.

[0026] The embodiment of the present invention predicts at least one predicted pinyin combination corresponding to each target question, thereby avoiding the situation where the printed pinyin in the image is incorrectly recognized when only one pinyin combination is predicted, which will lead to incorrect question grading. While improving the fault tolerance rate, it can also further improve the accuracy of subsequent grading.

[0027] Preferably, the updating of the first predicted pinyin combination according to the first probability to obtain at least one second predicted pinyin combination includes:

[0028] Confirming the current probability value of each first character in the pinyin area in order from largest to smallest in a round, until the number of the second predicted pinyin combinations reaches a quantity threshold; the pending probability value is a first probability other than a reference probability value corresponding to the corresponding first character, the reference probability value being the maximum probability value among the first probabilities corresponding to each first character;

[0029] Perform difference processing on the current probability values ​​confirmed in each round, and each difference processing confirms the difference between the reference probability value corresponding to the same first character and the current probability value;

[0030] Sort the differences of all first characters in the same round from largest to smallest, determine a target difference with the smallest value, and determine a target first character corresponding to the target difference;

[0031] Determine a target letter corresponding to the current probability value of the target first character, and replace the letter corresponding to the target first character in the first predicted pinyin combination with the target letter to obtain a second predicted pinyin combination.

[0032] After determining the letter corresponding to each first character, the embodiment of the present invention determines the first character in the first pinyin combination that has no high probability of being accurately predicted by calculating the difference between the first probability value corresponding to each first character and the values ​​of other first probabilities, and replaces the letter corresponding to the first character in the first predicted pinyin combination with other letters corresponding to the first character, thereby reducing the possibility of recognition errors due to similar glyphs and improving the accuracy of recognizing printed letters.

[0033] Preferably, the identifying the writing area of ​​the target question to obtain predicted text corresponding to each character written in the writing area includes:

[0034] Identifying the writing area of ​​the target question and detecting each second character within the writing area;

[0035] Each of the second characters is recognized to obtain at least one predicted word corresponding to each of the second characters.

[0036] Preferably, determining whether each target question is answered correctly based on the target words and the corresponding predicted text corresponding to each target question includes:

[0037] Combining the predicted words corresponding to each target question to obtain a plurality of predicted words;

[0038] Determining whether any of the predicted words matches any of the target words;

[0039] If they match, it is determined that the answer to the target question is correct.

[0040] Preferably, when identifying the writing area of ​​the target question, a second probability corresponding to each predicted word is also obtained;

[0041] The predicted words corresponding to each target question are combined to obtain a plurality of predicted words, including:

[0042] Combining the predicted words corresponding to each target question in pairs to obtain multiple candidate words;

[0043] Calculating the sum of the second probabilities of the two predicted characters in each candidate word;

[0044] Predicted words are selected from the candidate words according to the sum of the second probabilities.

[0045] In the embodiment of the present invention, multiple predicted words are determined in the candidate words based on the sum of the second probabilities of two predicted characters in the candidate words. This can avoid the situation where only one predicted word is predicted and handwritten characters are incorrectly recognized, which will lead to incorrect question grading. While improving the fault tolerance rate, it can also further improve the accuracy of subsequent grading.

[0046] In a second aspect, an embodiment of the present invention provides a homework grading device, comprising:

[0047] An image receiving module is used to receive an image of a test paper, wherein the image of the test paper includes the test paper content;

[0048] an area detection module, configured to input the test paper image into a preset model, so that the preset model detects the pinyin area and the writing area of ​​each question in the "Look at the Pinyin and Write Chinese Characters" question type from the test paper image, and detects whether there is writing in the writing area;

[0049] a pinyin recognition module for recognizing the pinyin area of ​​a target topic written in the writing area, and obtaining at least one predicted pinyin combination corresponding to the target topic;

[0050] a target word determination module, configured to determine at least one target word corresponding to each of the predicted pinyin combinations;

[0051] a character recognition module, configured to recognize the writing area of ​​the target question and obtain at least one predicted character corresponding to each character written in the writing area;

[0052] The question correction module is used to determine whether each target question is answered correctly based on the target words and corresponding predicted words corresponding to each target question.

[0053] In a third aspect, an embodiment of the present invention provides a homework grading device, the homework grading device including a processor and a memory;

[0054] The memory is used to store a computer program and transmit the computer program to the processor;

[0055] The processor is configured to execute the homework grading method as described in the first aspect according to instructions in the computer program.

[0056] In a fourth aspect, an embodiment of the present invention provides a storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to execute a homework grading method as described in the first aspect.

[0057] As described above, the embodiments of the present invention provide a method, apparatus, device and storage medium for grading homework. When grading questions of the type of writing Chinese characters by looking at pinyin, the embodiments of the present invention can use a preset model to identify the pinyin area and writing area of ​​each question of the type of writing Chinese characters by looking at pinyin in the test paper image, and at the same time detect whether there is writing in the writing area. While not needing to recognize the text of the entire test paper image, it is also not necessary to determine the pinyin print and handwritten text of the same question based on the positional relationship between the pinyin print and the handwritten text. At the same time, it can skip questions that have no answers, greatly improving the speed and efficiency of grading homework. In addition, the embodiments of the present invention can adapt to scenarios where there are multiple correct answers by predicting at least one target word corresponding to at least one predicted pinyin combination, improve the applicable scenarios, and prevent students from mistakenly grading words that have the same pronunciation but are meaningless as correct. Secondly, the embodiments of the present invention can reduce the situation of grading errors due to recognition of similar characters by identifying at least one predicted text corresponding to each character, further improving the accuracy of grading and reducing the error rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a diagram of the question type of writing Chinese characters based on pinyin.

[0059] Figure 2 A flowchart of a method for grading homework provided by an embodiment of the present invention.

[0060] Figure 3 A schematic diagram of the pinyin area and writing area in a question provided in an embodiment of the present invention.

[0061] Figure 4 A schematic diagram of the principles of a homework grading method provided by an embodiment of the present invention.

[0062] Figure 5 A schematic flow chart of another homework grading method provided by an embodiment of the present invention.

[0063] Figure 6 A schematic diagram of the structure of a preset model provided by an embodiment of the present invention.

[0064] Figure 7 A schematic diagram of an offset angle provided by an embodiment of the present invention.

[0065] Figure 8 A schematic diagram of the principles of another homework grading method provided by an embodiment of the present invention.

[0066] Figure 9 A schematic structural diagram of a homework grading device provided in an embodiment of the present invention.

[0067] Figure 10 A schematic structural diagram of a homework grading device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The following description and accompanying drawings sufficiently illustrate specific embodiments of the present application to enable those skilled in the art to practice them. The examples represent only possible variations. Unless expressly required, individual components and functions are optional, and the order of operations may vary. Portions and features of some embodiments may be included in or replace portions and features of other embodiments. The scope of the embodiments of the present application includes the entire scope of the claims, as well as all available equivalents of the claims. Herein, each embodiment may be referred to individually or collectively by the term "invention," which is merely for convenience and is not intended to automatically limit the scope of the application to any single invention or inventive concept if more than one invention is in fact disclosed. Herein, relational terms such as first and second, etc., are used solely to distinguish one entity or operation from another, and do not require or imply any actual relationship or order between these entities or operations. Furthermore, the terms "comprise," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or apparatus comprising a set of elements includes not only those elements, but also other elements not explicitly listed. The various embodiments herein are described in a progressive manner, with each embodiment focusing on the differences from the other embodiments. Reference can be made to the common and similar parts between the various embodiments. For structures, products, etc. disclosed in the embodiments, the description is relatively simple because they correspond to the parts disclosed in the embodiments. For relevant parts, refer to the method description.

[0069] Writing Chinese characters by looking at pinyin is a common question type in Chinese language teaching. Figure 1 As shown in the figure, due to the high frequency of this type of question, when there are a large number of students, the number of questions of this type is large, and the teacher's correction pressure is large. An automatic correction system is urgently needed to relieve the teacher's correction pressure. The automatic correction system in the existing technology can automatically detect the question of writing Chinese characters by looking at pinyin on the page and automatically judge whether it is correct or not. The specific steps are as follows:

[0070] (1) Receive the test paper image input by the user, perform handwritten and printed text detection on the test paper image, and obtain the coordinates of the handwritten text line and the printed text line.

[0071] (2) According to the coordinates of the handwritten text line and the printed text line, a text line image is cut out from the test paper image, and the text line image is input into the handwriting recognition model and the printing recognition model respectively to obtain the handwritten text and the printed text in the text line image respectively.

[0072] (3) Filter out the printed texts that only include pinyin, and determine the pinyin printed texts and handwritten texts of the same question based on the positional relationship between the pinyin printed texts and the handwritten texts.

[0073] (4) Determine whether the handwritten text of each question is correct.

[0074] In the process of correcting the questions of writing Chinese characters according to pinyin by the existing automatic correction system, the following technical problems exist:

[0075] (1) It is necessary to recognize the text of the entire test paper image, but ultimately only the text in the questions of writing Chinese characters according to pinyin is needed, which wastes computing power and reduces the correction speed at the same time.

[0076] (2) When judging whether it belongs to the same question according to the positional relationship between the pinyin printed text and the handwritten text, if there is rotation or layout deviation, it is easy to misjudge, resulting in missed detection of questions.

[0077] (3) In the process of correcting each question, whether the pinyin printed text and the handwritten text match can only be judged according to a single handwritten text and a single pinyin printed text, lacking semantic judgment. For example, writing "history" as "lì shǐ" will also be judged as correct.

[0078] (4) It is easy to misidentify similar Chinese characters. For example, the "person" written by the student may be misidentified as "enter", resulting in misjudgment.

[0079] Based on this, the embodiments of the present invention provide a homework correction method. When correcting the questions of writing Chinese characters according to pinyin in the embodiments of the present invention, it is possible to use a preset model to identify the pinyin area and writing area of each question of the writing Chinese characters according to pinyin type in the test paper image, and at the same time detect whether there is writing in the writing area, greatly improving the correction speed and correction efficiency of the homework. In addition, in the embodiments of the present invention, by predicting at least one target word corresponding to at least one predicted pinyin combination, it can adapt to scenarios with multiple correct answers. Secondly, in the embodiments of the present invention, by identifying at least one predicted character corresponding to each character, it is possible to reduce the situation of correction errors caused by misidentifying similar Chinese characters, further improving the accuracy of correction.

[0080] As Figure 2 shown, Figure 2 is a flowchart of a homework correction method provided by an embodiment of the present invention. The homework correction method provided by the embodiment of the present invention can be executed by a homework correction device. The homework correction device can be implemented in a software and / or hardware manner. The homework correction device can be composed of two or more physical entities, or can be composed of one physical entity. For example, the homework correction device can be a computer, a host computer, a tablet and other devices. The method includes the following steps:

[0081] Step 101, Receive a test paper image, where the test paper image includes the test paper content of the test paper.

[0082] In this embodiment, the homework grading device first needs to receive a test paper image. The test paper image is an image of the test paper content. The method for generating the test paper image can be configured according to actual needs. For example, the user can use a camera to capture the test paper to obtain the test paper image, or use a scanner to scan the test paper to obtain the test paper image. This embodiment does not specifically limit the method for generating the test paper image. In addition, when receiving the test paper image in this embodiment, the user can capture the test paper image and then transmit the test paper image to the homework grading device in real time. Alternatively, the user can capture the test paper image of all students and then transmit all the test paper images to the homework grading device. The specific process can be configured according to actual needs.

[0083] Step 102: Input the test paper image into the preset model so that the preset model detects the pinyin area and writing area of ​​each question in the "Look at the pinyin and write Chinese characters" question type from the test paper image, and detects whether there is writing in the writing area.

[0084] After receiving the test paper image, the homework grading device needs to further input the test paper image into a preset model, where the preset model is a pre-set network model. In this embodiment, the user can set the network structure of the preset model according to the functions that the preset model needs to implement, and use the training set to train the preset model. When the output error of the preset model is within the preset threshold, the trained preset model is obtained. After receiving the test paper image, the preset model will process and analyze the test paper image, and detect the pinyin area and writing area of ​​each question in the test paper image in the "Look at the Pinyin and Write Chinese Characters" question type. The pinyin area is the area in the test paper image that displays the printed pinyin, and the math area is the area in the test paper image that displays the student's handwritten text. Figure 3 As shown, Figure 3 Here, S1 is the pinyin area for each question, and S2 is the handwriting area for each question. The pre-set model also detects whether there is handwriting in the handwriting area of ​​each question. This allows the model to determine whether the student has answered the question based on whether there is handwriting in the handwriting area. For unanswered questions, the model can then directly mark them as incorrect during subsequent grading, saving time and improving the efficiency of grading.

[0085] Step 103: For a target topic written in a writing area, identify the pinyin area of ​​the target topic to obtain at least one predicted pinyin combination corresponding to the target topic.

[0086] After obtaining the output of the preset model, the homework grading device can determine the target questions with written content in the writing area of ​​the "Look at Pinyin and Write Chinese Characters" question type, i.e., the questions with answers. The homework grading device can then use a text recognition method to identify the Pinyin area of ​​each target question in the test paper image, thereby obtaining at least one predicted Pinyin combination corresponding to each target question, wherein the predicted Pinyin combination is a predicted Pinyin combination, and the Pinyin combination refers to the combination of Pinyin corresponding to each character to be filled in the Pinyin area. It can be understood that for the "Look at Pinyin and Write Chinese Characters" question type, Pinyin corresponding to at least two characters is generally printed, so that students can fill in the corresponding characters. In addition, the predicted Pinyin combination obtained in this embodiment is at least one. For example, the homework grading device can determine several predicted Pinyin combinations with the highest probability. Exemplarily, the homework grading device can use an existing character recognition algorithm to identify the characters in the Pinyin area of ​​the target question, thereby obtaining the probability that each character belongs to a different letter. Then, based on the probability that each character belongs to a different letter, several predicted Pinyin combinations with higher probability can be determined. The specific text recognition algorithm can be used in the existing technology and will not be described in detail in this embodiment.

[0087] Step 104: Determine at least one target word corresponding to each predicted pinyin combination.

[0088] After obtaining the predicted pinyin combination corresponding to the target question, the homework grading device can determine the target word corresponding to the predicted pinyin combination of the target question by querying a dictionary or other means. The target word is the word corresponding to the predicted pinyin combination. For example, when the predicted pinyin combination is "quán lì", the target words corresponding to the predicted pinyin combination are "quanli", "quanli", "quanli", etc. In addition, if the predicted pinyin combination does not exist in the dictionary, it can be considered a recognition error and can be directly discarded.

[0089] Step 105: Identify the writing area of ​​the target question and obtain predicted characters corresponding to each character written in the writing area. The number of predicted characters is at least one.

[0090] In this embodiment, when the homework grading device recognizes the pinyin area of each target question, it also needs to simultaneously recognize the writing area of each target question, so as to obtain the predicted text corresponding to each character written by the user in the writing area, where the predicted text refers to the predicted text. Similarly, in this embodiment, existing character detection algorithms can be used to detect each character in the writing area, and then each character can be recognized using an existing classification model, so as to obtain the possible text corresponding to each character written by the student. The existing character detection algorithms can be character detection algorithms such as YOLO or Mask R-CNN, or can be publicly available segmentation algorithms such as U-NET or Deeplab, or can be text detection algorithms such as pan++ or deepsolo, etc.; and the classification model can be a classification model such as Resnet50 or vgg16, etc. In this embodiment, the character detection algorithm and the classification model are not specifically limited and can be selected according to actual needs. In addition, because similar characters are very easy to be misrecognized. For example, for the character "人" in Chinese, if a student's writing is not rigorous, it is very easy to be misrecognized as "入". Therefore, at least one predicted text needs to be recognized to reduce the situation of misrecognition.

[0091] Step 106: Determine whether each target question is answered correctly according to the target word corresponding to each target question and the corresponding predicted text.

[0092] After determining the target word corresponding to each target question and the corresponding predicted text, the homework grading device can grade each target question according to the target word and the predicted text of each target question to determine whether the target question is answered correctly. For example, the homework grading device can combine the predicted texts corresponding to each target question to obtain multiple predicted words. Then, the predicted words are matched with the target word of the target question to determine whether there is a situation where any target word is the same as any predicted word. If so, it can be determined that the target question is answered correctly; otherwise, it is determined that the target question is answered incorrectly. The specific process is as Figure 4 shown.

[0093] As described above, an embodiment of the present invention provides a method for correcting homework. When correcting questions of the type of writing Chinese characters based on pinyin, the embodiment of the present invention can use a preset model to identify the pinyin area and writing area of ​​each question of the type of writing Chinese characters based on pinyin in the test paper image, and at the same time detect whether there is writing in the writing area. While not needing to identify the text of the entire test paper image, it is also not necessary to determine the pinyin print and handwritten text of the same question based on the positional relationship between the pinyin print and the handwritten text. At the same time, it can skip questions that have no answers, greatly improving the speed and efficiency of correcting homework. In addition, the embodiment of the present invention can adapt to scenarios where there are multiple correct answers by predicting at least one target word corresponding to at least one predicted pinyin combination, improving the applicable scenarios, and preventing students from mistakenly correcting words that have the same pronunciation but are meaningless as correct. Secondly, the embodiment of the present invention can reduce the situation of incorrect correction due to recognition of similar characters by identifying at least one predicted text corresponding to each character, further improving the accuracy of correction and reducing the error rate.

[0094] The embodiment of the present invention also provides another homework correction method, such as Figure 5 As shown, Figure 5 A flowchart of another homework grading method provided by an embodiment of the present invention is shown below. Figure 5 The homework grading method shown is a specific implementation of the above-mentioned homework grading method. The homework grading method provided in this embodiment includes:

[0095] Step 201: Receive a test paper image, where the test paper image includes the test paper content.

[0096] Step 202: Input the test paper image into the preset model so that the preset model detects the pinyin area and writing area of ​​each question in the "Look at the pinyin and write Chinese characters" question type from the test paper image, and detects whether there is writing in the writing area.

[0097] Based on the above embodiment, the preset model includes a feature extraction module, a feature integration module and a result output module.

[0098] The preset model provided in this embodiment is different from the traditional YOLO model algorithm or Mask R-CNN and other target detection algorithms in the prior art. The output of the traditional target detection algorithm generally includes the following dimensions: whether there is a target, the category of the target, the position and size of the target, etc. However, for the recognition scenario of the question type of looking at pinyin and writing Chinese characters, the traditional target detection algorithm or text detection algorithm is not applicable. The traditional target detection algorithm or text detection algorithm can only detect the written Chinese characters and the printed pinyin text. It is necessary to match the association between the pinyin text and the corresponding written Chinese characters (that is, it is necessary to determine whether a certain pinyin text and a certain written Chinese character belong to the same question). For some special scenarios such as test paper rotation or irregular layout, it is difficult to match. In order to solve this problem, the embodiment of the present invention provides a preset model. The network structure of the preset model is as follows: Figure 6 As shown, it includes a feature extraction module, a feature integration module and a result output module.

[0099] In step 202, the test paper image is input into a preset model, so that the preset model detects the pinyin area and the writing area of ​​each question in the "Look at the Pinyin and Write Chinese Characters" question type from the test paper image, and detects whether there is writing in the writing area, including:

[0100] Step 2021: Input the test paper image into the feature extraction module so that the feature extraction module extracts image features of multiple scales in the test paper image.

[0101] In this embodiment, after receiving the test paper image, the homework grading device first needs to input the test paper image into the feature extraction module of the preset model, wherein the feature extraction module is the backbone network of the preset model, which is used to extract features. The backbone network can adopt various backbone network structures disclosed in the prior art, such as Resnet50 or VGG16, etc. The specific network structure can refer to the prior art and will not be described in detail in this embodiment. In this embodiment, after the test paper image is input into the feature extraction module, the feature extraction module will output image features of multiple sizes, wherein the image features mainly include color features, texture features, shape features and spatial relationship features of the image. Assuming that the size of the input test paper image is H*W, the image features of multiple sizes output by the feature extraction module may be image features of several sizes such as (H / 4)*(W / 4), (H / 8)*(W / 8) and (H / 16)*(W / 16), which is related to the image size set by the user and the backbone network selected.

[0102] Step 2022: Input the image features into the feature integration module, so that the feature integration module is used to integrate the image features to generate a feature matrix.

[0103] After obtaining the image features of multiple sizes output by the feature extraction module, the homework grading device needs to further input the image features into the feature integration module, wherein the feature integration module is the network neck of the network model, which is used to integrate the features. In this embodiment, the feature integration module can adopt the existing public network neck structure such as FPN or FPEM. When the size of the test paper image input into the preset model is H*W, the feature integration module is used to scale the size of all image features output by the feature extraction module to the same size as the test paper image, and then process the image features after size scaling to obtain a feature matrix. The feature matrix generally refers to the eigenvalue. The processing process of the image features after size scaling is related to the structure of the network neck selected by the user. For example, the processing process can be addition, etc., which will not be repeated in this embodiment.

[0104] Step 2023: Input the feature matrix into the result output module so that the result output module processes the feature matrix and outputs the target information of the test paper image. The target information includes the probability that each point in the test paper image is a question, the center position offset of the question, the size and offset angle of the question, the center position offset of the pinyin area of ​​the question, the size and offset angle of the pinyin area of ​​the question, the center position offset of the writing area of ​​the question, the size and offset angle of the writing area of ​​the question, and the probability of answering the writing area.

[0105] After obtaining the feature matrix, the homework grading device needs to input the feature matrix into the result output module. In this embodiment, the result output module is the detection head in the field of neural networks. The result output module is a module for further extracting features and outputting feature extraction results of a specific number of channels. The result output module in this embodiment includes three layers of results: convolution layer, relu and convolution layer. The convolution layer is composed of several convolution units. The parameters of each convolution unit are optimized by the back propagation algorithm. The purpose of the convolution operation is to extract different features of the input. The first convolution layer may only be able to extract some low-level features such as edges, lines and corners. More layers of network structures can iteratively extract more complex features from low-level features. It should be noted that the last convolution layer in this embodiment is used to output feature extraction results of 17 channels, that is, output image information of 17 dimensions. Relu is a linear rectification function, also known as a rectified linear unit. It is an activation function of neurons commonly used in artificial neural networks. It usually refers to a nonlinear function represented by a ramp function and its variants.

[0106] In this embodiment, the result output module outputs 17 dimensions of image information for each pixel in the test paper image, which can be represented by an output matrix. The 17 dimensions of image information include the probability that each point in the test paper image is a question, the center position offset of the question, the size and offset angle of the question, the center position offset of the pinyin area of ​​the question, the size and offset angle of the pinyin area of ​​the question, the center position offset of the handwriting area of ​​the question, the size and offset angle of the handwriting area of ​​the question, and the probability of answering the handwriting area.

[0107] The center position offset refers to the position offset of the center; the offset angle refers to the rotation angle. The offset angle can be the angle of offset relative to the coordinate axis of the reference coordinate system on the test paper, such as the angle of offset relative to the x-axis of the image coordinate system, etc. Figure 7 As shown, Figure 7 The θ in is the angle of offset. Exemplarily, the center position offset of the question is used as an example for explanation. Assuming that the probability of the question type of the question at the coordinate position (3, 6) in the test paper is higher than the preset probability according to the output matrix, it is considered that there is a question frame corresponding to the question, wherein the question frame is used to indicate the total area occupied by the pinyin area and the writing area of ​​the question. In addition, the center position offset of the question is determined to be △x1 and △y1 according to the output matrix, then the center position of the question is (3+△x1, 6+△y1), and then the specific position of the question can be determined based on the size and offset angle of the question. The process of determining the specific position of the pinyin area and the specific position of the writing area is similar to the process of determining the specific position of the question, and will not be repeated in this embodiment. When outputting the specific position of the pinyin area, the specific position of the writing area and the specific position of the question, the preset model can simultaneously determine the matching relationship between the pinyin area, the writing area and the question.

[0108] It can be understood that the process of training the preset module in this embodiment is similar to the existing training method. Both methods obtain training images and mark the corresponding 17-dimensional image information on the training images, and then input them into the preset model for training until the error output by the preset model is within the preset threshold.

[0109] As mentioned above, the preset model provided by the embodiment of the present invention, compared with the target detection algorithm or text detection algorithm in the prior art, does not need to judge whether it belongs to a question based on the positional relationship between the pinyin print and the handwritten text. The preset model can directly predict the relationship between the question, the pinyin area and the writing area (because it is output at the same time). When the training data is diverse enough, the preset model can automatically adapt to various special scenarios such as rotation and print offset. Secondly, compared with the traditional algorithm, the preset model can also output the predicted offset angle, which is more conducive to recognition for rotation scenarios, and the preset model can also predict whether there is an answer in the writing area. For questions without writing, the recognition step can be omitted, thereby improving the efficiency of correction. In addition, the preset module in the embodiment of the present invention can also predict the position of the pinyin area and the position of the writing area, which provides important priors for subsequent recognition. When the preset module is subsequently trained, the model for recognizing the pinyin area and the model for recognizing the writing area can be trained separately, so that the model for recognizing the pinyin area does not need to support Chinese characters, outputs fewer categories, is faster and more accurate, and the model for recognizing the writing area can be trained with a higher proportion of writing data, while removing the category of pinyin characters, which is faster and more accurate.

[0110] Step 203: For a target question with writing in the writing area, the pinyin area of ​​the target question is identified to obtain a first probability that each identified first character belongs to a different letter.

[0111] In this embodiment, after determining the pinyin area, the homework grading device can first perform character recognition on the pinyin area, wherein character recognition refers to the process of identifying characters. By performing character recognition on the pinyin area of ​​the target question, the probability that each first printed character belongs to a different letter can be obtained, that is, the first probability. Here, a character refers to a glyph-like unit or symbol, including letters, numbers, operators, punctuation marks and other symbols, and the first character is the character recognized in the pinyin area of ​​the target question. In this embodiment, the public text recognition model PARSeq model can be used for character recognition. After character recognition, the PARSeq model will output a probability matrix, which includes the first probability that the recognized first character belongs to a different letter.

[0112] Step 204: Determine at least one predicted pinyin combination corresponding to the target topic based on the first probability prediction.

[0113] After identifying the first probability that each first character in the pinyin region of the target question belongs to a different letter, at least one predicted pinyin combination corresponding to each target question can be predicted based on the first probability. Based on the above embodiment, predicting and determining at least one predicted pinyin combination corresponding to the target question based on the first probability in step 204 includes:

[0114] Step 2041: determine the first letter with the highest first probability corresponding to each first character, and determine the first predicted letter corresponding to each first character based on the first letter, and determine the first predicted pinyin combination based on the first predicted letter.

[0115] In this embodiment, after determining the first probability that each first character in the pinyin area of ​​each target question belongs to a different letter, it is necessary to determine the letter with the largest first probability corresponding to each first character, that is, the first letter. After determining the first letter, it can be combined according to the first letter corresponding to each first character, and the resulting pinyin combination is the first predicted pinyin combination. Exemplarily, as shown in Table 1, Table 1 is the first probability that two first characters belong to different letters, the first letter corresponding to the first first character is y, and the first letter corresponding to the second second character is ā, the first letter corresponding to the third first character is l, and the first letter corresponding to the fourth first character is ì, then the first predicted pinyin combination is yālì.

[0116]

[0117]

[0118] Table 1

[0119] Step 2042: Update the first predicted pinyin combination according to the first probability to obtain at least one second predicted pinyin combination.

[0120] After obtaining the first predicted pinyin combination, the letters in the first predicted pinyin combination can be updated based on the first probability that each first character belongs to a different letter. The updated first predicted combination is the second predicted pinyin combination. In this embodiment, the number of second predicted pinyin combinations is at least one. In one embodiment, the homework grading device can determine that the first predicted pinyin combination does not have a high probability of accurately predicting a letter, and replace the letter with another letter with a closer first probability, thereby completing the update of the first predicted pinyin combination.

[0121] Based on the above embodiment, in step 2042, the first predicted pinyin combination is updated according to the first probability to obtain at least one second predicted pinyin combination, including:

[0122] Step 20421, confirm the current probability value of the pending probability value of each first character in the pinyin area in rounds from large to small until the number of second predicted pinyin combinations reaches the quantity threshold; the pending probability value is the first probability other than the reference probability value corresponding to the corresponding first character, and the reference probability value is the maximum probability value among the first probabilities corresponding to each first character.

[0123] In this embodiment, when updating the first predicted pinyin combination, it is first necessary to confirm the current probability value of the pending probability value of each first character in the pinyin area in rounds from large to small, where the pending probability value is the first probability other than the reference probability value corresponding to the corresponding first character, and the reference probability value is the maximum probability value among the first probabilities corresponding to each first character, that is, the pending probability value is the other first probabilities of the corresponding first character except the largest first probability.

[0124] Step 20422: Perform difference processing on the current probability values ​​confirmed in each round. Each difference processing determines the difference between the reference probability value corresponding to the same first character and the current probability value.

[0125] Afterwards, for the current probability value determined for each first character in each round, difference processing is performed on the current probability value, that is, the reference probability value corresponding to each first character is subtracted from the current probability value to obtain the difference value of the current round.

[0126] Step 20423: Sort the differences of all first characters in the same round from largest to smallest, determine the target difference with the smallest value, and determine the target first character corresponding to the target difference;

[0127] After obtaining the difference value for the current round, sort all the differences between the first characters in the same round from largest to smallest to obtain the sorted result. Then, the target difference value with the smallest value can be obtained based on the sorted result, and the target first character corresponding to the target difference value can be determined.

[0128] Step 20424: Determine the target letter corresponding to the current probability value of the target first character, and replace the letter corresponding to the target first character in the first predicted pinyin combination with the target letter to obtain a second predicted pinyin combination.

[0129] Finally, the target letter corresponding to the current probability value of the target first character is determined, and the target letter is substituted for the letter corresponding to the target first character in the first predicted pinyin combination to obtain a second predicted pinyin combination. Afterwards, a new round can be entered and the above process can be executed again until the number of second predicted pinyin combinations reaches a quantity threshold, where the quantity threshold is a pre-set number. In this embodiment, the number of quantity thresholds can be set according to actual needs, for example, the quantity threshold can be set to 2 or 3, etc., which is specifically limited in this embodiment.

[0130] Specifically, taking Table 1 as an example, in the first round, the first probability with the largest value for each first character is subtracted from the second largest first probability to obtain the difference. Then, among all the first characters, the first character with the smallest difference value is selected, and the letter with the second largest first probability corresponding to the first character replaces the letter corresponding to the first character in the first predicted pinyin combination, completing the update of the first predicted pinyin combination, and obtaining the first second predicted pinyin combination. For Table 1, the second predicted pinyin obtained in the first round is yálì. In the second round, the first probability with the largest value for each first character is subtracted from the third largest first probability to obtain the difference. Then, among all the first characters, the first character with the smallest difference value is selected, and the letter with the third largest first probability corresponding to the first character replaces the letter corresponding to the first character in the first predicted pinyin combination, completing the update of the first predicted pinyin combination, and obtaining the second second predicted pinyin combination. For Table 1, the second predicted pinyin obtained in the second round is yàlì.

[0131] As described above, after determining the letter corresponding to each first character, the embodiment of the present invention determines the first character in the first pinyin combination that has no high probability of being accurately predicted by calculating the difference between the numerical value of the first probability with the largest value corresponding to each first character and the numerical values ​​of other first probabilities, and replaces the letter corresponding to the first character in the first predicted pinyin combination with other letters corresponding to the first character, thereby reducing the possibility of recognition errors due to similar glyphs and improving the accuracy of recognizing printed letters.

[0132] Step 2043: Determine the predicted pinyin combination corresponding to the target question based on the first predicted pinyin combination and the second predicted pinyin combination.

[0133] After obtaining the first predicted pinyin combination and the second predicted pinyin combination, the first predicted pinyin combination and the second predicted pinyin combination can be determined as the predicted pinyin combination corresponding to the target question.

[0134] The above is the specific process for determining the predicted pinyin combination corresponding to each target question. By predicting at least one predicted pinyin combination corresponding to each target question, the present embodiment can avoid the situation where only one pinyin combination is predicted and the printed pinyin in the image is incorrectly recognized, which can lead to incorrect grading of the question. While improving the error tolerance rate, it can also further improve the accuracy of subsequent grading.

[0135] Step 205: Determine at least one target word corresponding to each predicted pinyin combination.

[0136] After determining the predicted phonetic combination corresponding to each target question, at least one target word corresponding to each predicted phonetic combination can be determined. In this embodiment, at least one target word corresponding to the predicted phonetic combination can be determined by querying a dictionary. For predicted phonetic combinations that do not have corresponding target words in the dictionary, they can be directly discarded. By querying the dictionary, it is possible to prevent students from mistakenly correcting words that have the same pronunciation but are meaningless. In addition, by querying the dictionary, it is also possible to determine whether the recognition of printed phonetic characters is correct, thereby improving the accuracy of the correction.

[0137] Step 206: Identify the writing area of ​​the target question and detect each second character in the writing area.

[0138] While identifying the pinyin area of ​​each target question, the homework grading device also needs to identify the writing area of ​​each question. After identifying the writing area using a character detection algorithm, each second character in the writing area can be identified.

[0139] Step 207: Recognize each second character to obtain at least one predicted word corresponding to each second character.

[0140] After identifying each second character within the writing area, the homework grading device may use the classification model to recognize each second character, thereby obtaining at least one predicted character corresponding to each second character, where the predicted character refers to a predicted character. Specifically, the classification model may obtain a second probability that each second character belongs to a different character when recognizing each second character. The homework grading device may then select three characters with the highest second probabilities as the predicted characters corresponding to the second character.

[0141] Step 208: Combine the predicted words corresponding to each target question to obtain multiple predicted words.

[0142] After obtaining the predicted words for each second character corresponding to each target question, the homework grading device can combine the predicted words for each second character corresponding to each target question to obtain multiple predicted words.

[0143] On the basis of the above embodiment, when the writing area of ​​the target question is recognized, the probability that each second character belongs to each predicted word, ie, the second probability, is also obtained.

[0144] In step 208, the predicted words corresponding to each target question are combined to obtain a plurality of predicted words, including:

[0145] Step 2081: Combine the predicted words corresponding to each target question in pairs to obtain multiple candidate words.

[0146] In this embodiment, when combining the predicted texts corresponding to the target questions, the predicted texts corresponding to each target question can be combined pairwise to obtain candidate words, that is, candidate words. For example, in the target questions, the predicted texts corresponding to the first second character are: "person", "enter", and "eight", and the predicted texts corresponding to the second character are "class", "committee", and "en". After combining the predicted texts corresponding to the two second characters, there are a total of 9 candidate words, and these 9 words are the candidate words.

[0147] Step 2082: Calculate the sum of the second probabilities of the two predicted texts in each candidate word.

[0148] Step 2083: Screen out the predicted words from the candidate words according to the sum of the second probabilities.

[0149] After obtaining the candidate words, the homework grading device can further calculate the sum of the second probabilities corresponding to the two predicted texts in the candidate words, and then sort them from largest to smallest according to the sum of the second probabilities, and take the candidate words with the sum in the top few as the predicted words. It can be understood that the number of candidate words can be set according to actual needs. For example, the top three candidate words with the sum in the sorting can be taken as the predicted words.

[0150] The above is the specific process of determining the predicted words in this embodiment. In the embodiment of the present invention, by determining multiple predicted words in the candidate words according to the sum of the second probabilities of the two predicted texts in the candidate words, it can avoid the situation that when only predicting one predicted word, if the handwritten characters are misrecognized, the whole question grading will be incorrect. While improving the error tolerance rate, it can also further improve the accuracy of subsequent grading.

[0151] Step 209: Determine whether any predicted word matches any target word.

[0152] Step 210: If they match, determine that the answer to the target question is correct.

[0153] After obtaining the predicted words corresponding to the target questions, it is further possible to determine whether any predicted word matches any target word, that is, whether any predicted word is the same as any target word. If they match, it can be determined that the answer to the target question is correct, otherwise it is determined that the answer to the target question is incorrect. The overall process is as Figure 8As shown in the figure. In this embodiment, when any predicted word matches any target word, it is determined that the target question is answered correctly, so that it can adapt to various scenarios of correct answers. In addition, because similar-looking characters are easily misrecognized. For example, for the character "人", due to the careless writing of students, it is very easy to be misrecognized as "入". However, in the actual marking process, the requirements are usually relaxed. The purpose of taking multiple predicted words is equivalent to relaxing the marking requirements. If the user needs to raise the marking requirements, only need to set the number of predicted words to one, so as to realize that by adjusting the number of predicted words, the marking requirements can be flexibly adjusted.

[0154] As described above, the embodiment of the present invention provides a homework marking method. When marking the questions of the type of writing Chinese characters according to pinyin in the embodiment of the present invention, it can use a preset model to identify the pinyin area and writing area of each question of the type of writing Chinese characters according to pinyin in the test paper image, and at the same time detect whether there is writing in the writing area. Without the need to recognize the text of the entire test paper image, nor to determine the pinyin printed text and handwritten text of the same question according to the positional relationship between the pinyin printed text and the handwritten text, and at the same time, the questions without answers can be skipped, which greatly improves the marking speed and efficiency of the homework. In addition, in the embodiment of the present invention, by predicting at least one predicted pinyin combination corresponding to each target question, it can avoid the situation that when only predicting one pinyin combination, if the pinyin printed in the image is misrecognized, the marking of the whole question will be incorrect. While improving the error tolerance rate, it can also further improve the accuracy of subsequent marking. Secondly, in the embodiment of the present invention, by determining multiple predicted words, it can adapt to scenarios with multiple correct answers, and reduce the situation of misrecognizing the written text as similar-looking characters, resulting in incorrect marking, further improving the marking accuracy and reducing the misjudgment rate.

[0155] The embodiment of the present invention also provides a homework marking device, as Figure 9 shown. Figure 9 As shown in the figure, it is a structural schematic diagram of a homework marking device provided by the embodiment of the present invention. As Figure 9 shown, the homework marking device provided by the embodiment of the present invention includes:

[0156] An image receiving module 301, configured to receive a test paper image, and the test paper image includes the test paper content of the test paper;

[0157] A region detection module 302, configured to input the test paper image into a preset model, so that the preset model detects the pinyin area and writing area of each question in the type of writing Chinese characters according to pinyin from the test paper image, and detect whether there is writing in the writing area;

[0158] A pinyin recognition module 303, configured to, for a target question with writing in the writing area, recognize the pinyin area of the target question to obtain at least one predicted pinyin combination corresponding to the target question;

[0159] A target word determination module 304 is configured to determine at least one target word corresponding to each predicted pinyin combination;

[0160] A character recognition module 305 is configured to recognize the writing area of ​​the target question and obtain at least one predicted character corresponding to each character written in the writing area;

[0161] The question correction module 306 is used to determine whether each target question is answered correctly based on the target words corresponding to each target question and the corresponding predicted text.

[0162] Based on the above embodiment, the preset model includes a feature extraction module, a feature integration module and a result output module;

[0163] Input the test paper image into the preset model, so that the preset model detects the pinyin area and writing area of ​​each question in the question type of "reading pinyin and writing Chinese characters" from the test paper image, and detects whether there is writing in the writing area, including:

[0164] Inputting the test paper image into the feature extraction module so that the feature extraction module extracts image features of multiple scales in the test paper image;

[0165] Inputting the image features into the feature integration module so that the feature integration module is used to integrate the image features to generate a feature matrix;

[0166] The feature matrix is ​​input into the result output module, so that the result output module processes the feature matrix and outputs the target information of the test paper image. The target information includes the probability that each point in the test paper image is a question, the center position offset of the question, the size and offset angle of the question, the center position offset of the pinyin area of ​​the question, the size and offset angle of the pinyin area of ​​the question, the center position offset of the writing area of ​​the question, the size and offset angle of the writing area of ​​the question, and the probability of answering the writing area.

[0167] Based on the above embodiment, the pinyin recognition module 303 includes:

[0168] The pinyin recognition submodule is used to recognize the pinyin region of the target question and obtain a first probability that each recognized first character belongs to a different letter;

[0169] The combination prediction submodule is used to determine at least one predicted pinyin combination corresponding to the target question based on the first probability prediction.

[0170] Based on the above embodiment, the combined prediction submodule includes:

[0171] A first combination determining unit is configured to determine a first letter with the greatest first probability corresponding to each first character, and determine a first predicted letter corresponding to each first character based on the first letter, and determine a first predicted pinyin combination based on the first predicted letter;

[0172] A second combination determining unit is configured to update the first predicted pinyin combination according to the first probability to obtain at least one second predicted pinyin combination;

[0173] The prediction combination determining unit is used to determine the predicted pinyin combination corresponding to the target question based on the first predicted pinyin combination and the second predicted pinyin combination.

[0174] Based on the above embodiment, the second combination determining unit includes:

[0175] The current probability value determination subunit is used to confirm the current probability value of the pending probability value of each first character in the pinyin area in rounds from large to small until the number of second predicted pinyin combinations reaches the quantity threshold; the pending probability value is the first probability other than the reference probability value corresponding to the corresponding first character, and the reference probability value is the maximum probability value among the first probabilities corresponding to each first character.

[0176] The difference calculation subunit is used to perform difference processing on the current probability value confirmed in each round, and each difference processing confirms the difference between the reference probability value corresponding to the same first character and the current probability value.

[0177] The target character determination unit is used to sort the differences of all first characters in the same round from large to small, determine the target difference with the smallest value, and determine the target first character corresponding to the target difference.

[0178] The letter replacement subunit is used to determine the target letter corresponding to the current probability value of the target first character, and replace the letter corresponding to the target first character in the first predicted pinyin combination with the target letter to obtain a second predicted pinyin combination.

[0179] Based on the above embodiment, the text recognition module 305 includes:

[0180] A character recognition submodule is used to recognize the writing area of ​​the target question and detect every second character in the writing area;

[0181] The character prediction submodule is used to recognize each second character and obtain at least one predicted character corresponding to each second character.

[0182] Based on the above embodiment, the question correction module 306 includes:

[0183] A word combination submodule is used to combine the predicted words corresponding to each target question to obtain multiple predicted words;

[0184] The word matching submodule is used to determine whether any predicted word matches any target word;

[0185] The correction judgment submodule is used to determine whether the answer to the target question is correct if there is a match.

[0186] Based on the above embodiment, when the writing area of ​​the target question is identified, a second probability corresponding to each predicted word is also obtained;

[0187] The text combination submodule includes:

[0188] A word combination unit is used to combine the predicted words corresponding to each target question in pairs to obtain multiple candidate words;

[0189] a probability calculation unit, configured to calculate the sum of the second probabilities of the two predicted characters in each candidate word;

[0190] The word screening unit is used to screen out predicted words from the candidate words according to the sum of the second probabilities.

[0191] The homework grading apparatus provided in the embodiment of the present invention is included in the homework grading device and can be used to execute the homework grading method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0192] It is worth noting that in the embodiment of the above-mentioned homework correction device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the various functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0193] This embodiment also provides a homework correction device, such as Figure 10 As shown, Figure 10 This is a structural diagram of a homework grading device provided by an embodiment of the present invention. The homework grading device 40 includes a processor 400 and a memory 401;

[0194] The memory 401 is used to store the computer program 402 and transmit the computer program 402 to the processor 400;

[0195] The processor 400 is configured to execute the steps of the above-mentioned embodiment of the homework grading method according to the instructions in the computer program 402 .

[0196] For example, the computer program 402 may be divided into one or more modules / units, one or more of which are stored in the memory 401 and executed by the processor 400 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 402 in the homework grading device 40.

[0197] The homework grading device 40 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The homework grading device 40 may include, but is not limited to, a processor 400 and a memory 401. Those skilled in the art will appreciate that Figure 10 It is only an example of the homework grading device 40 and does not constitute a limitation of the homework grading device 40. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the homework grading device 40 may also include input and output devices, network access devices, buses, etc.

[0198] The processor 400 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0199] The memory 401 may be an internal storage unit of the job grading device 40, such as a hard disk or memory of the job grading device 40. The memory 401 may also be an external storage device of the job grading device 40, such as a plug-in hard disk equipped on the job grading device 40, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Furthermore, the memory 401 may include both an internal storage unit of the job grading device 40 and an external storage device. The memory 401 is used to store computer programs and other programs and data required by the job grading device 40. The memory 401 may also be used to temporarily store data that has been output or is about to be output.

[0200] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0201] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0202] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0203] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0204] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes various media that can store computer programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0205] An embodiment of the present invention further provides a storage medium containing computer-executable instructions. When the computer-executable instructions are executed by a computer processor, the computer-executable instructions are used to perform a homework grading method. The method includes the following steps:

[0206] receiving a test paper image, wherein the test paper image includes test paper content;

[0207] Input the test paper image into the preset model, so that the preset model detects the pinyin area and the writing area of ​​each question in the "Look at the Pinyin and Write Chinese Characters" question type from the test paper image, and detects whether there is writing in the writing area;

[0208] For a target question with writing in the writing area, identifying the pinyin area of ​​the target question to obtain at least one predicted pinyin combination corresponding to the target question;

[0209] determining at least one target word corresponding to each predicted pinyin combination;

[0210] Identify the writing area of ​​the target question and obtain at least one predicted character corresponding to each character written in the writing area;

[0211] Determine whether each target question is answered correctly based on the target words and corresponding predicted text corresponding to each target question.

[0212] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the embodiments of the present invention have been described in detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the embodiments of the present invention, the embodiments of the present invention may also include more other equivalent embodiments, and the scope of the embodiments of the present invention is determined by the scope of the appended claims.

Claims

1. A method for correcting homework, characterized in that, it includes: Receiving a test paper image, which includes the content of the test paper; Inputting the test paper image into a preset model, so that the preset model detects the pinyin area and writing area of each question in the "write Chinese characters according to pinyin" question type from the test paper image, and detects whether there is writing in the writing area; For the target question with writing in the writing area, identifying the pinyin area of the target question to obtain at least one predicted pinyin combination corresponding to the target question; Determining at least one target word corresponding to each of the predicted pinyin combinations; Identifying the writing area of the target question to obtain predicted characters corresponding to each character written in the writing area, and the number of the predicted characters is at least one; Determining whether each target question is answered correctly according to the target word and the corresponding predicted characters corresponding to each target question.

2. The method for correcting homework according to claim 1, characterized in that, the preset model includes a feature extraction module, a feature integration module and a result output module; The step of inputting the test paper image into a preset model, so that the preset model detects the pinyin area and writing area of each question in the "write Chinese characters according to pinyin" question type from the test paper image, and detects whether there is writing in the writing area, includes: Inputting the test paper image into the feature extraction module, so that the feature extraction module extracts image features at multiple scales in the test paper image; Inputting the image features into the feature integration module, so that the feature integration module is used to integrate the image features to generate a feature matrix; Inputting the feature matrix into the result output module, so that the result output module processes the feature matrix and outputs the target information of the test paper image, and the target information includes the probability that each point in the test paper image is a question, the center position offset of the question, the size and offset angle of the question, the center position offset of the pinyin area of the question, the size and offset angle of the pinyin area of the question, the center position offset of the writing area of the question, the size and offset angle of the writing area of the question, and the answering probability of the writing area.

3. The method for correcting homework according to claim 1, characterized in that, the step of identifying the pinyin area of the target question to obtain at least one predicted pinyin combination corresponding to the target question, includes: Identifying the pinyin area of the target question to obtain the first probability that each identified first character belongs to a different letter; Predicting and determining at least one predicted pinyin combination corresponding to the target question according to the first probability.

4. The method for correcting homework according to claim 3, characterized in that, the step of predicting and determining the predicted pinyin combination corresponding to the target question according to the first probability, includes: Determine the first letter with the highest first probability corresponding to each of the first characters, and based on the first letter, determine the first predicted letter corresponding to each of the first characters, and determine the first predicted pinyin combination based on the first predicted letter; Update the first predicted pinyin combination according to the first probability to obtain at least one second predicted pinyin combination; Determine the predicted pinyin combination corresponding to the target title according to the first predicted pinyin combination and the second predicted pinyin combination.

5. A homework correction method according to claim 4, wherein, The updating the first predicted pinyin combination according to the first probability to obtain at least one second predicted pinyin combination includes: Sequentially confirm the current probability value of each first character in the pinyin area from largest to smallest until the number of the second predicted pinyin combinations reaches the number threshold; the undetermined probability value is the first probability other than the reference probability value corresponding to the corresponding first character, and the reference probability value is the maximum probability value among the first probabilities corresponding to each first character; Perform difference processing on the current probability values confirmed in each round, and each time the difference processing confirms the difference between the reference probability value and the current probability value corresponding to the same first character; Sort the differences of all first characters in the same round from largest to smallest, determine the target difference with the smallest value, and determine the target first character corresponding to the target difference; Determine the target letter corresponding to the current probability value of the target first character, and replace the letter corresponding to the target first character in the first predicted pinyin combination with the target letter to obtain a second predicted pinyin combination.

6. A homework correction method according to claim 1, wherein, The identifying the writing area of the target title to obtain the predicted text corresponding to each character written in the writing area includes: Identify the writing area of the target title and detect each second character in the writing area; Identify each of the second characters to obtain at least one predicted text corresponding to each of the second characters.

7. A homework correction method according to claim 1, wherein, The determining whether each target title is answered correctly according to the target word corresponding to each target title and the corresponding predicted text includes: Combine the predicted texts corresponding to each target title to obtain multiple predicted words; Determine whether any one of the predicted words matches any one of the target words; If they match, determine that the target title is answered correctly.

8. A homework correction method according to claim 7, wherein, When identifying the writing area of the target title, a second probability corresponding to each predicted text is also obtained; The combining the predicted texts corresponding to each target title to obtain multiple predicted words includes: Combine the predicted texts corresponding to each target title in pairs to obtain multiple candidate words; Calculate the sum of the second probabilities of the two predicted texts in each of the candidate words; Select the predicted words from the candidate words according to the sum of the second probabilities.

9. A homework marking device, characterized in that, it includes: An image receiving module for receiving a test paper image, where the test paper image includes the test paper content of the test paper; A region detection module for inputting the test paper image into a preset model, so that the preset model detects the pinyin region and the writing region of each question in the "write Chinese characters according to pinyin" question type from the test paper image, and detects whether there is writing in the writing region; A pinyin recognition module for, for a target question with writing in the writing region, recognizing the pinyin region of the target question to obtain at least one predicted pinyin combination corresponding to the target question; A target word determination module for determining at least one target word corresponding to each of the predicted pinyin combinations; A character recognition module for recognizing the writing region of the target question to obtain predicted texts corresponding to each character written in the writing region, and the number of the predicted texts is at least one; A question marking module for determining whether each target question is answered correctly according to the target word corresponding to each target question and the corresponding predicted text.

10. A homework marking device, characterized in that, the homework marking device includes a processor and a memory; the memory is used for storing a computer program and transmitting the computer program to the processor; the processor is used for executing a homework marking method according to any one of claims 1-8 according to the instructions in the computer program.

11. A storage medium storing computer-executable instructions, characterized in that, the computer-executable instructions are used for executing a homework marking method according to any one of claims 1-8 when executed by a computer processor.

Images