Title grading method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a question correction method and device, electronic equipment and storage medium, the method comprising: using a pre-trained question detection model to perform question detection on an image of a picture circle selection question to be corrected to obtain position information of a picture frame corresponding to each picture in a picture area of the picture circle selection question; using a pre-trained handwriting detection model to perform handwriting detection on the image to obtain a key point set of the circled handwriting in the picture area; determining a target picture circled in the picture area according to the key point set and the position information of the picture frame corresponding to each picture; and correcting the picture circle selection question according to the target picture to obtain a correction result of the picture circle selection question. The present scheme can effectively correct the picture circle selection question.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and particularly relates to a question correction method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the development of computer technology, online teaching has developed rapidly, and some teaching tool products have emerged as the times require, providing technical support and help for students, teachers and parents in education guidance. Many teaching tool products can provide a function of correcting questions by taking photos.

[0003] Taking photos to correct questions is an important application of artificial intelligence technology in the field of education. In primary school homework questions, picture circle selection questions are common question types for low-grade students. Since picture circle selection questions have complex layout and various picture styles, the current question correction method through text recognition has good effect on the correction of fill-in-the-blank questions and narrative questions, but is not suitable for the correction of picture circle selection questions.

[0004] Therefore, how to effectively correct picture circle selection questions has become a technical problem to be solved. SUMMARY

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a question correction method and device, an electronic device and a storage medium.

[0006] According to an aspect of the present disclosure, a question correction method is provided, comprising:

[0007] A pre-trained question detection model is used to perform question detection on an image of a picture circle selection question to be corrected to obtain position information of a picture frame corresponding to each picture in a picture area of the picture circle selection question;

[0008] A pre-trained handwriting detection model is used to perform handwriting detection on the image to obtain a key point set of circle selection handwriting in the picture area;

[0009] According to the key point set and the position information of the picture frame corresponding to each picture, a target picture circled in the picture area is determined;

[0010] According to the target picture, the picture circle selection question is corrected to obtain a correction result of the picture circle selection question.

[0011] According to another aspect of the present disclosure, a question correction device is provided, comprising:

[0012] The title detection module is configured to perform title detection on the image of the picture circle selection title to be corrected by using a pre-trained title detection model, to obtain position information of a picture frame corresponding to each picture in a picture region of the picture circle selection title.

[0013] The handwriting detection module is configured to perform handwriting detection on the image by using a pre-trained handwriting detection model, to obtain a key point set of the circle selection handwriting in the picture region.

[0014] The target picture determination module is configured to determine a target picture that is circled in the picture region according to the key point set and the position information of the picture frame corresponding to each picture.

[0015] The correction module is configured to correct the picture circle selection title according to the target picture, to obtain a correction result of the picture circle selection title.

[0016] According to another aspect of the present disclosure, an electronic device is provided, comprising:

[0017] a processor; and

[0018] a memory storing a program,

[0019] wherein the program includes instructions that, when executed by the processor, cause the processor to perform the title correction method according to the foregoing aspect.

[0020] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to perform the title correction method according to the foregoing aspect.

[0021] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements the title correction method according to the foregoing aspect.

[0022] One or more technical solutions provided in the embodiments of the present disclosure perform title detection on the image of the picture circle selection title to be corrected by using a pre-trained title detection model, to obtain position information of a picture frame corresponding to each picture in a picture region of the picture circle selection title, and perform handwriting detection on the image by using a pre-trained handwriting detection model, to obtain a key point set of the circle selection handwriting in the picture region, then determine a target picture that is circled in the picture region according to the key point set and the position information of the picture frame corresponding to each picture, and further correct the picture circle selection title according to the target picture, to obtain a correction result of the picture circle selection title. The solution of the present disclosure can effectively correct the picture circle selection title. BRIEF DESCRIPTION OF DRAWINGS

[0023] More details, features and advantages of the present disclosure are disclosed in the following description of exemplary embodiments in connection with the attached drawings, in which:

[0024] Figure 1 A flowchart of a question grading method according to an exemplary embodiment of the present disclosure is shown;

[0025] Figure 2 A flowchart of a question grading method according to another exemplary embodiment of the present disclosure is shown;

[0026] Figure 3 A flowchart of a question grading method according to yet another exemplary embodiment of the present disclosure is shown;

[0027] Figure 4 A schematic diagram of a result of a user's circle selection of a picture area of a picture question according to an exemplary embodiment of the present disclosure is shown;

[0028] Figure 5 A schematic diagram of a picture question according to an exemplary embodiment of the present disclosure is shown;

[0029] Figure 6 A schematic diagram of an image of a picture question according to an exemplary embodiment of the present disclosure is shown;

[0030] Figure 7 A schematic block diagram of a question grading apparatus according to an exemplary embodiment of the present disclosure is shown;

[0031] Figure 8 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0032] Embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the present disclosure are shown. Understanding that these drawings depict only some embodiments of the present disclosure and are not therefore to be considered to be limiting of its scope, the present disclosure will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:

[0033] It should be understood that various steps of the method embodiments of the present disclosure can be performed in different orders and / or in parallel. Furthermore, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present disclosure is not limited in this respect.

[0034] The term "include" and variations thereof, as used in this document, is an open term, and means "to include, but not limited to". The term "based on" means "based, at least in part, on". The term "one embodiment" means "at least one embodiment". The term "another embodiment" means "at least one additional embodiment". The term "some embodiments" means "at least some embodiments". Related terms shall be construed accordingly. It should be noted that "a" or "an" entity as used herein refers to one or more than one entity. The terms "first", "second" and the like as used herein do not connote any order, quantity, composition or importance, but rather are used to distinguish one element from another, and are more especially used to distinguish an entity from at least one of another entity.

[0035] It should be noted that the terms "one", "multiple", as mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that "one" or "multiple" should be understood as "one or more" unless otherwise explicitly indicated in the context.

[0036] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0037] The subject correction method, device, electronic device and storage medium provided by the present disclosure are described below with reference to the accompanying drawings.

[0038] Figure 1 A flowchart of a subject correction method according to an example embodiment of the present disclosure is shown, which can be executed by a subject correction device provided by an embodiment of the present disclosure, which can be implemented by software and / or hardware, and can be integrated in an electronic device, such as a mobile phone, a tablet computer, a wearable device, etc.

[0039] As shown in Figure 1 The subject correction method can include the following steps:

[0040] In step 101, a pre-trained subject detection model is used to detect the subject of a picture circle selection image to be corrected, to obtain the position information of the picture frame corresponding to each picture in the picture area of the picture circle selection image.

[0041] The picture circle selection subject described in the present disclosure refers to a subject containing multiple pictures (also referred to as patterns, such as small flowers, airplanes, triangles, etc.) for students to select, and requires students to select a corresponding number of pictures from multiple pictures. The subject can also contain an algorithm.

[0042] The topic detection model is pre-trained. A large number of image of picture circle selection topics can be collected in advance as training samples to train an initial network model. When the loss function of the initial network model is lower than a preset loss value, the trained topic detection model is obtained. The topic detection model can output at least the position information of the detection frame corresponding to each picture in the picture region of the picture circle selection topic. The topic detection model can also output the deflection angle of each picture relative to the horizontal direction (i.e., the width direction of the image), the position information of the detection frame corresponding to the picture region, and the like. The position information of the detection frame corresponding to the picture can be represented by the coordinates of the four vertices of the detection frame, denoted as [(x1, y1), (x2, y2), (x3, y3), (x4, y4)], or represented by the center point coordinates of the detection width and the width (w) and height (h) of the detection width, denoted as (x, y, w, h), or represented in other ways. The present disclosure does not limit this.

[0043] In addition, the present disclosure does not limit the network structure of the initial network model used to train the topic detection model. Exemplarily, the initial network model can adopt a CenterNet model. The CenterNet model is an Anchor-free target detection algorithm improved on the basis of the CornerNet algorithm. Compared with the single-stage target detection algorithm yolov3, the CenterNet algorithm improves the accuracy by 4 percentage points under the premise of ensuring the speed. Compared with other single-stage or double-stage target detection algorithms, the CenterNet algorithm has the following advantages:

[0044] (1) The algorithm removes the inefficient and complex Anchor operation, further improving the performance of the detection algorithm.

[0045] (2) The algorithm directly performs filtering operation on the heat map (heatmap), removes the time-consuming Non-Maximum Suppression (NMS) post-processing operation, and further improves the running speed of the entire algorithm.

[0046] (3) The algorithm can not only be applied to 2D target detection, but also can be applied to 3D target detection and human key point set detection and other tasks after simple changes, and has good versatility.

[0047] In the embodiment of the present disclosure, for the obtained picture circle selection topic image to be corrected, the image can be input into the pre-trained topic detection model, the topic detection model performs topic detection on the image, identifies the picture circle selection image contained in the image, and performs picture detection on the picture region of the picture circle selection topic, and outputs the position information of the picture frame corresponding to each picture in the picture region, wherein the picture frame is the detection frame corresponding to the picture.

[0048] It should be noted that in the embodiments of the present disclosure, the picture circle selection question can be any circle selection question in all circle selection questions contained in the image. If several circle selection sub-questions are contained in a circle selection question major, the picture circle selection question can be any circle selection sub-question in all circle selection sub-questions.

[0049] In step 102, a handwriting detection model pre-trained is used to perform handwriting detection on the image to obtain a key point set of the circle selection handwriting in the picture region.

[0050] The handwriting detection model is pre-trained, and the handwriting detection model can output the key point set of the circle selection handwriting in the image.

[0051] For example, the handwriting detection model can use an ultra-fast lane detection algorithm (Ultra-Fast-Lane-Detection, UFLD). The UFLD algorithm is a novel, simple and effective algorithm, which regards the lane detection process as a line-based selection problem using global features (defines lane detection as finding a set of positions of certain lines in the image, i.e., position selection and classification based on line direction). It can be understood that the UFLD algorithm is only an example, and the handwriting detection model can also use other algorithms to realize handwriting detection. The specific algorithm used by the handwriting detection model is not limited in the embodiments of the present disclosure.

[0052] In the embodiments of the present disclosure, for the obtained picture circle selection question image to be corrected, the image can be input into the pre-trained handwriting detection model, and the handwriting detection model is used to perform handwriting detection on the image. The handwriting detection model outputs a key point set of the circle selection handwriting contained in the picture region of the picture circle selection question. The key point set contains a plurality of key points.

[0053] It can be understood that when the picture region contains a plurality of circle selection handwritings, the circle selection handwritings are not connected, i.e., one circle selection handwriting circled a part of the picture region, for example, one circle selection handwriting circled a picture in the upper left of the picture region, and another circle selection handwriting circled two pictures in the lower right of the picture region. In this case, the key point set can contain a plurality of sub-sets, and one sub-set corresponds to one circle selection handwriting.

[0054] It should be noted that in the embodiments of the present disclosure, the execution order of steps 101 and 102 is not limited, and steps 101 and 102 can be executed sequentially or simultaneously. The embodiments only take step 102 executed after step 101 as an example to explain the present disclosure, but cannot be regarded as a limitation of the present disclosure.

[0055] In an optional embodiment of the present disclosure, before inputting the image of the picture circle selection question to be corrected into the question detection model and the handwriting detection model, the image of the picture circle selection question to be corrected can be preprocessed, including cropping the background area in the image to only retain the question part of the picture circle selection question, performing perspective transformation correction on the retained question part, and the like, so as to input the preprocessed picture circle selection question image into the question detection model and the handwriting detection model for detection respectively, so as to ensure the accuracy of the detection result.

[0056] In step 103, the target picture that is circled in the picture region is determined according to the key point set and the position information of the picture frame corresponding to each picture.

[0057] In the embodiments of the present disclosure, after obtaining the position information of the picture frame corresponding to each picture in the picture circle selection question picture region and the key point set of the circled handwriting in the picture region, the target picture that is circled in the picture region can be determined according to the key point set and the position information of the picture frame corresponding to each picture.

[0058] As an optional embodiment, the range circled by the circled handwriting in the picture region can be determined according to the obtained key point set, and then whether the picture is within the range circled by the circled handwriting is judged according to the position information of the picture frame corresponding to each picture. If a certain picture falls within the range circled by the circled handwriting, the picture can be determined as a target picture.

[0059] As an optional embodiment, the maximum value and the minimum value of the horizontal coordinate and the maximum value and the minimum value of the vertical coordinate of the picture frame can be determined according to the position information of the picture frame corresponding to each picture. For a certain picture, if the minimum value of the horizontal coordinate of the picture frame corresponding to the picture is not less than the horizontal coordinate value of at least one key point in the key point set, the maximum value of the horizontal coordinate of the picture frame corresponding to the picture is not greater than the horizontal coordinate value of at least one key point in the key point set, the minimum value of the vertical coordinate of the picture frame corresponding to the picture is not less than the vertical coordinate value of at least one key point in the key point set, and the maximum value of the vertical coordinate of the picture frame corresponding to the picture is not greater than the vertical coordinate value of at least one key point in the key point set, the picture is determined as a target picture that is circled.

[0060] In step 104, the picture circle selection question is corrected according to the target picture, and a correction result of the picture circle selection question is obtained.

[0061] In the embodiments of the present disclosure, after the target picture that is circled is determined from each picture contained in the picture region, the picture circle selection question can be corrected according to the target picture, and a correction result of the picture circle selection question is obtained.

[0062] As an optional implementation, the category of the target picture can be detected, for example, a pre-trained picture category detection model can be used to detect the category of the target picture, the category of the target picture and the position of the target picture in the picture region are compared with the standard answer corresponding to the picture circle selection picture, whether the target picture circled by the user is accurate is judged, if yes, a circle selection correct correction result is generated, otherwise, a circle selection error correction result is generated.

[0063] As an optional implementation, the category of the target picture can be detected, for example, a pre-trained picture category detection model can be used to detect the category of the target picture, the category of the target picture and the position of the target picture in the picture region are compared with the standard answer corresponding to the picture circle selection picture, whether the target picture circled by the user is accurate is judged, if yes, a circle selection correct correction result is generated, otherwise, a circle selection error correction result is generated.

[0064] The question correction method of the embodiment of the present disclosure, by using the pre-trained question detection model, the image of the picture circle selection question to be corrected is detected to obtain the position information of the picture frame corresponding to each picture in the picture region of the picture circle selection question, and the pre-trained handwriting detection model is used to detect the handwriting to obtain the key point set of the circle selection handwriting in the picture region, then, according to the key point set and the position information of the picture frame corresponding to each picture, the target picture circled in the picture region is determined, and then the picture circle selection question is corrected according to the target picture to obtain the correction result of the picture circle selection question. The scheme of the present disclosure can effectively correct the picture circle selection question.

[0065] In an optional embodiment of the present disclosure, as shown in Figure 2 based on the embodiment as shown in Figure 1 Step 103 can include the following sub-steps:

[0066] Step 201, generating the minimum circumscribed graph corresponding to the circle selection handwriting according to the key point set.

[0067] In the embodiment of the present disclosure, the minimum circumscribed graph of each key point in the key point set can be obtained according to the key point set.

[0068] Exemplarily, the minimum circumscribed graph can be a minimum circumscribed rectangle.

[0069] Step 202, obtaining the position information of the minimum circumscribed graph.

[0070] In the embodiment of the present disclosure, after the minimum circumscribed graph is generated, the position information of the minimum circumscribed graph can be obtained.

[0071] Exemplarily, the minimum circumscribed figure is a minimum circumscribed rectangle, and the coordinates of the four vertices of the minimum circumscribed rectangle can be obtained as the position information of the minimum circumscribed rectangle. It can be understood that the minimum circumscribed rectangle is generated based on the key point set, and the maximum and minimum values of the horizontal coordinates and the maximum and minimum values of the vertical coordinates of the key points in the key point set can be determined, and then the four coordinates can be determined according to the maximum and minimum values of the horizontal coordinates and the maximum and minimum values of the vertical coordinates, that is, the coordinates of the four vertices of the minimum circumscribed rectangle are obtained, so as to obtain the position information of the minimum circumscribed rectangle.

[0072] Exemplarily, the minimum circumscribed figure is a minimum circumscribed circle, and the center and the radius of the minimum circumscribed circle can be obtained to represent the position information of the minimum circumscribed circle. It can be understood that, since it is a minimum circumscribed circle, at least two key points in the key point set are located on the minimum circumscribed circle, and the position information of the two key points located on the minimum circumscribed circle can be obtained, and the position of the center and the radius can be determined according to the position information of the two key points, so as to obtain the position information of the minimum circumscribed circle.

[0073] In step 203, the target picture circled in the picture area is determined according to the position information of the picture frame corresponding to each picture and the position information of the minimum circumscribed figure.

[0074] In the embodiments of the present disclosure, after obtaining the position information of the minimum circumscribed figure, the target picture circled in the picture area can be screened from the pictures in the picture area according to the position information of the minimum circumscribed figure and the position information of the picture frame corresponding to each picture.

[0075] As an optional implementation, for any picture, the position information of the picture frame corresponding to the picture can be compared with the position information corresponding to the minimum circumscribed figure, to determine whether the picture frame falls within the minimum circumscribed figure, and if so, the picture is determined as a target picture circled.

[0076] As an optional implementation, for any picture (referred to as a first picture for convenience of description) in each picture of the picture area, the overlapping area between the picture frame corresponding to the first picture and the minimum circumscribed figure can be determined according to the position information of the picture frame corresponding to the first picture and the position information of the minimum circumscribed figure, and the picture frame area of the picture frame corresponding to the first picture can be determined according to the position information of the picture frame corresponding to the first picture, and then the ratio of the overlapping area to the picture frame area is calculated, and in the case that the ratio is greater than a preset value, the first picture is determined as a target picture, and in the case that the ratio is not greater than the preset value, it is determined that the first picture is not circled.

[0077] The preset value can be set according to actual needs, for example, the preset value can be set as 0.8.

[0078] For example, the position information of the minimum circumscribed rectangle and the position information of the picture frame are expressed by the coordinates of four vertices, then the maximum and minimum values on the horizontal coordinate axis can be determined according to the position information of the minimum circumscribed rectangle, denoted as c_x max and c_x min , and the maximum and minimum values on the vertical coordinate axis can be determined, denoted as c_y max and c_y min , and the maximum and minimum values on the horizontal coordinate axis can be determined according to the position information of the picture frame, denoted as x max and x min , and the maximum and minimum values on the vertical coordinate axis can be determined, denoted as y max and y min . Then, the overlap width and overlap height between the picture frame and the minimum circumscribed rectangle can be determined according to the maximum and minimum values. The overlap width is min(x max , c_x max )-max(x min , c_x min ), and the overlap height is min(y max , c_y max )-max(y min , y_x min ). Thus, the overlap area is the overlap width multiplied by the overlap height. The picture frame area is (x max -x min )*(y max -y min ). The ratio of the overlap area to the picture frame area is: overlap area / picture frame area. If the ratio is greater than a preset value, the first picture is determined as the target picture.

[0079] In the embodiments of the present disclosure, by calculating the ratio of the overlap area of the picture frame and the minimum circumscribed figure to the picture frame area, and determining the picture frame as the target picture frame when the ratio is greater than a preset value, compared with the traditional calculation method of the intersection-over-union ratio (i.e., the overlap area divided by (the sum of the areas-the overlap area)), the phenomenon that the picture is misjudged as not being selected due to the large area difference between the minimum circumscribed figure and the picture frame, resulting in a small intersection-over-union ratio, is avoided, and the accuracy of the judgment result of the selected picture is improved.

[0080] The subject correction method of the embodiment of the present disclosure generates the minimum circumscribed figure corresponding to the circled handwriting according to the key point set, and obtains the position information of the minimum circumscribed figure, and then determines the target picture circled in the picture area according to the position information of the picture frame corresponding to each picture and the position information of the minimum circumscribed figure, thereby ensuring the accuracy of the target picture determination.

[0081] In an optional embodiment of the present disclosure, as shown in Figure 3 On the basis of the foregoing embodiment, step 104 can include the following sub-steps:

[0082] Step 301: Obtain the feature vector corresponding to the target picture.

[0083] In the embodiment of the present disclosure, for the determined target picture, the feature vector corresponding to each target picture can be obtained.

[0084] The feature vector corresponding to the target picture can be extracted by using the currently commonly used feature extraction method.

[0085] For example, the Scale-Invariant Feature Transform (SIFT) algorithm, the Speeded Up Robust Features (SURF) algorithm, the ORB (Oriented FAST and Rotated BRIEF) algorithm, and other feature extraction algorithms can be used to extract the feature vector corresponding to each target picture.

[0086] The SIFT algorithm is a classic feature extraction algorithm, which is used to detect and describe local features in digital images and has an important position in computer vision research and application. It is widely used in image recognition, image matching, target tracking, and three-dimensional reconstruction. The core idea of the SIFT algorithm is to describe the local features of an image by finding stable key points (feature points) at different scales. It has scale invariance, that is, regardless of the size of the object in the image, SIFT can detect the corresponding feature points. This scale invariance is achieved by using a Gaussian pyramid, which simulates images at different scales by blurring the image at different levels. The main steps of the SIFT algorithm include:

[0087] Scale space extremum detection: find extremum points in the Gaussian pyramid image at different scales, which may be candidates for key points;

[0088] Key point positioning: accurately position the candidate key points in the scale space, and eliminate points with low contrast and weak edge response to obtain stable key points;

[0089] Direction assignment: Assign a dominant direction to each keypoint, which is usually determined by the image gradient directions around the keypoint, for subsequent feature description;

[0090] Feature description: Calculate local feature descriptors around each keypoint, which are rotation-invariant, illumination-invariant, and robust to changes in viewpoint and scale.

[0091] The output of the SIFT algorithm is a set of feature vectors that describe the local features of the image, which can be used for image matching, object tracking, and image stitching.

[0092] Step 302, match the feature vector with the preset feature vector library to determine the target feature vector with the highest similarity between the feature vector and the target feature vector in the feature vector library.

[0093] Wherein, the feature vector library is pre-set, the feature vector library records the mapping relationship between different picture categories and corresponding feature vectors, the feature vector and the picture category are one-to-one correspondence, the picture category can be but not limited to flower (different flowers represent different picture categories), triangle, circle, car, airplane, cat, dog, etc., that is, one picture is one picture category.

[0094] In the embodiment of the present disclosure, after obtaining the feature vector corresponding to each target picture in the target picture, the feature vector can be matched with each feature vector in the feature vector library, the similarity between the feature vector and each feature vector in the feature vector library is calculated, and according to the calculated similarity, the feature vector with the highest similarity between the feature vector corresponding to the target picture and the feature vector is determined as the target feature vector.

[0095] It should be noted that the calculation method of the similarity in the embodiment of the present disclosure is not limited, and the commonly used data comparison method can be used to compare the similarity between the feature vectors, such as variance matching method, normalized variance matching method, correlation matching method, normalized cross-correlation matching method, correlation coefficient matching method, and normalized correlation coefficient matching method, to determine the similarity between the feature vector of the target picture and each feature vector in the feature vector library.

[0096] Step 303, obtain the target picture category corresponding to the target feature vector.

[0097] In the embodiment of the present disclosure, after determining the target feature vector with the highest similarity to the feature vector of the target picture, the feature vector library can be queried, and the target picture category corresponding to the target feature vector can be determined according to the mapping relationship between each feature vector and the picture category in the feature vector library.

[0098] In step 304, the correction result of the picture circle selection question is determined according to the target picture category and the standard answer corresponding to the picture circle selection question.

[0099] The standard answer can be obtained by searching.

[0100] As an optional implementation, printed text recognition can be performed on the image of the picture circle selection question to be corrected to obtain the printed text contained in the image. Then, based on the obtained printed text, text search is performed in the preset question search library to determine the standard question image corresponding to the image. Further, the standard answer corresponding to the standard question image is obtained as the standard answer corresponding to the picture circle selection question. The standard question image corresponding to the image is searched and obtained by text search, and then the standard answer corresponding to the standard question image is obtained. Since the search speed of text search is very fast, the standard answer can be obtained more quickly, which helps to improve the speed and efficiency of question correction.

[0101] As another optional implementation, feature extraction can be performed on the image of the picture circle selection question to be corrected to obtain the image feature corresponding to the image. Then, based on the image feature, feature matching is performed in the preset question search library to calculate the similarity between the standard image feature of each standard question image in the question search library and the obtained image feature. The standard question image with the highest similarity is selected as the target standard question image, and then the standard answer corresponding to the target standard question image is obtained as the standard answer corresponding to the picture circle selection question in the image.

[0102] In the embodiments of the present disclosure, after the target picture category is obtained, the picture circle selection question can be corrected according to the target picture category and the obtained standard answer to determine the correction result of the picture circle selection question.

[0103] As an optional implementation, the total number of times each target picture category in the target picture category appears can be counted to obtain the number of each type of target picture in the circle selection. Then, the number of each type of target picture is compared with the number of pictures of the same category in the standard answer. In the case where the number of each type of target picture is consistent with the number of pictures of the same category in the standard answer, a correction result that the picture circle selection question is answered correctly is generated, otherwise, a correction result that the picture circle selection question is answered incorrectly is generated. In this way, the correction of the picture circle selection question in which multiple different categories of pictures are mixed together is realized, the types of correctable picture circle selection questions are enriched, and the application scope is wider.

[0104] For example, suppose there's a picture selection exercise with the standard answer being: △*3, ○*2, and assuming △ corresponds to target picture category 1, and ○ corresponds to target picture category 2. For each target picture, determine its corresponding target picture category. Assume the user's selection result in the picture selection area is as follows: Figure 4 As shown, Figure 4 The irregular curves in the diagram represent the circled handwriting, thus identifying the selected target image as "△△△○○". Next, the feature vectors corresponding to each target image are obtained and matched in the feature vector database, determining the corresponding target image category as "11122". This shows there are two target image categories: "1" and "2". Then, the total number of occurrences for each target image category is counted, showing that target image category "1" appears 3 times and target image category "2" appears 2 times. Based on the correspondence between target image categories and images, the number of circled △s is 3, and the number of circled ○s is 2. Comparing this with the standard answer confirms that the number of circled △s and ○s matches the number of circled △s in the standard answer, thus generating a corrected answer to the picture selection question. Assuming the target picture category is "1122", the final number of circled triangles is 2, and the number of circled circles is 2. By comparing with the standard answer, it can be determined that the number of circled triangles is inconsistent with the number of triangles in the standard answer. Therefore, the correction result of the incorrect answer to the picture circle question is generated.

[0105] In practical applications, it is common to encounter situations where the picture selection question includes not only pictures but also equations, for example... Figure 5 The image shown in the picture circle is the selected topic. Figure 5 The illustrated picture selection question includes not only picture area 1 but also equation 2, which contains two arithmetic problems. For this type of question, it is necessary to check not only whether the selected picture is correct but also whether the result of the equation is correct; only when both are correct is the answer to the question deemed correct. To achieve effective checking of this type of question, in an optional embodiment of this disclosure, the question checking method provided by this disclosure may further include: detecting whether the picture selection question contains an equation; and in response to the picture selection question containing an equation, performing handwriting recognition on the image to obtain the handwriting recognition result.

[0106] For example, a pre-trained question detection model can be used to detect whether a picture-circle question contains an equation. When the picture-circle question contains an equation, a pre-trained handwriting recognition model can be used to perform handwriting recognition on the image to obtain the handwriting recognition result. It is understood that the handwriting of the user's written answer is different from the handwriting of the printed text contained in the question; therefore, the handwriting recognition result obtained through handwriting recognition contains the user's written answer.

[0107] Therefore, in the embodiment, when determining the correction result of the picture circle selection question according to the target picture category and the standard answer corresponding to the picture circle selection question, the circle selection result of the picture circle selection question can be determined according to the target picture category and the standard answer corresponding to the picture circle selection question, and the arithmetic result of the picture circle selection question can be determined according to the handwriting recognition result and the standard answer. Then, the correction result of the picture circle selection question can be determined based on the circle selection result and the arithmetic result.

[0108] The circle selection result of the picture circle selection question can be determined by the related correction methods provided in the above embodiments, for example, whether the circle selection result is correct can be determined according to the total number of each target picture selected. When determining the arithmetic result, the handwriting recognition result can be compared with the standard answer, for example, the handwriting recognition result is 3 and the answer of the formula provided in the standard answer is also 3, then it can be determined that the arithmetic result is correct.

[0109] In the embodiment of the present disclosure, when the circle selection result and the arithmetic result of the same picture circle selection question are both correct, it is determined that the correction result of the picture circle selection question is correct, otherwise, it is determined that the correction result of the picture circle selection question is incorrect.

[0110] The question correction method in the embodiment of the present disclosure can obtain the feature vector corresponding to the target picture, match the feature vector with the preset feature vector library, determine the target feature vector with the highest similarity between the feature vectors in the feature vector library, and obtain the target picture category corresponding to the target feature vector. Then, the correction result of the picture circle selection question is determined according to the target picture category and the standard answer corresponding to the picture circle selection question. Therefore, the picture category of the target picture can be accurately identified, which lays a foundation for improving the accuracy of the correction result of the picture circle selection question.

[0111] In an optional embodiment of the present disclosure, the question detection model can also detect the direction of the picture area, that is, detect the direction of each picture in the picture area, to detect and output the deflection angle of each picture in the picture area relative to the width direction (i.e. the horizontal direction) of the image. The deflection angle can gradually increase in the counterclockwise direction, or it can gradually increase in the clockwise direction, which is not limited in the present disclosure. It can be understood that the deflection angle of each picture in the picture area relative to the horizontal direction is the same, so the deflection angle output by the question detection model is an angle value. Therefore, in the embodiment, the question correction method can also include correcting the position information of the picture frame corresponding to each picture according to the deflection angle of each picture relative to the horizontal direction.

[0112] Exemplarily, Figure 6 A schematic diagram of an image of a picture circle selection question in an exemplary embodiment of the present disclosure is shown as Figure 6As shown, the deflection angle gradually increases counterclockwise. In the image of the picture selection question to be graded, each picture in the picture area is deflected relative to the horizontal direction of the image, with a deflection angle of θ. Then, the question detection model outputs the picture box corresponding to each picture. Figure 6 When processing the position information of the solid-lined boxes (in each drawing), a deflection angle θ will also be output. Based on this deflection angle, the position can be adjusted. Figure 6 The position information of the horizontal picture frames corresponding to each picture in the problem is corrected. Specifically, based on the position information and deflection angle θ of the picture frames output by the problem detection model, trigonometric theorems are used to determine the corrected position information of the picture frames. The corrected picture frames are shown below. Figure 6 The dashed boxes in each drawing are shown. It should be noted that the deflection angle θ is the angle between the drawing and the horizontal direction of the image. Figure 6 In this diagram, θ is marked as the angle between the corrected drawing frame and the horizontal direction for ease of drawing. Since the corrected drawing frame and the drawing are aligned in the same direction, the angle between the corrected drawing frame and the horizontal direction is the same as the angle between the drawing and the horizontal direction.

[0113] Furthermore, after correcting the position information of the picture frame corresponding to each picture based on the deflection angle, the target picture in the picture area can be determined according to the key point set and the corrected position information of the picture frame corresponding to each picture. The process of determining the target picture can be referred to the relevant description in the foregoing embodiment on determining the target picture in the picture area according to the key point set and the position information of the picture frame corresponding to each picture. The implementation principle is similar and will not be repeated here.

[0114] Since the picture frames corresponding to each picture output by the question detection model are horizontal, when the picture area is deflected relative to the horizontal direction of the image, each picture is also deflected relative to the horizontal direction of the image. This may lead to inaccurate position information of the picture frames. Therefore, in this embodiment of the disclosure, the deflection angle of each picture in the picture area relative to the width direction of the image is output by the question detection model, and the position information of the picture frames corresponding to each picture is corrected according to the deflection angle. This makes the position information of the picture frames more accurate and has a better effect on the situation where the picture is not in the correct position in the image.

[0115] This exemplary embodiment also provides a question grading device. Figure 7 A schematic block diagram of a title correction device according to an exemplary embodiment of the present disclosure is shown, such as Figure 7 As shown, the question correction device 70 includes: a question detection module 710, a handwriting detection module 720, a target image determination module 730, and a correction module 740.

[0116] The title detection module 710 is configured to perform title detection on an image of a picture circle selection title to be corrected by using a pre-trained title detection model, to obtain position information of a picture frame corresponding to each picture in a picture region of the picture circle selection title.

[0117] The handwriting detection module 720 is configured to perform handwriting detection on the image by using a pre-trained handwriting detection model, to obtain a key point set of the circle selection handwriting in the picture region.

[0118] The target picture determination module 730 is configured to determine a target picture that is circled in the picture region according to the key point set and the position information of the picture frame corresponding to each picture.

[0119] The correction module 740 is configured to correct the picture circle selection title according to the target picture, to obtain a correction result of the picture circle selection title.

[0120] Optionally, the target picture determination module 730 includes:

[0121] A graph generation unit is configured to generate a minimum circumscribed graph corresponding to the circle selection handwriting according to the key point set.

[0122] A position acquisition unit is configured to acquire position information of the minimum circumscribed graph.

[0123] A target picture determination unit is configured to determine a target picture that is circled in the picture region according to the position information of the picture frame corresponding to each picture and the position information of the minimum circumscribed graph.

[0124] Optionally, the target picture determination unit is further configured to:

[0125] determine an overlapping area between the picture frame corresponding to a first picture and the minimum circumscribed image according to the position information of the picture frame corresponding to the first picture and the position information of the minimum circumscribed graph, the first picture being any picture in the each picture;

[0126] determine a picture frame area of the picture frame corresponding to the first picture according to the position information of the picture frame corresponding to the first picture;

[0127] calculate a ratio of the overlapping area to the picture frame area;

[0128] in a case where the ratio is greater than a preset value, determine the first picture as the target picture.

[0129] Optionally, the correction module 740 includes:

[0130] A feature acquisition unit is configured to acquire a feature vector corresponding to the target picture.

[0131] a feature matching unit, configured to match the feature vector with a preset feature vector library, and determine a target feature vector in the feature vector library that has the highest similarity with the feature vector;

[0132] a category obtaining unit, configured to obtain a target picture category corresponding to the target feature vector;

[0133] a correction unit, configured to determine a correction result of the picture circle selection question according to the target picture category and a standard answer corresponding to the picture circle selection question.

[0134] Optionally, the correction unit is further configured to:

[0135] count a total number of occurrences of each target picture category in the target picture category, to obtain a number of each type of target picture in the circle selection;

[0136] compare the number of each type of target picture with the standard answer;

[0137] in a case where the number of each type of target picture is consistent with a number of the same type of picture in the standard answer, generate a correction result that the picture circle selection question is answered correctly.

[0138] Optionally, the question correction apparatus 70 further includes:

[0139] an equation detection module, configured to detect whether the picture circle selection question contains an equation;

[0140] a handwriting recognition module, configured to, in response to the picture circle selection question containing the equation, perform handwriting recognition on the image to obtain a handwriting recognition result;

[0141] and wherein the correction unit is further configured to:

[0142] determine a circle selection result of the picture circle selection question according to the target picture category and the standard answer corresponding to the picture circle selection question;

[0143] determine an arithmetic result of the picture circle selection question according to the handwriting recognition result and the standard answer;

[0144] determine a correction result of the picture circle selection question based on the circle selection result and the arithmetic result.

[0145] Optionally, the question correction apparatus 70 further includes:

[0146] a printed text recognition module, configured to perform printed text recognition on the image to obtain printed text contained in the image;

[0147] The text search module is configured to perform text search in a preset topic search library based on the printed text, and determine a standard topic image corresponding to the image.

[0148] The answer determination module is configured to obtain a standard answer corresponding to the standard topic image as a standard answer corresponding to the picture circle selection topic.

[0149] Optionally, the topic detection model also outputs a deflection angle of each picture in the picture area relative to a width direction of the image.

[0150] And wherein the topic correction device 70 further comprises:

[0151] The position correction module is configured to correct position information of the picture frame corresponding to each picture according to the deflection angle of each picture relative to the horizontal direction.

[0152] The topic correction device provided in the embodiments of the present disclosure can perform any applicable topic correction method provided in the embodiments of the present disclosure, and has the corresponding function modules and beneficial effects of the execution method. The contents not described in detail in the device embodiments of the present disclosure can be referred to the description in any method embodiment of the present disclosure.

[0153] The exemplary embodiments of the present disclosure also provide an electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication. The memory stores a computer program capable of being executed by the at least one processor, and the computer program, when executed by the at least one processor, is configured to cause the electronic device to perform the topic correction method according to the embodiments of the present disclosure.

[0154] The exemplary embodiments of the present disclosure also provide a non-transitory computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is configured to cause the computer to perform the topic correction method according to the embodiments of the present disclosure.

[0155] The exemplary embodiments of the present disclosure also provide a computer program product comprising a computer program, wherein the computer program, when executed by a processor of a computer, is configured to cause the computer to perform the topic correction method according to the embodiments of the present disclosure.

[0156] Reference Figure 8The present invention describes a structural block diagram of an electronic device 1100 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0157] like Figure 8 As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. The RAM 1103 may also store various programs and data required for the operation of the device 1100. The computing unit 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0158] Multiple components in electronic device 1100 are connected to I / O interface 1105, including: input unit 1106, output unit 1107, storage unit 1108, and communication unit 1109. Input unit 1106 can be any type of device capable of inputting information to electronic device 1100. Input unit 1106 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 1107 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1108 may include, but is not limited to, disk and optical disk. Communication unit 1109 allows electronic device 1100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0159] The computing unit 1101 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs various methods and processes described above. For example, in some embodiments, the question grading method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. In some embodiments, the computing unit 1101 can be configured to perform the question grading method by any other suitable means, such as by means of firmware.

[0160] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, and partially on a remote machine or entirely on a remote machine or server.

[0161] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable storage media can include, without limitation, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media can include one or more lines of electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical storage devices, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0162] As used in this disclosure, the terms "machine-readable medium" and "computer- readable medium" refer to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0163] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0164] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0165] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

Claims

1. A method of grading a test, wherein, The method comprises: using a pre-trained question detection model to perform question detection on an image of a picture circle selection question to be corrected to obtain position information of a picture frame corresponding to each picture in a picture region of the picture circle selection question; using a pre-trained handwriting detection model to perform handwriting detection on the image to obtain a key point set of the circle selection handwriting in the picture region; determining a target picture that is circled in the picture region according to the key point set and the position information of the picture frame corresponding to each picture; correcting the picture circle selection question according to the target picture to obtain a correction result of the picture circle selection question, wherein a target picture category corresponding to the target picture is obtained, and the correction result is determined according to a number of target pictures of each target picture category contained in the target picture and a number of pictures of the same category in a standard answer of the picture circle selection question.

2. The subject correction method according to claim 1, wherein The determination of the target picture that is circled in the picture region according to the key point set and the position information of the picture frame corresponding to each picture comprises: generating a minimum circumscribed figure corresponding to the circle selection handwriting according to the key point set; obtaining position information of the minimum circumscribed figure; determining the target picture that is circled in the picture region according to the position information of the picture frame corresponding to each picture and the position information of the minimum circumscribed figure.

3. The subject correction method according to claim 2, wherein The determination of the target picture that is circled in the picture region according to the position information of the picture frame corresponding to each picture and the position information of the minimum circumscribed figure comprises: determining an overlapping area between the picture frame corresponding to a first picture and the minimum circumscribed figure according to the position information of the picture frame corresponding to the first picture and the position information of the minimum circumscribed figure, the first picture being any picture in the pictures; determining a picture frame area of the picture frame corresponding to the first picture according to the position information of the picture frame corresponding to the first picture; calculating a ratio of the overlapping area to the picture frame area; in a case where the ratio is greater than a preset value, determining the first picture as the target picture.

4. The question grading method according to any one of claims 1 to 3, wherein, The correction of the picture circle selection question according to the target picture to obtain the correction result of the picture circle selection question comprises: obtaining a feature vector corresponding to the target picture; matching the feature vector with a preset feature vector library to determine a target feature vector with the highest similarity between the feature vector and the target feature vector in the feature vector library; obtaining a target picture category corresponding to the target feature vector; determining the correction result of the picture circle selection question according to the target picture category and a standard answer corresponding to the picture circle selection question.

5. The subject correction method according to claim 4, wherein The determination of the correction result of the picture circle selection question according to the target picture category and the standard answer corresponding to the picture circle selection question comprises: counting a total number of times that each target picture category in the target picture category appears to obtain a number of each type of target picture that is circled; comparing the number of each type of target picture with a number of pictures of the same category in the standard answer; In a case where the number of each type of target picture is consistent with the number of the same type of picture in the standard answer, a correction result of the picture circle selection question is generated.

6. The subject correction method according to claim 4, wherein The method further includes: detecting whether the picture circle selection question contains an equation; in response to the picture circle selection question containing an equation, performing handwriting recognition on the image to obtain a handwriting recognition result; and wherein the determining of the correction result of the picture circle selection question according to the target picture category and the standard answer corresponding to the picture circle selection question includes: determining a circle selection result of the picture circle selection question according to the target picture category and the standard answer corresponding to the picture circle selection question; determining an arithmetic result of the picture circle selection question according to the handwriting recognition result and the standard answer; determining the correction result of the picture circle selection question based on the circle selection result and the arithmetic result.

7. The subject correction method as claimed in claim 4, wherein, The method further includes: performing printed text recognition on the image to obtain printed text contained in the image; based on the printed text, performing text search in a preset question search library to determine a standard question image corresponding to the image; obtaining a standard answer corresponding to the standard question image as the standard answer corresponding to the picture circle selection question.

8. The title correction method according to any one of claims 1 to 3, wherein, The question detection model further outputs a deflection angle of each picture in the picture region relative to the width direction of the image; and wherein the method further includes: correcting position information of a picture frame corresponding to each picture according to the deflection angle of each picture relative to the horizontal direction.

9. A test grading apparatus, wherein, The device includes: a question detection module configured to use a pre-trained question detection model to perform question detection on an image of a picture circle selection question to be corrected to obtain position information of a picture frame corresponding to each picture in a picture region of the picture circle selection question; a handwriting detection module configured to use a pre-trained handwriting detection model to perform handwriting detection on the image to obtain a key point set of circle selection handwriting in the picture region; a target picture determination module configured to determine a target picture selected in the picture region according to the key point set and the position information of the picture frame corresponding to each picture; a correction module configured to correct the picture circle selection question according to the target picture to obtain a correction result of the picture circle selection question, wherein a target picture category corresponding to the target picture is obtained, and the correction result is determined according to the number of each type of target picture contained in the target picture and the number of the same type of picture in a standard answer of the picture circle selection question.

10. An electronic device, comprising: a processor; and a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the question correction method according to any one of claims 1-8.

11. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to cause the computer to perform the question correction method according to any one of claims 1-8.

Images