Image recognition method, question answering and correcting method, and related devices, equipment and media

Abstract

The present application discloses an image recognition method, a method for correcting questions and related devices, equipment and media, wherein the image recognition method comprises: detecting the answer image to obtain a plurality of target areas; wherein the plurality of target areas at least include a first stem area; reorganizing the first stem area to obtain a second stem area of ​​each test question; wherein the second stem area includes a complete stem text of the test question to which the second stem area belongs; and then performing recognition based on the second stem area to obtain each stem text. The above scheme can improve the accuracy of obtaining the stem text.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image recognition method, a question-answering and grading method, and related devices, equipment, and media. Background Art

[0002] At present, when testing questions, it is usually necessary to conduct it on-site. For example, in the scenario of evaluating students' written test papers, teachers need to correct students' test papers and get the final results; in the scenario of companies' written tests for employees, the written test papers of employees need to be corrected before the final conclusion can be drawn, and so on.

[0003] The evaluation of test results is generally done by manual review, which is inefficient. As the time spent on manual review increases, it is inevitable that mistakes will occur due to factors such as fatigue. Therefore, how to conduct machine review has attracted more and more attention. However, one of the important prerequisites for machine review is to accurately identify the text of the question stem. In view of this, how to improve the accuracy of obtaining the text of the question stem has become an urgent problem to be solved. Summary of the invention

[0004] The main technical problem solved by this application is to provide an image recognition method, a question answering and correcting method and related devices, equipment and media, which can improve the accuracy of obtaining the question stem text.

[0005] In order to solve the above-mentioned technical problems, the first aspect of the present application provides an image recognition method, including: detecting the answer image to obtain a number of target areas; and the number of target areas include a first stem area and a question number area of ​​each test question; based on the positional relationship of the question number areas, reorganizing the first stem area to obtain a second stem area of ​​each test question; and the second stem area contains a complete stem text of the test question; and then performing recognition based on the second stem area to obtain each stem text.

[0006] In order to solve the above-mentioned technical problems, the second aspect of the present application provides a method for correcting answers, including: identifying the stem text of each test question in the answer image of the target object; and the stem text is obtained based on the image recognition method in the above-mentioned first aspect; obtaining the correction areas corresponding to each stem text belonging to the same test question; and then based on the correction marks in the correction area, obtaining the target object's answer status on each test question.

[0007] In order to solve the above-mentioned technical problems, the third aspect of the present application provides an image recognition device, including a detection module, a reorganization module and a recognition module. The detection module is used to detect the answer image to obtain a number of target areas; and the several target areas include a first stem area and a question number area of ​​each test question; the reorganization module is used to reorganize the first stem area based on the positional relationship of the question number area to obtain a second stem area of ​​each test question; and the second stem area contains the complete stem text of the test question; the recognition module is used to perform recognition based on the second stem area to obtain each stem text.

[0008] In order to solve the above-mentioned technical problems, the fourth aspect of the present application provides a question answer correction device, including an identification module, an acquisition module and a determination module; the identification module is used to identify the stem text of each test question in the answer image of the target object; and the stem text is obtained based on the image recognition method in the above-mentioned first aspect; the acquisition module is used to obtain the correction areas corresponding to each stem text belonging to the same test question; the determination module is used to obtain the target object's answer status on each test question based on the correction marks in the correction area.

[0009] In order to solve the above-mentioned technical problems, the fifth aspect of the present application provides an electronic device, including a memory and a processor coupled to each other, the memory storing program instructions, and the processor being used to execute the program instructions to implement the image recognition method in the above-mentioned first aspect or to implement the question grading method in the above-mentioned second aspect.

[0010] In order to solve the above technical problems, the sixth aspect of the present application provides a computer-readable storage medium storing program instructions that can be executed by a processor, wherein the program instructions are used to implement the image recognition method in the above first aspect or the question grading method in the above second aspect.

[0011] The above scheme detects the answer image to obtain several target areas; and the several target areas at least include the first stem area; and the first stem area is reorganized to obtain the second stem area of ​​each test question; wherein the second stem area contains the complete stem text of the test question to which the second stem area belongs; and then the second stem area is identified to obtain each stem text. On the one hand, the reorganization of the stem area helps to more conveniently identify the stem area, and on the other hand, the reorganized second stem area contains the complete stem text of the test question. Therefore, the accuracy of obtaining the stem text can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flowchart of an embodiment of the image recognition method of the present application;

[0013] Figure 2 is a schematic diagram of an embodiment of an answer image;

[0014] Figure 3 yes Figure 1 A schematic diagram of a flow chart of an embodiment of step S12;

[0015] Figure 4 It is a schematic diagram of an embodiment of the subject text;

[0016] Figure 5 It is a flowchart of an embodiment of the method for correcting answers to questions in this application;

[0017] Figure 6 This is a schematic diagram of the network structure of an embodiment of the image recognition model of the present application;

[0018] Figure 7 This is a schematic diagram of the codec architecture of an embodiment of the image recognition model of the present application;

[0019] Figure 8 It is a schematic diagram of the framework of an embodiment of the image recognition device of the present application;

[0020] Fig. 9 It is a schematic diagram of the framework of an embodiment of the apparatus for correcting questions in the present application;

[0021] Fig.10 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0022] Fig.11 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0023] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0024] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0025] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0026] See also Figure 1 , Figure 1 It is a flowchart of an embodiment of the image recognition method of the present application.

[0027] Specifically, the following steps may be included:

[0028] Step S11: Detect the answer sheet image to obtain a number of target regions.

[0029] In an implementation scenario, before detecting the answer sheet image, the answer sheet image can be obtained according to the answer content. Exemplarily, the answer content can be photographed to obtain the answer sheet image, or the answer content can be scanned to obtain the answer sheet image. The method of obtaining the answer sheet image can be selected according to the actual situation and will not be specifically limited here.

[0030] In a specific implementation scenario, the answer content can be graded according to a preset rule. The preset rule can be selected according to the actual situation. Exemplarily, if the answer taker is a student, the preset rule can include different grading rules. Specifically, for example, "√" indicates correct; "O", "×" indicate completely wrong; "?" indicates there is a question and needs to be corrected; "乄" indicates partially wrong and needs to be corrected; "☆" indicates good. After the student finishes answering the homework (test paper, teaching aid book, etc.), it can be graded by the student (teacher, parent) according to the agreed grading rules. However, in the actual grading process, there may be uncertain factors. For example, in the scoring area of the page, there may be grading symbols other than the agreed ones. Therefore, the answer content can be marked according to the preset rule to distinguish different answer areas. Exemplarily, the red pen score that appears can be marked as "dignum"; the question stem area can be marked as "prtarea"; the question number area can be marked as "quenum", and all other handwritten symbols that appear are marked as the "other" category. The preset grading rule for the answer content can be set according to the actual situation and will not be specifically limited here.

[0031] Please refer to Figure 2 , Figure 2 which is a schematic diagram of an embodiment of the answer sheet image. As Figure 2 described, the representation of the target region in the figure is a rectangle, and the target region can also be represented as a circle. The representation method of the target region can be set according to the actual situation and will not be specifically limited here.

[0032] In an implementation scenario, before detecting the answer image and obtaining several target areas, the gaps between characters in the answer image can also be filled. Since there are gaps between normal typesetting fonts, there will also be gaps between handwriting area fonts, and there will also be line breaks between paragraphs, etc., illustratively, the gaps between characters in the answer image can be filled by morphological processing of the answer image. By filling the gaps between characters in the answer image, the text lines in the answer image can be made easier to identify. The filled answer image can also be edge detected to obtain the detection results. The purpose of edge detection is to identify points in the image where the brightness changes significantly. Significant changes in image attributes usually reflect important events and changes in the attributes. Edge detection of the answer image can make the standardization of the answer image meet the requirements. illustratively, the detection algorithm can use Sobel edge detection algorithm, Gaussian filtering, etc. The specific edge detection algorithm is not specifically limited here. For edge detection results, including detection results that do not include edge information and detection results that include edge information, in response to the detection result not including edge information, a prompt is given to reacquire the answer image. If the edge detection result does not include edge information, it indicates that the answer image is abnormal. It may be that the content of the answer image is too blurry, or it may be that the light is abnormal when acquiring the answer image, resulting in a blank answer image. Therefore, the answer image is reacquired to avoid as much as possible the situation where work delays are caused by abnormalities found when detecting the answer image, thereby improving work efficiency as much as possible. The above method, by preprocessing the answer image, can, on the one hand, standardize the text of the answer image, and on the other hand, make the standardization of the answer image meet the requirements, which is helpful for reorganizing and identifying the answer image, thereby improving work efficiency.

[0033] In an implementation scenario, several target areas are obtained by detecting the answer image. The semantic segmentation model can be used to detect the answer image. Semantic segmentation is a basic task in computer vision. In semantic segmentation, the visual input needs to be divided into different semantically interpretable categories. For example, in outdoor scenes, it is necessary to distinguish all pixels belonging to cars in the image and process these pixels; in indoor scenes, it is necessary to distinguish all pixels belonging to furniture in the image and process them; in the answer image, it is necessary to distinguish different target areas in the image and distinguish different types of target areas. In practical applications, the semantic segmentation model can be selected according to the actual application scenario, and no specific limitation is made here.

[0034] In the disclosed embodiment, several target areas include the first stem area and question number area of ​​each test question. In actual applications, the target area may include the first stem area of ​​each test question, and the first stem area represents the stem area displayed in the answer image. It should be noted that the first stem area may be displayed across regions, and there may also be problems such as overlapping text lines; the target area may include the final score of each test question, and the final score may be the same as the score verification of each test question; the target area may also include a question number area, and the question number area may also be divided into different categories of question numbers. The target area can be selected according to actual application needs and is not specifically limited here.

[0035] Step S12: Reorganize the first stem area to obtain the second stem area of ​​each test question.

[0036] In an implementation scenario, the question number category of the question number contained in the question number area is obtained, and based on the question number category of the question number, the positional relationship between the question numbers is determined, and the first stem area corresponding to each question number is obtained, and the positional relationship includes whether there is a nested relationship between the question numbers; if there is a nested relationship between the question numbers and the first stem areas corresponding to the question numbers are not displayed across regions, that is, there is a nested relationship between at least two question numbers, and the first stem area corresponding to the latter question number in the nested relationship is not displayed across regions, then the first stem area corresponding to the question numbers is directly determined as the second stem area; if there is no nested relationship between the question numbers and the first stem areas corresponding to the question numbers are displayed across regions, that is, there is no nested relationship between at least two question numbers, if the first stem area corresponding to any of the latter question numbers is displayed across regions at this time, then the first stem area corresponding to the question numbers is directly determined as the second stem area. It should be noted that when one of the conditions of the nested relationship between at least two question numbers and the cross-region display of the first stem area corresponding to the latter question number is not met, the first stem area corresponding to the question numbers is directly determined as the second stem area.

[0037] In one implementation scenario, the question number category of the question number contained in the question number area is obtained, and then the second question stem area of ​​each test question is obtained based on the first question stem area and the question number category. The question number contained in the question number area is identified, and then the question number category is determined. The question number category can be obtained by using the recognition model encoding and decoding structure, etc. The method for obtaining the question number category can be selected according to the actual application, and is not specifically limited here. The recognition results of question number categories are diverse. For example, when the question number range is "0~9", there are 10 corresponding question number categories, when the question number range is ")", there is 1 corresponding question number category, when the question number range is ">", there is 1 corresponding question number category, when the question number range is "1~9", there are 10 corresponding question number categories, when the question number range is "a~g", there are 7 corresponding question number categories, when the question number range is "A~G", there are 7 corresponding question number categories, when the question number range is "UNK", there is 1 corresponding question number category, when the question number range is "a~z", there are 26 corresponding question number categories, and the specific form of the question number range can be selected according to actual conditions, and no specific limitation is made here. Based on the first question stem area and the question number category, the second question stem area of ​​each test question is reorganized, wherein the second question stem area contains the complete question stem text of the test question, and then each question stem text in the second question stem area can be more conveniently identified. The above method, based on the reorganization of the first stem area and the question number category to obtain the second stem area of ​​each test question, can obtain a more complete stem area, which facilitates the acquisition of each stem text.

[0038] In a specific implementation scenario, as mentioned above, based on the first question stem area and the question number category, in order to obtain the second question stem area of ​​each test question, first obtain the first question stem area corresponding to each question number, and based on the question number category of the question number, determine the positional relationship between the question numbers; and the positional relationship includes whether there is a nested relationship between the question numbers, and when there is a nested relationship between the question numbers, the question numbers belong to the same test question; you can first obtain the first question stem area corresponding to each question number, and then determine the positional relationship between the question numbers based on the question number category of the question number; you can also first determine the positional relationship between the question numbers based on the question number category, and then obtain the first question stem area corresponding to each question number. It should be noted that the specific order will not affect the result of obtaining the second question stem area of ​​the test question. In actual applications, you can choose according to the situation and there is no limitation here. Based on whether there is a nested relationship between the question numbers, the question stem area corresponding to each question number can be distinguished. For example, after obtaining the question number, it is necessary to determine the positional relationship between the question numbers. First, determine the number of question number categories M. The corresponding page has the most question number nesting range of M layers, and the outer layout of the question number is consistent. The layout type can be traversed. According to the different number of layers of each question number, the positional relationship between the question numbers can be determined, and then determine whether there is a nested relationship between the question numbers; first, the question stem text can be divided into columns. A page will have multiple question number categories and multiple columns. The question number at the upper left corner of the question stem text is the baseline question number position. Find the leftmost edit in each column, find the question number position at the leftmost of the column, and determine the category of the question number, for example, such as uppercase numbers, lowercase numbers, letters, etc. Traverse the left position of the column to find the question number belonging to the same category, and then determine the question number category. For example, if after searching, it is determined that the first category of question number is marked as question number category 1, then find the second category at the bottom right of question number category 1, and the maximum number of categories found is M. Then the category with the largest combined question number is 1_M-1_M-2_M-3…M, and then the outer question numbers of the adjacent columns are compared and judged to determine whether the first question stem areas corresponding to the question numbers are displayed across regions. In response to the nested relationship between the question numbers and the first question stem areas corresponding to the question numbers being displayed across regions, the first question stem areas corresponding to the question numbers are spliced ​​to obtain the second question stem areas of the test questions to which the question numbers belong. In the above method, by splicing the first question stems displayed across regions, the recognition results of the spliced ​​second question stem areas are more accurate, thereby improving the degree of mastery of the target object's learning situation.

[0039] See also Figure 3 , Figure 3 yes Figure 1 The flowchart of step S12 in the embodiment of FIG. 1 is shown in FIG. 1 . Specifically, the steps may include:

[0040] Step S31: Based on the positional relationship of the question number areas, the first question stem area is reorganized to obtain the second question stem area of ​​each question.

[0041] Step S32: Perform text line detection on the second question stem area to obtain a plurality of text line areas in the second question stem area.

[0042] In an implementation scenario, text line detection is performed on the second question stem area to determine the text line area in the second question stem area. The text line area can be divided according to each text line. The text line area can also be divided according to the question number category. The text line area can be divided according to a fixed number of text lines. The division form of the text line area can be selected according to actual conditions and is not specifically limited here.

[0043] Step S33: Based on the contour points of each text line area in the second question stem area, the text line areas in the second question stem area are reorganized to obtain a new second question stem area.

[0044] In one implementation scenario, based on the contour points of each text line area, the overlap between each text line area is detected. The overlap between each text line area can be detected based on the edge points of the contour points of each text line area. For example, the edge points of the contour points can be used to form a boundary line for detection. When the boundary line has an intersection, it indicates that there is overlap between text lines. The overlap between each text line area can also be detected based on the area formed by the contour points of each text line area. For example, the contour points of each text line area form a closed area with an empty interior. If the closed area has an intersection, it indicates that there is overlap between text lines. The overlap between each text line area can also be detected based on an image recognition algorithm. The specific detection method can be selected based on actual conditions and is not limited here. The order of detecting the overlap between each text line area can be detected line by line or based on a fixed number of lines. The specific detection order can be selected based on actual conditions and is not specifically limited here. After determining the overlap result between the text line areas, in response to the overlap between the text line areas satisfying the second condition, the text line areas are arranged to obtain new text line areas. The arrangement method of the text line areas can be adjusted according to the specific situation. For example, if the overlap is as high as 90%, the text line area with the lower height is moved in a specified direction. The specific value of the movement can be set according to different adaptation scenarios and is not specifically limited here. In the above method, the overlapping area is detected by contour points, and the text lines are rearranged. The new text line area can more clearly represent the content of the text line area, thereby further improving the accuracy of image recognition.

[0045] In a specific implementation scenario, the second condition may be that there is overlap between text line areas; it may also be that there is an intersection between the boundary lines formed by the contour points; it may also be that the overlap between the text line areas meets a preset threshold, and the specific threshold may be set according to the usage scenario. The second condition may be set according to actual conditions and is not specifically limited here.

[0046] In an implementation scenario, as described above, based on the contour points of each text line area in the second stem area, the text line areas in the second stem area are reorganized. In order to obtain a new second stem area, firstly, based on the contour points of each text line area in the second stem area, a first angle value characterizing the relative inclination relationship between each text line area is obtained. Based on the fact that the angle value does not satisfy the first condition, it is determined that the current second stem area is a new second stem area.

[0047] In an implementation scenario, as described above, based on the contour points of each text line area in the second stem area, the text line areas in the second stem area are reorganized. In order to obtain a new second stem area, based on the contour points of each text line area in the second stem area, a first angle value representing the relative inclination relationship between the text line areas is obtained. First, first coordinates and second coordinates representing the edge positions on both sides of the text line area are obtained; and the first coordinate and the second coordinate are respectively obtained based on the coordinate values ​​of the contour points located on both sides of the text line area. The text line areas on both sides are obtained, and the positions of the text lines on both sides can be determined based on the contour points of the text lines on both sides. The calculation can be performed based on the coordinate values ​​of the contour points located on both sides of the text line area. Here, the calculation method can obtain the average value of the coordinates, and the calculation method can also take the intermediate value. The specific calculation method can be selected according to the actual situation and is not specifically limited here. Exemplarily, if the first coordinate and the second coordinate of contour 1 are A(x11, y11) and B(x12, y12), respectively, and the first coordinate and the second coordinate of contour 2 are C(x21, y21) and D(x22, y22), respectively, the first coordinate and the second coordinate are obtained by taking the average value of the coordinate values ​​of the contour points on the edges of the text line area. Based on the first coordinate and the second coordinate of each text line area, the second angle value of the angle between the connecting line between any two edge positions and the horizontal direction is obtained. As mentioned above, if the first coordinate and the second coordinate are obtained, the angle between the connecting line between any two edge positions and the horizontal direction is obtained as the second angle value, and the second angle values ​​are respectively recorded as angle1112, angle1122, angle1121, angle1222, angle1221, angle2122. Then, statistics are performed based on the second angle value to obtain the first angle value. The statistical method for the second angle value can be selected according to the specific situation. Exemplarily, the second angle value can be averaged to obtain the first angle value; the median of the second angle value can also be taken to obtain the first angle value. The statistical method for the second angle value can be selected according to the actual situation and is not specifically limited here. In the above method, by obtaining the first angle value representing the relative tilt relationship between each text line area, the tilt angle of each text line area can be determined, thereby improving the understanding of the specific situation of the answer image.

[0048] Furthermore, if the angle value does not meet the first condition, the current second stem area is directly determined as the new second stem area; if the angle value meets the first condition, the text line area is spliced ​​to obtain a new second stem area. The method of splicing the text lines needs to be selected according to the actual situation. For example, if the text lines are the same line but are not displayed continuously, the text lines at the back need to be spliced ​​behind the previous text line; if the text lines are not the same line but the angle also meets the first condition, the next text line is spliced ​​behind the previous text line, and the next blank text line is deleted. The deletion method can move the text line behind as a whole, or it can be selected according to the actual situation. The text line splicing method can be selected according to the actual situation, and no specific limitation is made here. The above method, by splicing the text lines to obtain a new second stem area, helps to understand the content of the answer image more clearly and further improves the mastery of the target object's learning situation.

[0049] In a specific implementation scenario, the first condition may be that the second angle value is greater than the preset angle value, and the preset angle value may be set according to the actual situation. For example, if the second angle value is obtained by taking the average value, when the preset angle value is 25 degrees, most of the tilt situations will be solved, and the accuracy rate can reach 99.5%, when the preset angle value is 20 degrees, the accuracy rate can reach 97.3%, and when the preset angle value is 15 degrees, the accuracy rate can reach 95.2%; if the second angle value is obtained by taking the median value, the preset angle value can be appropriately reduced. The first condition can be set according to the actual situation and is not specifically limited here.

[0050] Step S13: Recognize based on the second question stem area to obtain each question stem text.

[0051] In an implementation scenario, the recognition of the second question stem area can be achieved through a semantic segmentation model, and then the text of each question stem can be obtained. The semantic segmentation model can be selected according to the actual situation. The obtained text of each question stem can be distinguished based on different question number types. The way to distinguish the text of each question stem can be set according to the actual situation, and no specific limitation is made here. Please refer to Figure 4 , Figure 4 It is a schematic diagram of an embodiment of the subject text, such as Figure 4 As described above, after identifying the answer image, each question stem text 40 is finally obtained.

[0052] The above scheme detects the answer image to obtain several target areas; and the several target areas at least include the first stem area; and the first stem area is reorganized to obtain the second stem area of ​​each test question; wherein the second stem area contains the complete stem text of the test question to which the second stem area belongs; and then the second stem area is identified to obtain each stem text. On the one hand, the reorganization of the stem area helps to more conveniently identify the stem area, and on the other hand, the reorganized second stem area contains the complete stem text of the test question. Therefore, the accuracy of obtaining the stem text can be improved.

[0053] See also Figure 5 , Figure 5 This is a flow chart of an embodiment of the method for correcting answers in this application. Specifically, it may include the following steps:

[0054] Step S51: Identify the text of each test question in the answer image of the target object.

[0055] In an implementation scenario, before identifying the stem text of each test question in the answer image of the target object, the answer image of the target object can be preprocessed. The purpose of the preprocessing is to reduce the interference of the external environment on the answer image and improve the accuracy of identifying the answer image of the target object. The preprocessing method can be selected according to the actual situation and is not specifically limited here. For example, the answer image can be morphologically processed first to fill the gaps in the text, and then the processed answer image can be subjected to Sobel edge detection. The operator can be set to [-1, 0, 1, -2, 0, 2, -1, 0, 1] to extract the vertical edge. If the edge information cannot be extracted, an exception is thrown, indicating that the answer image of the target object may be due to the tilt during the scanning process or the background interference during the photo shooting process. Problems such as light problems cause the answer image content to be unclear and need to be re-acquired. In order to improve the recognition of the stem text of each test question in the answer image of the target object, an image recognition model can be pre-trained so that the stem text of each test question can be recognized by the image recognition model. After preprocessing the answer image, it is necessary to obtain the text of each test question. In this process, the network model can be used to obtain multiple segmentation structures in the answer image, such as multiple grading types, question numbers, question stems, etc. The network model can be a fully connected network structure (Fully Connected, FC), the network model can also be a convolutional neural network (Convolutional Neural Network, CNN), and the network model can also be a recurrent neural network (Recurrent Neural Network, RNN). The specific type of the network model is not limited here and can be selected according to actual conditions.

[0056] For further information, see Figure 6 , Figure 6 This is a schematic diagram of the network structure of an embodiment of the image recognition model of the present application. After preprocessing the answer image, the answer image can be normalized into 3 channels, and then scaled according to the long side (empirical value), and the maximum processing size is set at the same time to avoid crossing the boundary. The value of each pixel point on each channel is subtracted from the mean and multiplied by the variance, for example: mean, variance is an empirical value. In this process, the answer image can also be normalized into a single channel. The number of channels can be selected according to actual conditions and is not specifically limited here. After that, the new pixel values ​​are re-integrated into a new set of feature matrices. The feature matrix can be directly input Figure 6 In the convolutional neural network, the corresponding ID number is output. The ID number has a mapping relationship with multiple segmentation structures. For example, ID number 1 can correspond to the correction type, ID number 1 can correspond to the question number, ID number 10 corresponds to question numbers "one to nine", and ID number 11 corresponds to question numbers "0 to 9". The mapping relationship between the ID number and multiple segmentation structures can be set according to actual conditions, and no specific limitation is made here. It should be noted that the mapping relationship between the ID number and the multiple segmentation structures is set before the network structure recognizes the answer image, so that the network structure can directly output the corresponding ID number after recognizing the answer image.

[0057] See also Figure 7 , Figure 7 This is a schematic diagram of the codec architecture of an embodiment of the image recognition model of the present application. After obtaining multiple correction types, question numbers, question stems, etc., it is also possible to verify whether the question number recognition result in the answer image is correct. Before verifying the question number recognition result in the answer image, the answer image can be preprocessed. The answer image is processed according to 3 channels, the height is scaled to 60, the value is normalized to 0-1, and then stretched to 1 in the height direction to form a set of feature data. In this process, the number of channels and height scaling can be selected according to actual conditions, and no specific restrictions are made here. Input the feature data into the encoder-decoder structure (such as Figure 7 ), and obtain the corresponding ID number. The ID number has a mapping relationship with multiple question numbers. For example, the ID number 1 can correspond to question numbers "one to nine", and the ID number 10 corresponds to question numbers "0 to 9". The mapping relationship between the ID number and multiple question numbers can be set according to actual conditions, and no specific limitation is made here. It should be noted that the mapping relationship between the ID number and the multiple question numbers is set before verifying the question number recognition result in the answer image, and then the corresponding ID number can be directly output after the answer image is recognized.

[0058] In one implementation scenario, the process of obtaining the question text can refer to the relevant description of the aforementioned image recognition, which will not be repeated here.

[0059] In an implementation scenario, the target object can be an employee, a student, etc. The number of target objects can be one or multiple, and the target object can be selected according to the actual situation, which is not specifically limited here.

[0060] In an implementation scenario, the answer sheet image can be the test content of company employees, or the weekend quiz content of students, or the answer sheet content jointly completed by multiple respondents. The answer sheet image can be selected according to the actual situation and will not be elaborated one by one here.

[0061] Step S52: Obtain the marking areas corresponding to the respective stem texts of the same test questions.

[0062] In an implementation scenario, an answer sheet image of the same test question is recognized, and the marking areas corresponding to the respective stem texts are obtained based on the answer sheet image of the same test question.

[0063] Step S53: Based on the marking notations in the marking areas, obtain the answering situations of the target object on each test question.

[0064] In an implementation scenario, in order to obtain the answering situations of the target object on each test question based on the marking notations in the marking areas, first, based on the marking notations in the marking areas, count the number of various marking notations in the test questions. Among them, the marking notations are the marking notations after marking based on a preset rule. Then, based on the number of various marking notations in the test questions, obtain the answering situations of the target object on each test question. The answering situation can be the mastery situation of the target object on each test question, or the mastery situation of the target object on various types of test questions. Among them, the mastery situation of the target object on each test question can be reflected by the number of various marking notations, and the mastery situation of the target object on each test question can also be scored by calculating with various marking notations and reflected by the scoring situation. Exemplarily, if the marking notation is "√", it is recorded as 2 points; if the marking notations are "O", "×", it is recorded as 0 points; if the marking notations are "?", "乄", it is recorded as 1 point. Finally, for the total score value X of all scores, if the number of marking notations is doubled as the score value Y, the final answering situation can be determined according to the ratio of X to Y. Exemplarily, if the ratio is 0.8, it indicates that the target object has mastered 80% of the knowledge points on this test question. A threshold can also be preset. Exemplarily, the threshold is set to 95%. When the ratio is not less than this threshold, it indicates that the target object has generally mastered the knowledge points in the test question. Otherwise, the knowledge points in the test question need to be consolidated and learned. The preset threshold can be set according to the actual situation and is not specifically limited here. The mastery situation of the target object on each test question can be selected according to the actual situation and is not specifically limited here. Through the above method, by counting the number of various marking notations in the test questions, the answering situations of the target object on each test question are further determined, and the understanding of the target object is further improved.

[0065] The above scheme detects the answer image to obtain several target areas; and the several target areas at least include the first stem area; and the first stem area is reorganized to obtain the second stem area of ​​each test question; wherein the second stem area contains the complete stem text of the test question to which the second stem area belongs; and then the second stem area is identified to obtain each stem text. On the one hand, the reorganization of the stem area helps to more conveniently identify the stem area, and on the other hand, the reorganized second stem area contains the complete stem text of the test question. Therefore, the accuracy of obtaining the stem text can be improved.

[0066] See also Figure 8 , Figure 8 It is a schematic diagram of the framework of an embodiment of the image recognition device of the present application. The image recognition device 60 includes a detection module 61, a reorganization module 62 and a recognition module 63. Among them, the detection module 61 is used to detect the answer image to obtain a plurality of target areas; and the plurality of target areas at least include a first stem area; the reorganization module 62 is used to reorganize the first stem area to obtain a second stem area for each test question; and the second stem area contains a complete stem text of the test question to which the second stem area belongs; the recognition module 63 is used to perform recognition based on the second stem area to obtain each stem text.

[0067] The above scheme, on the one hand, helps to more conveniently identify the question stem area because the question stem area is reorganized, and on the other hand, because the reorganized second question stem area contains the complete question stem text of the test question, it can improve the accuracy of obtaining the question stem text.

[0068] In some disclosed embodiments, the identification module 63 includes an acquisition submodule, which is used to obtain the question number category of the question number contained in the question number area; the identification module 63 also includes a recombination submodule, which is used to reorganize the first question stem area and the question number category to obtain the second question stem area for each test question.

[0069] Therefore, by reorganizing the second stem area of ​​each question based on the first stem area and the question number category, a more complete stem area can be obtained, which facilitates the acquisition of each stem text.

[0070] In some disclosed embodiments, the recombination submodule includes an acquisition unit, which is used to acquire the first stem areas corresponding to each question number, and determine the positional relationship between the question numbers based on the question number category of the question number; and the positional relationship includes whether there is a nested relationship between the question numbers, and when there is a nested relationship between the question numbers, the question numbers belong to the same test question; the recombination submodule also includes a recombination unit, which is used to respond to the existence of a nested relationship between the question numbers and the cross-region display of the first stem areas corresponding to the question numbers, and splice the first stem areas corresponding to the question numbers to obtain the second stem area of ​​the test question to which the question number belongs.

[0071] Therefore, by splicing the first question stem displayed across regions, the recognition result of the spliced ​​second question stem region is made more accurate, thereby improving the grasp of the target object's learning situation.

[0072] In some disclosed embodiments, the image recognition device 60 also includes a determination module, which is used to perform text line detection on the second question stem area to obtain a plurality of text line areas in the second question stem area; and then reorganize the text line areas in the second question stem area based on the contour points of each text line area in the second question stem area to obtain a new second question stem area.

[0073] Therefore, detecting contour points and reorganizing the second question stem area helps to improve the integrity of answer image recognition.

[0074] In some disclosed embodiments, the determination module includes an acquisition submodule, which is used to acquire a first angle value characterizing the relative inclination relationship between each text line area based on the contour points of each text line area in the second question stem area; the determination module also includes a determination submodule, which is used to splice the text line areas based on the angle value satisfying the first condition to obtain a new second question stem area.

[0075] Therefore, by splicing the text lines to obtain a new second question stem area, it is helpful to understand the content of the answer image more clearly and further improve the grasp of the target object's learning situation.

[0076] In some disclosed embodiments, the determination module also includes a detection submodule, which is used to detect the overlap between each text line area based on the contour points of each text line area; in response to the overlap between the text line areas satisfying the second condition, the text line areas are arranged to obtain new text line areas.

[0077] Therefore, the overlapping area is detected through the contour points, and the text lines are rearranged. The new text line area can more clearly represent the content of the text line area, thereby further improving the accuracy of image recognition.

[0078] In some disclosed embodiments, the acquisition submodule includes an acquisition unit, which is used to acquire first coordinates and second coordinates that respectively represent the edge positions on both sides of the text line area; and the first coordinate and the second coordinate are respectively obtained based on the coordinate values ​​of the contour points located on the edges of the text line area on both sides; the acquisition submodule includes a calculation unit, which is used to obtain a second angle value of the angle between the line between any two side edge positions and the horizontal direction based on the first coordinates and the second coordinates of each text line area; the acquisition submodule also includes a statistical unit, which is used to perform statistics based on the second angle value to obtain the first angle value.

[0079] Therefore, by obtaining the first angle value representing the relative tilt relationship between the text line regions, the tilt angle of each text line region can be determined, thereby improving the understanding of the specific situation of the answer image.

[0080] In some disclosed embodiments, the image recognition device 60 also includes a filling module, which is used to fill the gaps between characters in the answer image; perform edge detection on the filled answer image to obtain a detection result; in response to the detection result not containing edge information, prompt to re-acquire the answer image.

[0081] Therefore, by preprocessing the answer image, on the one hand, the text of the answer image can be normalized, and on the other hand, the standardization degree of the answer image can meet the requirements, which is helpful for reorganizing and identifying the answer image, thereby improving work efficiency.

[0082] See also Fig. 9 , Fig. 9 It is a schematic diagram of the framework of an embodiment of the answer correction device of the present application. The answer correction device 70 includes a recognition module 71, an acquisition module 72 and a determination module 73. Among them, the recognition module 71 is used to recognize the stem text of each test question in the answer image of the target object, and the stem text is obtained based on any of the above-mentioned image recognition methods; the acquisition module 72 is used to obtain the correction areas corresponding to each stem text belonging to the same test question; the determination module 73 is used to obtain the answer status of the target object on each test question based on the correction marks in the correction area.

[0083] The above scheme, on the one hand, helps to more conveniently identify the question stem area because the question stem area is reorganized, and on the other hand, because the reorganized second question stem area contains the complete question stem text of the test question, it can improve the accuracy of obtaining the question stem text.

[0084] In some disclosed embodiments, the determination module 73 includes a statistical submodule, which is used to count the number of various correction marks in the test questions based on the correction marks in the correction area; the determination module 73 also includes a determination submodule, which is used to obtain the target subject's answer status on each test question based on the number of various correction marks in the test questions.

[0085] Therefore, by counting the number of various types of correction marks in the test questions, and then determining the target subject's answers on each test question, the understanding of the target subject is further improved.

[0086] See also Fig.10 , Fig.10 1 is a schematic diagram of a framework of an embodiment of an electronic device of the present application. The electronic device 80 includes a memory 81 and a processor 82 coupled to each other, the memory 81 stores program instructions, and the processor 82 is used to execute the program instructions to implement the steps in any of the above-mentioned image recognition methods or answer correction method embodiments. Specifically, the electronic device 80 may include but is not limited to: a desktop computer, a laptop computer, a server, a mobile phone, a tablet computer, etc., which are not limited here.

[0087] Specifically, the processor 82 is used to control itself and the memory 81 to implement the steps in any of the above-mentioned image recognition methods or the answer correction method embodiments. The processor 82 can also be called a CPU (Central Processing Unit). The processor 82 may be an integrated circuit chip with signal processing capabilities. The processor 82 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 82 can be implemented by an integrated circuit chip.

[0088] The above scheme, on the one hand, helps to more conveniently identify the question stem area because the question stem area is reorganized, and on the other hand, because the reorganized second question stem area contains the complete question stem text of the test question, it can improve the accuracy of obtaining the question stem text.

[0089] See also Fig.11 , Fig.11The computer-readable storage medium 90 stores program instructions 91 that can be executed by a processor, and the program instructions 91 are used to implement the steps in any of the above-mentioned image recognition methods or answer correction method embodiments.

[0090] The above scheme, on the one hand, helps to more conveniently identify the question stem area because the question stem area is reorganized, and on the other hand, because the reorganized second question stem area contains the complete question stem text of the test question, it can improve the accuracy of obtaining the question stem text.

[0091] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0092] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0093] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0094] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0095] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0096] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. An image recognition method, It is characterized in that include: Detecting the answer image to obtain a plurality of target areas; wherein the plurality of target areas at least include a first question stem area and a question number area; Obtain the question number category of the question number contained in the question number area; Based on the first stem area and the question number category, reorganize to obtain a second stem area for each test question; wherein the second stem area contains a complete stem text of the test question to which the second stem area belongs; Recognize based on the second question stem area to obtain each question stem text; Wherein, after reorganizing the second stem area of ​​each test question based on the first stem area and the question number category, and before identifying the second stem area to obtain each stem text, the method further includes: Performing text line detection on the second question stem area to obtain a plurality of text line areas in the second question stem area; Based on the contour points of each of the text line regions in the second question stem region, obtaining a first angle value representing a relative tilt relationship between each of the text line regions; Based on the first angle value satisfying the first condition, the text line areas are spliced ​​to obtain a new second question stem area.

2. The method according to claim 1, It is characterized in that The reorganization based on the first question stem area and the question number category to obtain the second question stem area of ​​each of the test questions includes: Obtaining the first question stem area corresponding to each of the question numbers, and determining the positional relationship between the question numbers based on the question number categories of the question numbers; wherein the positional relationship includes whether there is a nested relationship between the question numbers, and when the nested relationship exists between the question numbers, the question numbers belong to the same test question; In response to the nested relationship between the question numbers and the cross-region display of the first question stem areas corresponding to the question numbers, the first question stem areas corresponding to the question numbers are spliced ​​to obtain the second question stem area of ​​the test question to which the question number belongs.

3. The method according to claim 1, It is characterized in that Before acquiring the first angle value representing the relative inclination relationship between the text line regions based on the contour points of the text line regions in the second question stem region, the method further includes: Based on the contour points of each of the text line areas, detecting and obtaining the overlap between the text line areas; In response to the overlap between the text line regions satisfying a second condition, the text line regions are arranged to obtain new text line regions.

4. The method according to claim 1, It is characterized in that The obtaining of a first angle value representing a relative tilt relationship between the text line regions includes: Obtaining first coordinates and second coordinates representing the positions of the edges of both sides of the text line area, respectively; wherein the first coordinates and the second coordinates are respectively obtained based on the coordinate values ​​of the contour points located at the edges of both sides of the text line area; Based on the first coordinates and the second coordinates of each of the text line regions, a second angle value of an angle between a line connecting the edge positions on any two sides and a horizontal direction is obtained; Statistics are performed based on the second angle value to obtain the first angle value.

5. The method according to claim 1, It is characterized in that Before detecting the answer image to obtain a plurality of target areas, the method further includes: Filling the gaps between characters in the answer image; Performing edge detection on the filled answer image to obtain a detection result; In response to the detection result not including edge information, a prompt is provided to reacquire the answer image.

6. A method for marking answers. It is characterized in that include: Recognize the stem text of each test question in the answer image of the target object; wherein the stem text is obtained based on the image recognition method according to any one of claims 1 to 5; Obtaining the correction areas corresponding to each question stem text belonging to the same test question; Based on the correction marks in the correction area, the answer status of the target subject on each of the test questions is obtained.

7. The method according to claim 6, It is characterized in that The step of obtaining the answer status of the target subject on each of the test questions based on the correction marks in the correction area includes: Based on the correction marks in the correction area, counting the number of each type of correction marks in the test question; Based on the number of each type of correction marks in the test questions, the answer status of the target subject on each of the test questions is obtained.

8. An image recognition device, It is characterized in that include: A detection module, used to detect the answer image and obtain a plurality of target areas; wherein the plurality of target areas at least include a first question stem area and a question number area; A first acquisition submodule is used to acquire the question number category of the question number contained in the question number area; A reorganization submodule, configured to reorganize the second stem area of ​​each test question based on the first stem area and the question number category; wherein the second stem area contains the complete stem text of the test question to which the second stem area belongs; A recognition module, used for performing recognition based on the second question stem area to obtain each question stem text; Wherein, after the recombination submodule and before the identification module, the device further includes: A determination module, configured to perform text line detection on the second question stem area to obtain a plurality of text line areas in the second question stem area; A second acquisition submodule is used to acquire a first angle value representing a relative inclination relationship between each of the text line regions based on the contour points of each of the text line regions in the second question stem region; A determination submodule is used to splice the text line area based on the first angle value satisfying a first condition to obtain a new second question stem area.

9. A device for correcting answers, It is characterized in that include: A recognition module, used to recognize the stem text of each test question in the answer image of the target object; wherein the stem text is obtained based on the image recognition method according to any one of claims 1 to 5; An acquisition module, used for acquiring the correction areas corresponding to each question stem text belonging to the same test question; The determination module is used to obtain the answer status of the target subject on each of the test questions based on the correction marks in the correction area.

10. An electronic device, It is characterized in that It comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the image recognition method described in any one of claims 1 to 5, or to implement the answer correction method described in any one of claims 6 to 7.

11. A computer-readable storage medium, It is characterized in that Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the image recognition method described in any one of claims 1 to 5, or the answer correction method described in any one of claims 6 to 7.

Images