Question stem and answer mixed segmentation method, device, and storage medium

Abstract

The present invention provides a method, device, storage medium, and computer equipment for segmenting a mixed question stem and answer. First, based on the acquired question stem and answer layout of the image to be detected, the image to be detected is divided into multiple option boxes, and the type of the option box is determined to achieve automatic segmentation of the question stem and the answer, thereby reducing the workload of manual segmentation and greatly improving the efficiency of recording questions. Secondary detection is then performed on different types of option boxes, which can solve various problems of layout types of questions and answers, and the corresponding second detection result has a higher accuracy rate in segmenting the question stem and the answer. Therefore, after obtaining the second detection result, the second detection result is used to check whether there are any option boxes that have been missed in the first detection result. If so, the question stem and answer area of ​​the image to be detected are segmented based on the second detection result, so that the accuracy rate of the overall segmentation frame is further improved, and the segmentation effect is better.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, storage medium, and computer equipment for segmenting a question stem and a mixed answer. Background Art

[0002] In the educational scenario, question entry is a very important and tedious task, involving many links, which makes question entry not only inefficient but also costly. In recent years, AI technology has developed rapidly and has achieved great success in fields such as object detection and OCR. The introduction of OCR has solved the problem of text transcription in question entry, greatly improving the efficiency of question entry. However, when batch entering supplementary books or test questions, before transcribing the text area, the question and answer areas must be separated to achieve batch entry of questions.

[0003] In the prior art, since there are various types of layouts for questions and answers, some of them are conflicting, that is, in the case of the same layout distribution, some are answer areas, some are question stem areas, some are multiple questions, some are just one question, some divide the question into several parts, some are located next to other questions, and there are various distributions of illustrations and tables, etc., all of which bring great difficulties to the segmentation of questions and answers, resulting in inaccurate division of question and answer areas and low efficiency in question entry. Summary of the invention

[0004] The purpose of the present invention is to solve at least one of the above-mentioned technical defects, especially the technical defects in the prior art that the layout types of questions and answers are various, resulting in inaccurate division of question and answer areas and low question entry efficiency.

[0005] The present invention provides a method for segmenting a question stem and a mixed answer, the method comprising:

[0006] Acquire an image to be detected, and detect the question stem and answer layout of the image to be detected to obtain a first detection result;

[0007] Determine a plurality of option boxes and types of option boxes corresponding to the image to be detected according to the first detection result, and perform secondary detection on each type of option box to obtain a second detection result;

[0008] The second detection result is compared with the first detection result to determine whether there are any missed option boxes in the first detection result. If so, the question stem and answer area of ​​the image to be detected are segmented according to the second detection result.

[0009] Optionally, the method for dividing the question stem and the mixed answer further includes:

[0010] Determine whether there is any missed option box in the first detection result;

[0011] If not, the question stem and answer area of ​​the image to be detected are segmented according to the first detection result.

[0012] Optionally, the types corresponding to the various option boxes include a composite question box without a common question stem, a question stem box with illustrations, and an answer sheet area box;

[0013] The step of performing secondary detection on each type of option box to obtain a second detection result includes:

[0014] Traversing the first test results, and determining the composite question frames without common question stems in the first test results;

[0015] The sub-question number is detected in the area where the composite question frame without a common question stem is located, and the detected sub-question number is identified to determine the corresponding question number and answer area.

[0016] Optionally, the step of performing secondary detection on each type of option box to obtain a second detection result further includes:

[0017] Traversing the first test result, and determining a question stem frame with an illustration in the first test result;

[0018] The illustration area of ​​the question stem frame to be illustrated is covered, and the question stem is identified on the covered question stem frame with illustration to determine the corresponding question stem area.

[0019] Optionally, the step of performing secondary detection on each type of option box to obtain a second detection result further includes:

[0020] Traversing the first test result, and determining the answer sheet area frame in the first test result;

[0021] The answer sheet area frame is subjected to option detection to determine the corresponding option block information.

[0022] Optionally, the step of comparing the second detection result with the first detection result to determine whether there is a missed detection option box in the first detection result includes:

[0023] The option boxes corresponding to the question number and the answer area are compared with the option boxes in the first detection result to determine whether there are any option boxes that are missed in the first detection result.

[0024] Optionally, the step of comparing the second detection result with the first detection result to determine whether there is a missed detection option box in the first detection result includes:

[0025] The option boxes corresponding to the question stem area are compared with the option boxes in the first detection result to determine whether there are any option boxes missed in the first detection result.

[0026] Optionally, the step of comparing the second detection result with the first detection result to determine whether there is a missed detection option box in the first detection result includes:

[0027] The option box corresponding to the option block information is compared with the option box in the first detection result to determine whether there is any missed option box in the first detection result.

[0028] The present invention also provides a question stem and answer mixed segmentation device, comprising:

[0029] A first detection module is used to obtain an image to be detected, and detect the question stem and answer layout of the image to be detected to obtain a first detection result;

[0030] A second detection module is used to determine a plurality of option boxes and types of option boxes corresponding to the image to be detected according to the first detection result, and perform secondary detection on each type of option box to obtain a second detection result;

[0031] The segmentation module is used to compare the second detection result with the first detection result to determine whether there are any missed option boxes in the first detection result. If so, the question stem and answer area of ​​the image to be detected are segmented according to the second detection result.

[0032] The present invention also provides a storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method for dividing the question stem and the mixed answer as described in any of the above embodiments.

[0033] The present invention also provides a computer device, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method for dividing the question stem and the mixed answer as described in any of the above embodiments.

[0034] It can be seen from the above technical solutions that the embodiments of the present invention have the following advantages:

[0035] The present invention provides a method, device, storage medium, and computer equipment for segmenting a mixed question stem and answer, comprising: acquiring an image to be detected, and detecting the question stem and answer layout of the image to be detected to obtain a first detection result; determining multiple option boxes and the types of option boxes corresponding to the image to be detected based on the first detection result, and performing secondary detection on each type of option box to obtain a second detection result; comparing the second detection result with the first detection result to determine whether there are any option boxes that have been missed in the first detection result, and if so, segmenting the question stem and answer area of ​​the image to be detected based on the second detection result.

[0036] In the present invention, firstly, according to the obtained layout of the question stem and answer of the image to be detected, the image to be detected is divided into multiple option boxes, and the type of the option box is determined to realize automatic segmentation of the question stem and the answer, thereby reducing the workload of manual segmentation and greatly improving the efficiency of recording questions; and then a secondary test is performed on different types of option boxes respectively, which can solve various problems of the layout types of questions and answers, and the corresponding second test result has a higher accuracy rate in the segmentation of the question stem and the answer; therefore, after obtaining the second test result, the second test result is used to check whether there are any option boxes missed in the first test result, and if so, the question stem and the answer area of ​​the image to be detected are segmented according to the second test result, so that the accuracy rate of the overall segmentation frame is further improved, and the segmentation effect is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 A schematic diagram of an application environment provided by an embodiment of the present invention;

[0039] Figure 2 A schematic diagram of a flow chart of a method for segmenting a question stem and a mixed answer provided in an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of the first structure of a composite question frame without a common question stem provided by an embodiment of the present invention;

[0041] Figure 4 A second structural schematic diagram of a composite question frame without a common question stem provided by an embodiment of the present invention;

[0042] Figure 5A schematic diagram of a detection process for a composite question frame without a common question stem provided by an embodiment of the present invention;

[0043] Figure 6 A schematic diagram of the structure of a question stem frame with illustrations provided in an embodiment of the present invention;

[0044] Figure 7 A schematic diagram of a detection process of a question stem frame with illustrations provided in an embodiment of the present invention;

[0045] Figure 8 A schematic diagram of the detection process of the answer sheet area frame provided in an embodiment of the present invention;

[0046] Fig. 9 A schematic diagram of the structure of a device for segmenting a question stem and a mixed answer provided in an embodiment of the present invention;

[0047] Fig.10 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0050] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as in the embodiments of this application.

[0051] refer to Figure 1 As shown, Figure 1 is a schematic diagram of the application environment provided by an embodiment of the present invention; the technical solution of the present application can be implemented on a server, such as Figure 1In the embodiment of the present application, a plurality of detection models are deployed in the server 102. After the server 102 obtains the image to be detected input by the terminal device 101, the question stem and answer layout of the image to be detected are detected by one of the detection models to obtain a first detection result. Then, the option boxes of each type detected are subjected to a secondary detection by other detection models to obtain a second detection result. Then, the second detection result is compared with the first detection result to determine whether there are any missed option boxes in the first detection result. If so, the question stem and answer areas of the image to be detected are segmented according to the second detection result, thereby realizing the mixed segmentation function of the question stem and answer.

[0052] It should be noted that the terminal device 101 here includes a smart phone, a tablet computer, an e-reader, a desktop computer or a laptop computer, etc., and there is no limitation on this; the terminal device 101 in this embodiment can realize a scanning function, and the client installed in the terminal device 101 can enter the question based on the scanning function.

[0053] The server 102 here refers to a server that provides question stem and answer segmentation services for questions entered by the client installed in the terminal device 101, and can be implemented by an independent server or a server cluster composed of multiple servers.

[0054] In one embodiment, Figure 2 As shown, Figure 2 The present invention provides a flow chart of a method for segmenting a question stem and a mixed answer provided in an embodiment of the present invention. The present invention provides a method for segmenting a question stem and a mixed answer, which specifically includes the following steps:

[0055] S110: Acquire an image to be detected, and detect the question stem and answer layout of the image to be detected to obtain a first detection result.

[0056] In this step, before the server divides the question stem and answer of the question entered into the question bank, it needs to obtain the image to be detected corresponding to the question, and use the trained target detection model to detect the question stem and answer layout of the image to be detected, so as to obtain a first detection result, which includes but is not limited to multiple option boxes and the types of option boxes corresponding to the image to be detected.

[0057] It should be noted that the image to be detected here can be a scanned image or an image containing question information obtained in other scenarios; the option box here is obtained by detecting the question stem and answer layout of the image to be detected based on the target detection model; the target detection model here is trained based on manually annotated images.

[0058] For example, according to the layout type of the image to be detected, it is divided into question stem, answer, analysis, answer sheet options, illustrations, tables, compound questions with no common stem, and answer sheet area; among them, when the target detection model detects the answer sheet options, each option box only frames the options of a small question, and the answer sheet area combines the option boxes of all small questions into a large option box; compound questions with no common stem are large questions composed of multiple sub-questions, and there is no common stem description text between the multiple sub-questions. Therefore, when detecting the answers and analyses of compound questions with no common stem, the answers and analyses of multiple sub-questions are also framed into a large option box.

[0059] Therefore, after the image to be detected is input into the target detection model, the target detection model can detect various option boxes and corresponding types according to the content in the image to be detected. The types include but are not limited to compound question boxes without common question stems, question stem boxes with illustrations, answer sheet area boxes, etc.

[0060] It can be understood that the target detection model here can be the Faster-RCNN model, and other target detection models can also be used in actual use. The purpose of selecting this model in this solution is to solve the problem of drastic changes in the sizes of various types of regions corresponding to the question stem and the answer. This model is a relatively excellent network model in the field of target detection in recent years, and it has the properties of detecting both large and small targets.

[0061] S120: Determine multiple option boxes and types of option boxes corresponding to the image to be detected according to the first detection result, and perform secondary detection on each type of option box to obtain a second detection result.

[0062] In this step, the question stem and answer layout of the image to be detected are detected through step S110. After obtaining the first detection result, the server can use the relevant detection model to perform secondary detection on the detected different types of option boxes to obtain the second detection result.

[0063] It can be understood that the first detection result obtained through step S110 detection includes various types of option boxes, and the detection accuracy of the target detection model for different types of option boxes is relatively lower than that of using a more suitable target detection model for the same type of option boxes; and directly using the first detection result to segment the question stem and answer area of ​​the image to be detected may result in missed option boxes.

[0064] Therefore, in the present application, after using the target detection model to detect the question stem and answer layout in the image to be detected and obtaining the first detection result, a secondary detection can be performed based on the various option boxes and corresponding types in the first detection result, and then different detection models can be used for different types of option boxes to obtain a second detection result.

[0065] It can be understood that the types corresponding to the option boxes here include but are not limited to compound question boxes without common stems, question boxes with illustrations, answer sheet area boxes, etc. Therefore, appropriate target detection models can be selected for each type to perform detection in order to improve the accuracy of the detection results.

[0066] For example, when conducting secondary testing on compound question frames without a common stem, the SSD model can be used. This model has the advantage of being relatively efficient while ensuring a high accuracy rate. It is a relatively stable model in the field of target detection in recent years, and it is also a milestone model in the field of target detection. Of course, in actual use, other relatively excellent models can also be used, such as the YOLO-V3 model.

[0067] S130: Compare the second detection result with the first detection result to determine whether there are any missed option boxes in the first detection result. If yes, segment the question stem and answer area of ​​the image to be detected according to the second detection result.

[0068] In this step, after the second detection results corresponding to the option boxes of each type are obtained in step S120, the second detection results may be compared with the first detection results to determine whether there are any option boxes that are missed in the first detection results.

[0069] For example, when the type of the option box in the second detection result corresponds to the answer sheet area box, the judgment rule for determining whether there is an option box that is missed in the first detection result is:

[0070] (1) Traversing the answer sheet option boxes detected in the second detection result;

[0071] (2) Calculate the intersection and union ratio of the answer sheet option box detected in the current second detection result and all the answer sheet option boxes detected in the first detection result, and determine whether there is a situation where the intersection and union ratio is greater than a preset threshold. If so, it is determined to be a non-missed option box, and continue with step (3); otherwise, it is determined to be a missed option box, and continue with step (3);

[0072] (3) Determine whether there are any untraversed option boxes in the second detection result. If yes, return to step (2).

[0073] From the above judgment rules, it can be seen that the steps for judging the option boxes as missed detection can be expressed as: first select an answer sheet area box from the second test result, and compare it with all the option boxes in the first test result to see whether the answer sheet area box has been detected. If not, keep it; if it has been detected, discard it and keep the corresponding option box in the first test result.

[0074] It should be noted that the intersection-over-union ratio, i.e., the term IOU, is IOU = the area of ​​the intersection of two rectangles / the area of ​​the union of two rectangles. When the IOU is greater than a preset threshold, the two option boxes are merged or one of them is discarded.

[0075] Furthermore, if a missed option box is detected through the second detection result, the option box in the second detection result is retained, and the question stem and answer area in the image to be detected are segmented through the option box.

[0076] In addition, some questions or explanations that contain sub-questions may not be marked with unified manual marking standards, resulting in the detection of a large box for the entire question and multiple small boxes for the sub-questions. Therefore, the duplicate option boxes need to be merged.

[0077] In the above embodiment, first, according to the obtained layout of the question stem and answer of the image to be detected, the image to be detected is divided into multiple option boxes, and the type of the option box is determined to realize automatic segmentation of the question stem and the answer, thereby reducing the workload of manual segmentation and greatly improving the efficiency of recording questions; and then a secondary test is performed on different types of option boxes respectively, which can solve various problems of the layout types of questions and answers, and the corresponding second test result has a higher accuracy rate in the segmentation of the question stem and the answer; therefore, after obtaining the second test result, the second test result is used to check whether there are any option boxes that have been missed in the first test result. If so, the question stem and answer area of ​​the image to be detected are segmented according to the second test result, so that the accuracy of the overall segmentation frame is further improved, and the segmentation effect is better.

[0078] In one embodiment, the method for segmenting the question stem and the mixed answer may further include:

[0079] S140: Determine whether there are any missed option boxes in the first detection result. If not, segment the question stem and answer area of ​​the image to be detected according to the first detection result.

[0080] In this embodiment, when the second detection result is used to detect whether there are any missed option boxes in the first detection result, if there are no missed option boxes, it means that the first detection result contains all the option boxes. At this time, the option boxes in the second detection result can be discarded, and the corresponding option boxes in the first detection result are retained and used as the basis for segmenting the question stem and answer area of ​​the image to be detected.

[0081] Alternatively, the option box in the second detection result is merged with the option box in the first detection result, and the merged option box is used to segment the question stem and answer area of ​​the image to be detected.

[0082] In one embodiment, the types corresponding to the various option boxes may include a composite question box without a common question stem, a question stem box with an illustration, and an answer sheet area box; and the step of performing a secondary test on each type of option box in step S120 to obtain a second test result may include:

[0083] S121: traverse the first test results, and determine the composite question frames without common question stems in the first test results;

[0084] S122: Detect sub-question numbers in the area where the composite question frame without a common question stem is located, identify the detected sub-question numbers, and determine the corresponding question numbers and answer areas.

[0085] In this embodiment, due to the diversity of question stem and answer distribution, and the same layout, the segmentation area is different in different scenarios, which makes the segmentation process of question stem and answer area more difficult. Figure 3 , 4 As shown, Figure 3 A first structural diagram of a composite question frame without a common question stem provided by an embodiment of the present invention, Figure 4 A second structural schematic diagram of a composite question frame without a common stem provided in an embodiment of the present invention.

[0086] Figure 3 The content of the entire option box is the content of the question, among which the parentheses a / b / c are the sub-question numbers. Figure 4 In the scenario where this option box corresponds to the answer scenario, the content in the parentheses in the answer scenario is the answer option, and the right side of the option is the analysis. In the teaching and research recording scenario, the answer and analysis need to be entered separately.

[0087] Therefore, for a composite question frame without a common question stem, this embodiment can be used for detection. Schematically, Figure 5 As shown, Figure 5 A schematic diagram of a detection process for a composite question frame without a common question stem provided by an embodiment of the present invention; Figure 5 In the test, the detection process for compound question frames without common stems includes the following steps:

[0088] a101: traverse the option box of the first detection result output;

[0089] a102: Determine whether there are any untraversed option boxes. If yes, proceed to step a103, otherwise end;

[0090] a103: Determine whether the current option box is a composite question box without a common question stem. If yes, proceed to step a104; otherwise, return to step a101;

[0091] a104: Perform sub-question number detection in the area corresponding to the current composite question frame without a common question stem;

[0092] a105: Further judge the question number detected in step a104 to determine whether it is an answer option.

[0093] In the above steps, the SSD model or the YOLO-V3 model can be used for the sub-question number detection in step a104, and the relatively lightweight Lenet5 model can be used to identify the detected question number area.

[0094] It is understandable that since the question number is generally 26 letters or numbers within 100, the question number area can be divided into 126 categories. In fact, there are very few sub-questions with numbers exceeding 20, so setting it to 46 categories is also acceptable. In order to ensure accurate recognition in actual use, this solution sets more categories.

[0095] Of course, when the number of classes is small or the samples are sufficient, it is possible to directly use the SSD model for detection and recognition instead of using a separate recognition solution.

[0096] Furthermore, the judgment rule for determining whether the question number is an answer option in step a105 is:

[0097] 1) There is only one question number. If the numerical question number is greater than 1 or the letter question number is greater than a, the question number is judged as the answer option. If the numerical question number is equal to 1 or the letter question number is equal to a, it is judged as the question number;

[0098] 2) When there are multiple question numbers, if the question numbers are from small to large and continuous, they are judged as question numbers, otherwise they are judged as answer options.

[0099] In one embodiment, the step of performing secondary detection on each type of option box in step S120 to obtain a second detection result may further include:

[0100] S123: traverse the first detection result, and determine the question stem frame with illustrations in the first detection result;

[0101] S124: Cover the illustration area of ​​the question stem frame to be illustrated, and perform question stem recognition on the covered question stem frame with illustration to determine the corresponding question stem area.

[0102] In this embodiment, due to the diversity of the content of the question stem, it is inevitable that there will be expressions of non-text content in the question stem, such as illustrations. The illustrations divide the question into multiple parts, such as Figure 6 As shown, Figure 6 A schematic diagram of the structure of a question stem frame with illustrations provided in an embodiment of the present invention; Figure 6 In the question, the contents of the three option boxes are all the content of the question stem, among which the middle option box is an illustration.

[0103] against Figure 6 The scene shown in the figure can be detected by using the detection process of this embodiment. Figure 7 As shown, Figure 7 A schematic diagram of a detection process of a question stem frame with illustrations provided in an embodiment of the present invention; Figure 7 In the test, the detection process for the illustrated question frame includes the following steps:

[0104] b101: traverse the option box of the first detection result output;

[0105] b102: Determine whether there are any untraversed option boxes. If so, proceed to step b103, otherwise end;

[0106] b103: Determine whether the current option box is a question box with illustrations. If yes, proceed to step b104; otherwise, return to step b101;

[0107] b104:crop illustration;

[0108] b105: Mask the illustration in the original image.

[0109] Figure 7 In the process, firstly, all the option boxes in the first detection result are traversed, and the illustrated question stem box in the first detection result is determined, and then the illustration area of ​​the question stem box to be illustrated is covered. In this way, the question stem of the covered question stem box with illustration can be identified and the corresponding question stem area can be determined.

[0110] It should be noted that for the question stem frame with illustrations, the question stem and the illustration that are divided into multiple areas are framed in a large frame as the question stem frame, and the illustration part is framed in a separate frame. The two frames are of different types, but both belong to the question stem content. If the divided question stem is divided into multiple parts for testing, there will be problems with question stem recovery and the risk of partial loss.

[0111] Therefore, when sending the question stem area to OCR recognition, the illustration area in the question stem area is masked, that is, it is set to 0 or 255. In this way, the question stem will not be disturbed by the illustrations when it is recognized. At the same time, the integrity of the question stem is guaranteed, which perfectly solves the problem of interference from non-stem factors in the question stem and improves the OCR recognition accuracy of the question stem content.

[0112] Of course, the same method can be used to remove other interfering factors in the question stem to ensure the completeness and correctness of the question stem.

[0113] In one embodiment, the step of performing secondary detection on each type of option box in step S120 to obtain a second detection result may further include:

[0114] S125: traversing the first test result, and determining the answer sheet area frame in the first test result;

[0115] S126: Perform option detection on the answer sheet area frame to determine corresponding option block information.

[0116] In this embodiment, since the options in the answer sheet scenario are small, numerous, and dense, and the layout is diverse, some answer sheet areas have letter options filled in at the positions corresponding to the question numbers, and are distributed in a rectangular pattern. Some question numbers have multiple options in the answer options corresponding to them, and some have different display methods due to the different number of question numbers.

[0117] Therefore, for scenarios with multiple layouts in the answer sheet area frame, the detection process of this embodiment can be used for detection. Figure 8 As shown, Figure 8 A schematic diagram of the detection process of the answer sheet area frame provided in an embodiment of the present invention; Figure 8 In the process of detecting the answer sheet area frame, the following steps are included:

[0118] c101: traverse the option box of the first detection result output;

[0119] c102: Determine whether there are any untraversed option boxes. If so, proceed to step c103, otherwise end;

[0120] c103: Determine whether the current option box is the answer sheet area box, if yes, proceed to step c104, otherwise return to step c101;

[0121] c104: Perform option detection in the current answer sheet area frame to obtain option block information.

[0122] In the above step c104, the detection model mainly used for option detection in the answer sheet area box is the SSD model. This model has the advantage of high efficiency while ensuring high accuracy. It is a model that has performed relatively stably in the field of target detection in recent years, and is also a milestone model in the field of target detection.

[0123] Moreover, the option detection in this embodiment is only divided into target and non-target, and no recognition is required. The recognition task is completed by the subsequent OCR module. In actual use, other relatively excellent detection models such as YOLO-V3 can also be used.

[0124] In one embodiment, the step of comparing the second detection result with the first detection result in step S130 to determine whether there is a missed option box in the first detection result may include:

[0125] The option boxes corresponding to the question number and the answer area are compared with the option boxes in the first detection result to determine whether there are any option boxes that are missed in the first detection result.

[0126] In this embodiment, since the detection model used in the second detection result is only for a certain type of option box, such as the detection of compound question boxes without a common question stem, the accuracy of the detection result is relatively high compared to the first detection result. Therefore, by comparing the option boxes corresponding to the question number and answer area in the second detection result with the option boxes in the first detection result, it is possible to determine whether there are any missed option boxes in the first detection result, thereby improving the accuracy of the model.

[0127] Furthermore, the comparison process can be implemented through an intersection-and-union ratio, where the area of ​​the intersection of the option boxes corresponding to the question number and answer area in the second test result and the option boxes in the first test result is divided by the area of ​​the union. When the intersection-and-union ratio of the two is greater than a preset threshold, it is determined that there are no missed option boxes. After determining that there are no missed option boxes, the two option boxes can be merged or one of them can be discarded.

[0128] In one embodiment, the step of comparing the second detection result with the first detection result in step S130 to determine whether there is a missed option box in the first detection result may include:

[0129] The option boxes corresponding to the question stem area are compared with the option boxes in the first detection result to determine whether there are any option boxes missed in the first detection result.

[0130] In this embodiment, since the detection model used in the second detection result is only for a certain type of option box, such as the detection of a question stem box with an illustration, the accuracy of the detection result is relatively high compared to the first detection result. Therefore, by comparing the option boxes corresponding to the question stem area in the second detection result with the option boxes in the first detection result, it is possible to determine whether there are any missed option boxes in the first detection result, thereby improving the accuracy of the model.

[0131] Furthermore, the comparison process can be implemented through an intersection-and-union ratio, where the area of ​​the intersection of the option box corresponding to the question area in the second detection result and the option box in the first detection result is divided by the area of ​​the union. When the intersection-and-union ratio of the two is greater than a preset threshold, it is determined that there are no missed option boxes. After determining that there are no missed option boxes, the two option boxes can be merged or one of them can be discarded.

[0132] In one embodiment, the step of comparing the second detection result with the first detection result to determine whether there is a missed option box in the first detection result includes:

[0133] The option box corresponding to the option block information is compared with the option box in the first detection result to determine whether there is any missed option box in the first detection result.

[0134] In this embodiment, since the detection model used in the second detection result is only for a certain type of option box, such as detecting the answer sheet option box in the answer sheet area box, the accuracy rate of the answer sheet option box detected is generally much higher than the accuracy rate of the answer sheet option box in the first detection result, so this module is set to perform answer sheet option detection on the answer sheet area, and the detection results of the two are merged to achieve the optimal effect.

[0135] Furthermore, the comparison process can be implemented through an intersection-and-union ratio, where the area of ​​the intersection of the option box corresponding to the option block information in the second detection result and the option box in the first detection result is divided by the area of ​​the union. When the intersection-and-union ratio of the two is greater than a preset threshold, it is determined that there are no missed option boxes. After determining that there are no missed option boxes, the two option boxes can be merged or one of them can be discarded.

[0136] In one embodiment, Fig. 9 As shown, Fig. 9 A schematic diagram of the structure of a device for segmenting a question stem and a mixed answer provided in an embodiment of the present invention; Fig. 9 The present invention also provides a segmentation device for a mixed question stem and answer, comprising a first detection module 210, a second detection module 220, and a segmentation module 230, which specifically includes the following:

[0137] The first detection module 210 is used to obtain an image to be detected, and detect the question stem and answer layout of the image to be detected to obtain a first detection result;

[0138] A second detection module 220 is used to determine a plurality of option boxes and types of option boxes corresponding to the image to be detected according to the first detection result, and perform secondary detection on each type of option box to obtain a second detection result;

[0139] The segmentation module 230 is used to compare the second detection result with the first detection result to determine whether there are any missed option boxes in the first detection result. If so, the question stem and answer area of ​​the image to be detected are segmented according to the second detection result.

[0140] In the above embodiment, first, according to the obtained layout of the question stem and answer of the image to be detected, the image to be detected is divided into multiple option boxes, and the type of the option box is determined to realize automatic segmentation of the question stem and the answer, thereby reducing the workload of manual segmentation and greatly improving the efficiency of recording questions; and then a secondary test is performed on different types of option boxes respectively, which can solve various problems of the layout types of questions and answers, and the corresponding second test result has a higher accuracy rate in the segmentation of the question stem and the answer; therefore, after obtaining the second test result, the second test result is used to check whether there are any option boxes that have been missed in the first test result. If so, the question stem and answer area of ​​the image to be detected are segmented according to the second test result, so that the accuracy of the overall segmentation frame is further improved, and the segmentation effect is better.

[0141] For the specific definition of the segmentation device for the mixed question stem and answer, please refer to the definition of the segmentation method for the mixed question stem and answer in the above text, which will not be repeated here. Each module in the above-mentioned segmentation device for the mixed question stem and answer can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the terminal device in the form of hardware, or can be stored in the memory in the terminal device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0142] In one embodiment, the present invention also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method for dividing the question stem and the mixed answer as described in any of the above embodiments.

[0143] In one embodiment, the present invention also provides a computer device having computer-readable instructions stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method for dividing the question stem and the mixed answer as described in any one of the above embodiments.

[0144] Indicatively, if Fig.10 As shown, Fig.10 The internal structure diagram of a computer device provided in an embodiment of the present invention is shown in FIG. 300 . The computer device 300 may be provided as a server. Fig.10 The computer device 300 includes a processing component 302, which further includes one or more processors, and a memory resource represented by a memory 301, for storing instructions executable by the processing component 302, such as an application. The application stored in the memory 301 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 302 is configured to execute instructions to perform the segmentation method of the question stem and the mixed answer of any of the above embodiments.

[0145] The computer device 300 may further include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate based on an operating system stored in the memory 301, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0146] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0147] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for segmenting a question stem and a mixed answer, characterized in that: The method comprises: Acquire an image to be detected, and use the target detection model to detect the question stem and answer layout of the image to be detected to obtain a first detection result; Determine multiple option boxes and types of option boxes corresponding to the image to be detected according to the first detection result, and perform secondary detection on each type of option box to obtain a second detection result; wherein different types of option boxes are respectively secondary detected using different detection models, and the types corresponding to each option box include a composite question box without a common question stem, a question stem box with an illustration, and an answer sheet area box; The second detection result is compared with the first detection result to determine whether there are any missed option boxes in the first detection result. If so, the question stem and answer area of ​​the image to be detected are segmented according to the second detection result; if not, the question stem and answer area of ​​the image to be detected are segmented according to the first detection result.

2. The method for segmenting a question stem and a mixed answer according to claim 1, characterized in that: The step of performing secondary detection on each type of option box to obtain a second detection result includes: Traversing the first test results, and determining the composite question frames without common question stems in the first test results; The sub-question number is detected in the area where the composite question frame without a common question stem is located, and the detected sub-question number is identified to determine the corresponding question number and answer area.

3. The method for segmenting a question stem and a mixed answer according to claim 1, characterized in that: The step of performing secondary detection on each type of option box to obtain a second detection result also includes: Traversing the first test result, and determining a question stem frame with an illustration in the first test result; The illustration area of ​​the illustrated question stem frame is covered, and question stem recognition is performed on the covered illustrated question stem frame to determine the corresponding question stem area.

4. The method for segmenting a question stem and a mixed answer according to claim 1, characterized in that: The step of performing secondary detection on each type of option box to obtain a second detection result also includes: Traversing the first test result, and determining the answer sheet area frame in the first test result; The answer sheet area frame is subjected to option detection to determine the corresponding option block information.

5. The method for segmenting a question stem and a mixed answer according to claim 2, characterized in that: The step of comparing the second detection result with the first detection result to determine whether there is an option box that is missed in the first detection result includes: The option boxes corresponding to the question number and the answer area are compared with the option boxes in the first detection result to determine whether there are any option boxes that are missed in the first detection result.

6. The method for segmenting a question stem and a mixed answer according to claim 3, characterized in that: The step of comparing the second detection result with the first detection result to determine whether there is an option box that is missed in the first detection result includes: The option boxes corresponding to the question stem area are compared with the option boxes in the first detection result to determine whether there are any option boxes missed in the first detection result.

7. The method for segmenting a question stem and a mixed answer according to claim 4, characterized in that: The step of comparing the second detection result with the first detection result to determine whether there is an option box that is missed in the first detection result includes: The option box corresponding to the option block information is compared with the option box in the first detection result to determine whether there is any missed option box in the first detection result.

8. A device for dividing a question stem and an answer, characterized in that: include: A first detection module is used to obtain an image to be detected, and use a target detection model to detect the question stem and answer layout of the image to be detected to obtain a first detection result; A second detection module is used to determine multiple option boxes and types of option boxes corresponding to the image to be detected according to the first detection result, and perform secondary detection on each type of option box to obtain a second detection result; wherein different types of option boxes are respectively secondary detected using different detection models, and the types corresponding to each option box include a composite question box without a common question stem, a question stem box with an illustration, and an answer sheet area box; A segmentation module is used to compare the second detection result with the first detection result to determine whether there are any missed option boxes in the first detection result. If so, the question stem and answer area of ​​the image to be detected are segmented according to the second detection result; if not, the question stem and answer area of ​​the image to be detected are segmented according to the first detection result.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to execute the steps of the method for dividing the question stem and the mixed answer as described in any one of claims 1 to 7.

10. A computer device, characterized in that: The computer device stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method for dividing the question stem and the mixed answer as described in any one of claims 1 to 7.

Images