Recognition method and apparatus

Abstract

The present specification provides an identification method and device, wherein the identification method comprises: acquiring a target image input by a user and containing a to-be-identified question type; performing identification processing on the target image, and generating a target character box set corresponding to the to-be-identified question type according to an identification processing result; determining attribute information of each character box based on category information corresponding to each character box in the target character box set; and splicing character boxes contained in the target character box set by traversing the attribute information of each character box, to obtain an identification result corresponding to the to-be-identified question type.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to identification methods and devices. Background Technology

[0002] With the development of internet technology, the online education industry has attracted increasing attention from users. Within this industry, educational tools not only provide tutoring and assistance to students but also offer supplementary educational support to parents and teachers. Current technologies, when recognizing questions in user-captured images, mostly employ object detection technology to analyze the image layout, using bounding boxes to distinguish the spatial locations of different questions. Then, OCR technology is used to analyze the content within each bounding box to obtain the text information of the question. Finally, the text information is directly or indirectly matched with questions in a pre-set question bank to obtain the question recognition result. However, the above methods not only rely on the richness of the pre-set question bank but also cannot guarantee the accuracy of question recognition. Furthermore, when a question contains multiple sub-questions, it is even more difficult to guarantee accurate recognition of each sub-question, severely impacting the user experience. Therefore, an effective solution is urgently needed to address these issues. Summary of the Invention

[0003] In view of this, embodiments of this specification provide an identification method. This specification also relates to an identification device, a computing device, and a computer-readable storage medium to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, an identification method is provided, comprising:

[0005] Obtain the target image containing the question type to be identified, as input by the user;

[0006] The target image is subjected to recognition processing, and a set of target character boxes corresponding to the question type to be recognized is generated based on the recognition processing results;

[0007] The attribute information of each character box is determined based on the category information corresponding to each character box in the target character box set;

[0008] By traversing the attribute information of each character box, the character boxes contained in the target character box set are concatenated to obtain the recognition result corresponding to the question type to be recognized.

[0009] Optionally, the step of performing recognition processing on the target image and generating a set of target character boxes corresponding to the question type to be recognized based on the recognition processing result includes:

[0010] The target image is input into the question type recognition module for processing to obtain the initial question type bounding box corresponding to the question type contained in the target image, and

[0011] inputting the target image into a character recognition module for processing to obtain initial character boxes corresponding to characters contained in the target image;

[0012] generating the target character box set corresponding to the to-be-recognized question type based on the initial question type box and the initial character box.

[0013] Optionally, the generating the target character box set corresponding to the to-be-recognized question type based on the initial question type box and the initial character box comprises:

[0014] determining character box coordinates corresponding to the initial character box and question type box coordinates corresponding to the initial question type box based on the target image;

[0015] performing division processing on the initial character box based on the question type box coordinates and the character box coordinates, and obtaining at least one initial character box set according to a division processing result;

[0016] screening an initial character box set matching the to-be-recognized question type from the at least one initial character box set as the target character box set.

[0017] Optionally, the screening an initial character box set matching the to-be-recognized question type from the at least one initial character box set as the target character box set comprises:

[0018] determining a question type keyword corresponding to the to-be-recognized question type and a keyword corresponding to each initial character box set contained in the at least one initial character box set;

[0019] calculating a matching degree of the question type keyword and the keyword corresponding to each initial character box set;

[0020] selecting an initial character box set corresponding to a keyword with the highest matching degree as the target character box set corresponding to the to-be-recognized question type.

[0021] Optionally, the determining attribute information of each character box based on category information corresponding to each character box in the target character box set comprises:

[0022] determining category information corresponding to each character box based on characters contained in each character box in the target character box set;

[0023] adjusting a state of each character box according to the category information, and determining attribute information of each character box according to an adjustment result.

[0024] Optionally, the category information corresponding to each character box comprises at least one of:

[0025] Word unit category, condition category, answer category, invalid category.

[0026] Optionally, the step of concatenating the character frames contained in the target character frame set by traversing the attribute information of each character frame to obtain the recognition result corresponding to the question type to be recognized includes:

[0027] Read the question recognition conditions corresponding to the question type to be identified;

[0028] Based on the question recognition conditions, the attribute information of each character box is traversed, and the target character box set is divided according to the traversal results to obtain multiple sub-character box sets;

[0029] Multiple target questions are obtained by concatenating the character boxes contained in each sub-character box set, and the multiple target questions are used as the recognition results corresponding to the question type to be recognized.

[0030] Optionally, after the step of concatenating the character boxes contained in the target character box set by traversing the attribute information of each character box to obtain the recognition result corresponding to the question type to be recognized, the method further includes:

[0031] The user's identification intent is determined based on the identification results;

[0032] When the identification intent is a detection intent, the target answer information corresponding to the question type to be identified is obtained by reading a preset knowledge base based on the question information in the identification result.

[0033] The target answer information is compared with the user answer information in the recognition result, and a detection result image is generated based on the comparison result and displayed to the user.

[0034] Optionally, generating a detection result image based on the comparison result and displaying it to the user includes:

[0035] Based on the comparison results, the colors of each question box in the recognition results are adjusted to obtain the correct question box and the incorrect question box;

[0036] The incorrect question boxes and the correct question boxes are updated based on the target answer information, and the detection result image is generated and displayed to the user based on the update result.

[0037] Optionally, after the step of concatenating the character boxes contained in the target character box set by traversing the attribute information of each character box to obtain the recognition result corresponding to the question type to be recognized, the method further includes:

[0038] The user's identification intent is determined based on the identification results;

[0039] In a case where the identified intention is a search intention, target answer information corresponding to the to-be-identified question type is obtained based on the question information in the identification result and from a preset knowledge base.

[0040] A search result image is generated based on the question information in the identification result and the target answer information, and is displayed to the user.

[0041] According to a second aspect of the embodiments of the present specification, an identification device is provided, comprising:

[0042] An acquisition module is configured to acquire a target image input by a user and containing a to-be-identified question type;

[0043] An identification module is configured to perform identification processing on the target image, and generate a target character box set corresponding to the to-be-identified question type according to an identification processing result;

[0044] A determination module is configured to determine attribute information of each character box based on category information corresponding to each character box in the target character box set;

[0045] A splicing module is configured to splice character boxes contained in the target character box set by traversing the attribute information of each character box, and obtain an identification result corresponding to the to-be-identified question type.

[0046] According to a third aspect of the embodiments of the present specification, a computing device is provided, comprising:

[0047] A memory and a processor;

[0048] The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions to implement the steps of the identification method.

[0049] According to a fourth aspect of the embodiments of the present specification, a computer readable storage medium is provided, which stores computer executable instructions, and the instructions are executed by a processor to implement the steps of the identification method.

[0050] The recognition method provided in the specification can perform recognition processing on the target image after obtaining the target image input by the user and containing the to-be-recognized question type, to generate a target character box set corresponding to the to-be-recognized question type according to the recognition result, then determine the attribute information of each character box based on the category information corresponding to each character box in the target character box set, and finally splice the character boxes contained in the target character box set by traversing the attribute information of each character box, so as to obtain the recognition result corresponding to the to-be-recognized question type according to the splicing result, realize splicing of the text content corresponding to the to-be-recognized question type based on the attribute information of the character box, and not only can ensure the accuracy of recognizing the to-be-recognized question type, but also can improve the recognition efficiency of the question type, so as to quickly feed back the recognition result to the user and improve the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 is a flowchart of a recognition method provided by an embodiment of the specification;

[0052] Figure 2 is a schematic diagram of a question type box in a recognition method provided by an embodiment of the specification;

[0053] Figure 3 is a schematic diagram of a character box in a recognition method provided by an embodiment of the specification;

[0054] Figure 4 is a schematic diagram of a recognition result in a recognition method provided by an embodiment of the specification;

[0055] Figure 5 is a schematic diagram of a detection result image in a recognition method provided by an embodiment of the specification;

[0056] Figure 6 is a schematic diagram of a search result image in a recognition method provided by an embodiment of the specification;

[0057] Figure 7 is a processing flowchart of a recognition method applied to an answer search scenario provided by an embodiment of the specification;

[0058] Figure 8 is a structural schematic diagram of a recognition device provided by an embodiment of the specification;

[0059] Figure 9 is a structural block diagram of a computing device provided by an embodiment of the specification. DETAILED DESCRIPTION

[0060] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. However, the present description can be practiced without the specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the present description.

[0061] The terminology used in this description is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present description. As used in this description and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0062] It will be understood that, although the terms first, second, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used solely to distinguish one from another only. For example, without departing from the scope of one or more embodiments, first can be termed second, and similarly, second can be termed first. The term "if' as used herein can be interpreted as meaning "when" or "in response to determining" depending on the context.

[0063] First, the noun terms related to one or more embodiments of the present description are explained.

[0064] Computer vision: Computer vision is a science that uses computers to simulate the human visual system, enabling computers to have the ability to extract, process, understand and analyze images and image sequences similar to humans. Autonomous driving, robots, intelligent medical care and other fields all need to extract and process information from visual signals through computer vision technology.

[0065] Object detection: A neural network model that takes an image as input and outputs multiple rectangular boxes. Each rectangular box is represented by the coordinates of the upper left corner and the lower right corner, which encloses the visually salient object (such as text area, picture area, etc.) in the image. At the same time, each rectangular box also has a category attribute (such as Chinese characters, English, etc.). Common models include but are not limited to RCNN series, YOLO series.

[0066] OCR: Optical Character Recognition (OCR), the process of analyzing image-form text information to obtain the position and content of the text, mainly composed of two parts: text detection and text recognition.

[0067] In the specification, an identification method is provided, and the specification also relates to an identification device, a computing device, and a computer-readable storage medium, which are described in detail in the following embodiments.

[0068] In practical applications, the identification of the question mostly depends on the quality of the question bank. If the content of the question bank is not sufficient, the algorithm can only match results similar to but not completely consistent with the actual question content in the image, and even no matching result can be obtained for rare questions. If the content of the question bank is too sufficient, the matching process will consume a large amount of computing resources and time, and cannot achieve the real-time performance expected by the user. At the same time, for the question type containing sub-questions, the current identification method cannot distinguish the level of the sub-questions, which greatly affects the accuracy of identification. Therefore, an effective solution is urgently needed to solve the above problems.

[0069] The identification method provided in the specification can perform identification processing on the target image after obtaining the target image input by the user and containing the to-be-identified question type, generate a target character box set corresponding to the to-be-identified question type according to the identification result, determine the attribute information of each character box based on the category information corresponding to each character box in the target character box set, and finally splice the character boxes contained in the target character box set by traversing the attribute information of each character box. According to the splicing result, the identification result corresponding to the to-be-identified question type can be obtained, the text content corresponding to the to-be-identified question type is spliced based on the attribute information of the character box, the accuracy of identifying the to-be-identified question type is ensured, the identification efficiency of the question type is improved, and the user can quickly receive the identification result, thereby improving the user experience.

[0070] Figure 1 A flowchart of an identification method according to an embodiment of the specification is shown, which specifically includes the following steps:

[0071] In step S102, a target image containing a to-be-identified question type input by a user is obtained.

[0072] Specifically, the user specifically refers to a user who has an identification intention for the to-be-identified question type, and the identification intention includes but is not limited to searching for the answer of the question corresponding to the to-be-identified question type, detecting the correctness of the answer of the question corresponding to the to-be-identified question type, and querying other questions similar to the question corresponding to the to-be-identified question type. Correspondingly, the to-be-identified question type specifically refers to a question type that needs to be identified in text content. It needs to be noted that the to-be-identified question type can be any question type of a question answered by a student, such as the rule finding question type in the mathematics subject, the idiom filling question type in the Chinese subject, and the word writing question type according to requirements in the English subject. The operation of identifying any question type can be referred to the corresponding description content of this embodiment, and is not limited here.

[0073] Further, the target image specifically refers to an image uploaded by the user after capturing the to-be-recognized question type, which at least contains the to-be-recognized question type and can also contain other questions belonging to the same page as the to-be-recognized question type.

[0074] In actual application, in order to facilitate subsequent identification processing operation of the to-be-recognized question type in the target image, the collection strategy can be shown to the user when the user collects the target image through the terminal device, so as to assist the user to successfully collect the target image containing the to-be-recognized question type, improve the subsequent identification processing efficiency, and quickly feed back the identification result of the user; wherein the collection strategy specifically refers to a strategy of guiding the user to focus on capturing and uploading the to-be-recognized question type.

[0075] Step S104, performing identification processing on the target image, and generating a target character box set corresponding to the to-be-recognized question type according to the identification processing result.

[0076] Specifically, on the basis of obtaining the target image uploaded by the user and containing the to-be-recognized question type, further, in order to accurately recognize the to-be-recognized question type from the target image, the target image can be preliminarily identified to generate a target character box set corresponding to the to-be-recognized question type according to the identification processing result, so that the text content output of the to-be-recognized question type can be spliced based on the target character box set in subsequent operation, and the purpose of recognizing the to-be-recognized question type is achieved.

[0077] Wherein, the identification processing on the target image specifically refers to identifying the question types contained in the target image and the corresponding quantities, and the characters and the corresponding quantities, so as to facilitate subsequent division of the characters contained in each question type, thereby determining the target character box set corresponding to the to-be-recognized question type; correspondingly, the target character box set specifically refers to a set of character boxes corresponding to the characters contained in the to-be-recognized question type, and the text content corresponding to the to-be-recognized question type can be obtained by splicing the character boxes contained in the target character box set, so as to achieve the purpose of recognizing the to-be-recognized question type.

[0078] Further, since the target image uploaded by the user can also contain other question types, if each question type is identified, not only a large amount of computing resources will be consumed, but also the user cannot be accurately reached, therefore, in order to accurately complete the identification processing operation of the to-be-recognized question type, the determination of the target character box set can be combined with question type identification and character identification, and in this embodiment, the specific implementation manner is as follows:

[0079] In step S1042, the target image is input to the question type recognition module for processing to obtain initial question type frames corresponding to question types contained in the target image, and the target image is input to the character recognition module for processing to obtain initial character frames corresponding to characters contained in the target image.

[0080] Specifically, the question type recognition module refers to a module for recognizing all question types contained in the target image. Correspondingly, the initial question type frame refers to a rectangular frame corresponding to each question type contained in the target image. The initial question type frame is used to frame each question type contained in the target image, and each question type is framed by an initial question type frame. The character recognition module refers to a module for recognizing all characters contained in the target image. Correspondingly, the initial character frame refers to a rectangular frame corresponding to each character contained in the target image. The initial character frame is used to frame each character contained in the target image, and each character is framed by an initial character frame.

[0081] Based on this, after obtaining the target image containing the question type to be recognized, the target image can be input to the question type recognition module to recognize each question type contained in the target image to obtain an initial question type frame corresponding to each question type. Meanwhile, the target image is input to the character recognition module to recognize each character contained in the target image to obtain an initial character frame corresponding to each character. After obtaining the initial character frame and the initial question type frame, the target character frame set corresponding to the question type to be recognized can be classified in the subsequent process to complete the recognition processing operation of the question type to be recognized.

[0082] In specific implementation, the question type recognition module is a module for recognizing question types contained in the target image. Considering the nature of the processing task of the question type recognition module, YOLO v3 or Faster-RCNN target detection algorithm can be used for implementation, that is, the question type recognition module is used to locate the question types contained in the target image to obtain the initial question type frames framing each question type, thereby realizing the positioning of the positions of each question type according to the target image.

[0083] The character recognition module is a module for recognizing characters contained in the target image. Considering the nature of the processing task of the character recognition module, OCR recognition algorithm and text detection algorithm can be used for joint implementation. That is, the character recognition module is used to locate the characters contained in the target image. By combining the text detection and text recognition parts in the OCR recognition algorithm, the characters in the target image are obtained. Then, the text detection algorithm (YOLO v3 or Faster-RCNN target detection algorithm) is used to frame each character to obtain the initial character frame corresponding to each character, thereby realizing the positioning of the positions of each character according to the target image.

[0084] Further, after the identification of the question type and the character is completed by the question type identification module and the character identification module, in order to facilitate the subsequent accurate splicing of the identification result corresponding to the to-be-identified question type, the characters in the target image can also be identified in sequence according to the positions of each initial character box at this time, so as to obtain text information composed of characters as the basis, which corresponds to the target image. Facilitating the subsequent splicing of characters can also splice the characters, so that the identification result corresponding to the to-be-identified question type can contain the text content corresponding to the to-be-identified question type, to facilitate the use and processing of downstream business. Wherein, the identification of the characters in the target image can be realized by using the CRNN algorithm.

[0085] In step S1044, the target character box set corresponding to the to-be-identified question type is generated based on the initial question type box and the initial character box.

[0086] Specifically, on the basis of obtaining the initial question type box and the initial character box, further, since the target image can contain multiple question types, the obtained initial question type box will correspond to multiple question types, and the initial character box will also correspond to multiple question types. In order to accurately determine the target character box set corresponding to the to-be-identified question type, each initial character box can be divided based on the position of each initial character box and the position of each initial question type box, so as to obtain the character box set corresponding to each question type. Then, the character box set corresponding to the to-be-identified question type is selected from the character box set, that is, the target character box set, to facilitate the subsequent identification processing operation.

[0087] Further, in the process of determining the target character box set in combination with the initial character box and the initial question type box, since each question type box contains part of the initial character box, at least one initial character box set can be obtained by dividing the initial character box. Since the user only has an identification requirement for the to-be-identified question type, the target character box set can be selected in combination with the keyword of the to-be-identified question type. In this embodiment, the specific implementation manner is as follows:

[0088] Determine the character box coordinates corresponding to the initial character box and the question type box coordinates corresponding to the initial question type box based on the target image;

[0089] Divide the initial character box based on the question type box coordinates and the character box coordinates, and obtain at least one initial character box set according to the division result;

[0090] Select the initial character box set matched with the to-be-identified question type from the at least one initial character box set as the target character box set.

[0091] Specifically, the character box coordinates specifically refer to the position coordinates of the initial character boxes in the target image, and the question type box coordinates specifically refer to the position coordinates of the initial question type boxes in the target image. Correspondingly, the at least one initial character box set specifically refers to a set obtained by dividing the initial character boxes based on the question type box coordinates and the character box coordinates, and each initial character box set corresponds to a question type.

[0092] Based on this, after obtaining the initial question type boxes and the initial character boxes, in order to realize subsequent accurate identification processing of the to-be-recognized question type, the character box coordinates corresponding to each initial character box and the question type box coordinates corresponding to each initial question type box can be determined based on the target image. Then, each initial character box is divided based on the question type box coordinates and the character box coordinates. According to the division processing result, at least one initial character box set can be generated. Finally, by screening in the at least one initial character box, the target character box matching the to-be-recognized question type can be obtained.

[0093] In practical applications, the initial character boxes are divided based on the question type box coordinates and the character box coordinates. In fact, the character box combination in which the initial character boxes and the initial question type boxes exist is selected as a set. That is, the area corresponding to each question type box is determined based on the question type box coordinates. Then, the character boxes corresponding to the character box coordinates located in each area are selected to form a set. After all initial character boxes are traversed, multiple initial character box sets can be determined. Finally, the initial character box set matching the to-be-recognized question type is selected as the target character box set, and subsequent identification processing operations can be completed.

[0094] Further, in the process of screening the target character box set from the at least one initial character box set, in order to accurately select the character box set corresponding to the to-be-recognized question type, a keyword matching degree calculation method can be used to complete the selection. In this embodiment, the specific implementation is as follows:

[0095] Determine the question type keyword corresponding to the to-be-recognized question type and the keywords corresponding to each initial character box set contained in the at least one initial character box set.

[0096] Calculate the matching degree of the question type keyword and the keywords corresponding to each initial character box set.

[0097] Select the initial character box set corresponding to the keyword with the highest matching degree as the target character box set corresponding to the to-be-recognized question type.

[0098] Specifically, the question type keyword specifically refers to a keyword contained in the to-be-identified question type, and the keyword is the basis for determining the target character box set subsequently; correspondingly, the keyword corresponding to each initial character box set specifically refers to a keyword determined based on characters corresponding to character boxes contained in each initial character box set. Since each initial character box set corresponds to one question type, the keyword can be composed of characters corresponding to several character boxes ranked at the front in each initial character box set.

[0099] Based on this, in order to be able to screen out the target character box set matching the to-be-identified question type from at least one initial character box set, first, the question type keyword corresponding to the to-be-identified question type and the keyword corresponding to each initial character box set can be determined; second, the matching degree of the question type keyword and the keyword corresponding to each initial character box set is calculated; and finally, the initial character box set with the highest matching degree of the question type keyword and the keyword is selected as the target character box set, so as to accurately determine the target character box set corresponding to the to-be-identified question type and ensure the accuracy of subsequent recognition processing.

[0100] In actual application, in the process of calculating the matching degree, the highest matching degree can also be compared with a preset matching degree threshold. If the highest matching degree is greater than the preset matching degree threshold, it means that the initial character box set corresponding to the highest matching degree is the target character box set corresponding to the to-be-identified question type, which can be used for subsequent processing. If the highest matching degree is less than or equal to the preset matching degree threshold, it means that the initial character box set corresponding to the highest matching degree does not match the to-be-identified question type, which further indicates that the target image uploaded by the user may not contain the to-be-identified question type. At this time, the user can be fed back to re-collect the question type information of the target image, so as to request the user to re-collect the target image containing the to-be-identified question type, so as to achieve the purpose of re-identifying the to-be-identified question type. The preset matching degree threshold can be set according to the actual application scenario, which is not limited in this embodiment.

[0101] This embodiment takes the to-be-identified question type as the question type of writing words as an example for description, and the identification process of other question types can refer to the corresponding description content of this embodiment, which will not be described in detail here. Based on this, the target image containing the question type of writing words according to requirements uploaded by the user is obtained. At this time, the target image is input into the question type identification module for question type identification processing to obtain the question type frame B i The result of identifying the question type is shown in Figure 2 The question type frame is {B1, B2, B3, B4}, that is, the image uploaded by the user contains four question types. At the same time, in order to be able to determine the question type of writing words according to requirements, the target image is also input into the character recognition module for character recognition processing to obtain the character frame b i The result of identifying the character is as followsFigure 3 The character boxes are {b1, b2, …, bn} respectively. n}(wherein n≥0, and n is a positive integer), that is, the target image uploaded by the user contains n characters; meanwhile, each character is identified to determine the character content contained in each character box.

[0102] Further, a reference coordinate system is constructed based on the target image, and then the question type box coordinates of each question type box in the coordinate system and the character box coordinates of each character box in the coordinate system are determined. Then each question type box and each character box is traversed in turn, and the character box set S i comprising the character boxes contained in each question type box is counted. i S1={b|b∩B1=b}, S2={b|b∩B2=b}, S3={b|b∩B3=b}, and S4={b|b∩B4=b}. i That is, S1={b1, b2, …, bn1}, S2={bn1+1, bn1+2, …, bn2}, S3={bn2+1, bn2+2, …, bn3}, and S4={bn3+1, bn3+2, …, bn4}. n1 (n1)+1 (n1)+2 n2 (n2)+1 (n2)+2 n3 (n3)+1 (n3)+2 n4 n1 n2 n3 n4 n It should be noted that n1, n2, n3, and n4 are only used to represent the serial numbers of the corresponding character boxes. Since the number of character boxes obtained after identifying the characters is large, for the convenience of description, the embodiment expresses the corresponding of each character box in the form of n1, n2, n3, and n4. That is, the character boxes contained in each character box set are composed of character boxes with serial numbers increasing in order in the question type box. For the convenience of description, only the labels of the first and last character boxes in each question type are shown in the drawings, and the labels of the character boxes in the middle are omitted for the convenience of description. i That is, the text content T i corresponds to the character box set S i , the character box set S i corresponds to the question type box B i , and the question type box B​​​​​​​​​​​​​​i This corresponds to one of the four question types, so based on this mapping relationship, we can obtain four text contents, namely T1, T2, T3, and T4.

[0103] Furthermore, in order to select the target character box set that matches the "Write Words as Required" question type from the four character box sets, we can extract the keyword "Write Words as Required" corresponding to the question type, and extract the keywords corresponding to each character box set based on the text content T1, T2, T3, and T4: the keyword corresponding to character box set S1 is "Listen to the recording and answer the question", the keyword corresponding to character box set S2 is "Listen to the recording and choose the answer", the keyword corresponding to character box set S3 is "Listen to the recording and judge whether it is correct or not", and the keyword corresponding to character box set S4 is "Write Words as Required". Then, we calculate the matching degree between the keyword "Write Words as Required" corresponding to the question type and the keywords "Listen to the recording and answer the question", "Listen to the recording and choose the answer", "Listen to the recording and judge whether it is correct or not", and "Write Words as Required" corresponding to each character box set. According to the calculation results, the keyword corresponding to character box set S4 has the highest matching degree with the question type keyword. Therefore, character box set S4 can be selected as the target character box set corresponding to the "Write Words as Required" question type to assist in the subsequent recognition and processing task of the "Write Words as Required" question type.

[0104] In summary, by combining the question type recognition module and the character recognition module to process the target image, we can accurately determine the various question types and characters contained in the target image. Then, we divide each initial character frame according to the coordinates of the character frame and the question type frame, achieving accurate division of the character frame set corresponding to each question type. Finally, we filter the target character frame set corresponding to the question type to be recognized by calculating the keyword matching degree, laying the foundation for subsequent recognition processing. This ensures that the task of recognizing the question type to be recognized can be completed accurately and efficiently, enabling rapid response to user business needs and improving user experience.

[0105] Step S106: Determine the attribute information of each character box based on the category information corresponding to each character box in the target character box set.

[0106] Specifically, based on the above-mentioned generation of the target character box set corresponding to the question type to be identified, in order to accurately identify the question type to be identified and avoid the inability to distinguish the various sub-questions within the question type to be identified, the attribute information of each character box can be determined based on the category information of each character box in the target character box set. This allows for the concatenation of various character boxes by traversing the attribute information, thereby achieving the purpose of identifying the question type to be identified.

[0107] Based on this, the category information of each character box specifically refers to the category of the characters contained in each character box, such as "number" or "handwritten English". The attribute information of each character box refers to its attribute for this question type, used for the subsequent sub-question classification, including "word box attribute", "condition box attribute", "answer box attribute", and "invalid box attribute". Specifically, the word box attribute means that the characters contained in the character box are the word to be transformed; the condition box attribute means that the characters contained in the character box are the transformation requirement; the answer box attribute means that the characters contained in the character box are underlines or handwritten characters; and the invalid box attribute means that the characters contained in the character box are characters unrelated to the question type, such as the question stem or question number.

[0108] Furthermore, since each character box in the target character box set corresponding to the question type to be identified comes from each character in the question type to be identified, and the question type to be identified contains multiple sub-questions, if the character boxes involved in each sub-question are not distinguished, the question type to be identified cannot be accurately identified. Therefore, considering the differentiation of each sub-question, the attribute information of each character box can be determined based on the category information of each character box, so as to facilitate the subsequent division of each sub-question by traversing the attribute information, so as to complete the identification processing operation of the question type to be identified.

[0109] In practical applications, to quickly complete the recognition process when determining the attribute information of each character box, a finite state machine can be used. That is, the finite state machine assigns attribute information to each character box, thereby distinguishing them. In this embodiment, the specific implementation process of the finite state machine is as follows:

[0110] Based on the characters contained in each character box in the target character box set, determine the category information corresponding to each character box;

[0111] Adjust the state of each character box according to the category information, and determine the attribute information of each character box based on the state adjustment results.

[0112] Specifically, after obtaining the set of target character boxes corresponding to the question type to be identified, the characters contained in each character box in the target character box set can be read, and then the category information of each character can be determined. The category information of the characters is assigned to the character boxes, thereby determining the category information of each character box. Based on the category information corresponding to each character box, the state of each character box is adjusted, and the attribute information of each character box is determined according to the state adjustment result, which facilitates subsequent recognition processing operations.

[0113] The characters contained in each character box can be completed by the character recognition module, that is, when each character box is determined by the character recognition module, the recognition processing of each character can be completed by the character recognition module, so as to determine the character content, facilitate the determination of the attribute information of each character box, and support the recognition processing operation of the to-be-recognized question type. Correspondingly, the state adjustment can be completed based on the finite state machine, that is, the finite state machine will complete the state adjustment of each character box based on the category information, so as to achieve the purpose of determining the attribute information of each character box.

[0114] In practical applications, the finite state machine traverses the category information corresponding to each character box in turn according to the arrangement order of each character box in the target character box set, sets the state of the finite state machine based on the category information, and adjusts the state of each character box based on the state of the finite state machine, and then assigns attribute information to each character box according to the adjustment result, to facilitate subsequent recognition processing operations.

[0115] It should be noted that, in order to ensure the accuracy of the attribute information of each character box, the correspondence between the preset state and the attribute information is determined, so that the attribute information corresponding to the current state of each character box can be determined by reading the correspondence.

[0116] In addition, in different scenarios, the to-be-recognized question type that the user needs to recognize may be different. If the same state adjustment strategy is used to assign attribute information to each character box, the character box after assigning attribute information may not be able to complete subsequent splicing processing. Therefore, in order to achieve the purpose of recognition in different to-be-recognized question type recognition scenarios, different states and attribute information corresponding relationships can be preset for different question types, that is, different question types will have different states and attribute information corresponding relationships, so that in different question type recognition scenarios, the state and attribute corresponding relationship corresponding to the question type can be selected, and the attribute information of the character box involved in the question type is assigned, so that subsequent splicing and recognition processing operations can be completed. In specific implementation, the state and attribute information corresponding relationship in different question type recognition scenarios can be preset according to actual needs, as long as it can support different question type recognition processing, which is not limited in this embodiment.

[0117] In the above example, on the basis of determining that the target character box set corresponding to the write word as required question type is the character box set S4{b (n3)+1 , b (n3)+2 , … b n4}, it is determined that the character box set contains 40 character boxes at this time, that is, b n4 -b (n3)+1 =40, and each character box in turn is character box b 51 , b 52, ..., b 90 Simultaneously, the character recognition module identifies the characters contained in each character box to determine the character box b. 51 The character contained is "1"; character box b 52 The character contained is "drink"; character box b 53 The character contained is in the past tense; character box b 54 The characters contained are " drank "...; Then, based on the content of each character box, its category information is determined. In the example above, character box b..." 51 The category information is number, and the character box is b. 52 The category information is non-handwritten English, character box b 53 The category information is non-handwritten Chinese characters, character box b 54 The category information is handwritten English... and so on, until the category information corresponding to each character box is determined.

[0118] Furthermore, after determining the category information corresponding to each character box, the state of each character box can be adjusted based on Table 1 below:

[0119] Table 1

[0120] Current character box State1 State2 State3 State4 Number 4 4 4 4 Handwritten English 3 3 3 4 Non-handwritten English 1 2 1 1 Handwritten Chinese 4 4 4 4 Non-handwritten Chinese 2 2 2 4 Horizontal line 3 3 3 4 Other character 4 4 4 4

[0121] Initialize the state machine to State = 4, at which point the character box b is determined. 51 If the category information is number, the state machine will transition to State 4 and assign it to the character box b. 51 Select the next character box b 52 Character box b 52 If the category information is "non-handwritten English", then the state machine will transition to 1 and assign it to the character box b. 52 Then select the next character box b. 53 Character box b 53 If the category information is "non-handwritten Chinese characters", then the state machine will transition to State 2 and assign it to the character box b. 53 Then select the next character box b. 54 Character box b 54 If the category information is handwritten English, then the state machine will transition to State 3 and assign it to the character box b. 54 …; and so on, until the state of each character box is determined, attribute information can be assigned to each character box based on its current state. The correspondence between state and attribute information is as follows: State1 corresponds to the word box attribute, State2 corresponds to the requirement box attribute, State3 corresponds to the answer box attribute, and State4 corresponds to the invalid box attribute.

[0122] Further, according to the correspondence between the above state and attribute information, the attribute information of the character box b 51 is invalid box attribute, the attribute information of the character box b 52 is word box attribute, the attribute information of the character box b 53 is required box attribute, and the attribute information of the character box b 54 is answer box attribute, and so on, until the attribute assignment for all character boxes is completed, so as to complete the subsequent recognition processing of the required word writing type.

[0123] In summary, based on the category information of each character box, the attribute information assignment processing is performed for each character box, which not only ensures the accuracy of the assigned attribute information, but also improves the recognition processing efficiency, so as to quickly respond to downstream business processing operations and improve the user experience.

[0124] In step S108, the character boxes contained in the target character box set are spliced by traversing the attribute information of each character box, and the recognition result corresponding to the to-be-recognized question type is obtained.

[0125] Specifically, after determining the attribute information of each character box in the target character box set, the character boxes contained in the target character box set can be spliced by traversing the attribute information of each character box, so as to obtain a plurality of sub-questions contained in the to-be-recognized question type, and finally the recognition result corresponding to the to-be-recognized question type can be obtained by combining each sub-question.

[0126] In specific implementation, since each sub-question contained in the to-be-recognized question type is composed of the same parts, it can be determined that each sub-question is composed of characters contained in character boxes with different attributes, such as in the judgment type, each sub-question contains three parts of question number, question, and brackets or horizontal line for writing "x" and "√". The attribute of the character box corresponding to the question number is invalid box attribute, the attribute of the character box corresponding to the question is required box attribute, and the attribute of the character box corresponding to the brackets is answer box attribute. It is further determined that each sub-question is composed of a character box with invalid box attribute, a character box with required box attribute, and a character box with answer box attribute. Therefore, when traversing the attribute information, each character box can be spliced according to the preset traversal condition, so as to obtain the recognition result corresponding to the to-be-recognized question type.

[0127] Further, when splicing each character box contained in the target character box set, considering that the to-be-recognized question type contains a plurality of sub-questions, each sub-question can be divided while splicing the character boxes, so as to obtain the recognition result corresponding to the to-be-recognized question type. In this embodiment, the specific implementation is as follows:

[0128] Read the question recognition condition corresponding to the to-be-recognized question type;

[0129] traverse attribute information of each character box based on the question recognition condition, divide the target character box set according to the traversal result, and obtain a plurality of sub-character box sets;

[0130] Splice the character boxes contained in each sub-character box set to obtain a plurality of sub-questions, and take the plurality of sub-questions as the recognition result corresponding to the to-be-recognized question type.

[0131] Specifically, the question recognition condition specifically refers to the condition for dividing the sub-questions contained in the to-be-recognized question type. This condition is generated based on the attribute information, that is, the question recognition condition is a condition defined for the attribute information, so as to accurately divide each sub-question. Correspondingly, the sub-character box set specifically refers to a set composed of character boxes corresponding to each sub-question; and the sub-question specifically refers to a sub-question contained in the to-be-recognized question type.

[0132] Based on this, after obtaining the attribute information of each character box in the target character box set, the question recognition condition corresponding to the to-be-recognized question type can be read first, so as to determine how to divide each character box based on the attribute information; secondly, the attribute information of each character box is traversed based on the question recognition condition, so as to divide the character boxes contained in the target character box set according to the traversal result, and obtain the sub-character box set corresponding to each sub-question; finally, the character boxes contained in each sub-character box set are spliced to obtain a plurality of sub-questions, and the plurality of sub-questions are taken as the recognition result corresponding to the to-be-recognized question type.

[0133] In the above example, after determining the attribute information corresponding to each character box in the character box set S4{b 51 , b 52 , … b 90}, the question recognition condition corresponding to the word writing as required question type is read as "divide once when a character box with a state of State4 is detected (divide once when a character box with an invalid box attribute is detected)", at this time, the attribute information corresponding to each character box in the character box set S4{b 51 , b 52 , … b 90} is traversed based on the question recognition condition, the character box set S4 is divided according to the traversal result, and 10 sub-character box sets are obtained; further, the character boxes contained in each sub-character box set are spliced to determine 10 sub-questions, the positions of the character boxes corresponding to each sub-question in the target image, and the characters contained in each character box, the bounding rectangle and the text information corresponding to each sub-question are obtained, and the recognition result of the word writing as required question type is determined according to the bounding rectangle and the text information, as shown in Figure 4 , so as to facilitate the downstream business to complete the correction or answer search according to the word writing as required question type based on the recognition result.

[0134] In summary, by traversing the attribute information to splice the character box, the accuracy of each subject title included in the spliced to-be-recognized question type can be ensured, thereby realizing finer granularity of distinguishing question content, improving the recognition accuracy and efficiency of the to-be-recognized question type, and improving the user experience.

[0135] In addition, after obtaining the recognition result corresponding to the to-be-recognized question type, the user has different recognition intentions for the to-be-recognized question type in different scenarios, such as detecting the correctness of the answer intention or searching for the correct answer intention, so under different recognition intentions, different processing will be performed based on the recognition result to accurately respond to the user's recognition intention and feed back the information required by the user. In the embodiment, the first aspect will perform the following steps in the case of the detection intention as the recognition intention:

[0136] determining the recognition intention of the user according to the recognition result;

[0137] in the case of the detection intention as the recognition intention, obtaining the target answer information corresponding to the to-be-recognized question type based on the question information in the recognition result and the preset knowledge base;

[0138] comparing the target answer information with the user answer information in the recognition result, generating a detection result image according to the comparison result, and showing the detection result image to the user.

[0139] Specifically, the detection intention specifically refers to the intention of detecting the correctness of the answer information of the user in the to-be-recognized question type. Correspondingly, the question information specifically refers to the information composed of the stems corresponding to each question in the to-be-recognized question type. The target answer information specifically refers to the information corresponding to the correct answer of each question. The user answer information specifically refers to the information of the user's answer to each question. The detection result image specifically refers to the image after correcting each question included in the to-be-recognized question type.

[0140] Based on this, after obtaining the recognition result corresponding to the to-be-recognized question type, the recognition intention of the user can be determined based on the recognition result. In the case of the detection intention as the recognition intention, it is indicated that the user needs to detect the correctness of the answer. At this time, the target answer information corresponding to the to-be-recognized question type can be obtained based on the question information in the recognition result and the preset knowledge base. Then, the target answer information is compared with the user answer information in the recognition result, the correctness of the user's answer is determined, and finally a detection result image is generated based on the comparison result and is shown to the user, so that the user can determine the accuracy of the answer.

[0141] Further, when the detection result image is generated and displayed to the user, considering that the correct answer and the wrong answer can intuitively reflect the learning situation of the user, when the detection result image is generated, different two question frame displays can be selected for the correct answer and the wrong answer to facilitate the user to watch. In the embodiment, the specific implementation manner is as follows:

[0142] According to the comparison result, the color of each question frame contained in the recognition result is adjusted to obtain the correct question frame and the wrong question frame.

[0143] Based on the target answer information, the wrong question frame and the correct question frame are updated, and a detection result image is generated according to the update result and displayed to the user.

[0144] In the above example, it is determined that the required writing word type contains 10 questions, and each question contains the user's handwritten answer information. At this time, the preset knowledge base is read based on the question information of each question in the recognition result to obtain the target answer information corresponding to each question. Then, the target answer information corresponding to each question is compared with the user's answer information to determine that question 1 is answered incorrectly, and questions 2 to 10 are answered correctly. At this time, it is determined that the rectangular frame corresponding to question 1 is a wrong question frame (red), and the rectangular frames corresponding to questions 2 to 10 are correct question frames (green). Then, in order to facilitate the user to understand the correct answer rate and the correct answer of each question, the wrong question frame and the correct question frame can be updated based on the target answer information of each question, and a detection result image as shown in Figure 5 is generated according to the update result and displayed to the user (where “****” represents the user's handwritten answer).

[0145] In summary, when the user's intention is the detection intention, in order to facilitate the user to understand the correct answer rate and the correct answer information of each question, different colors can be selected to display the answer of different questions, and the target answer information can be added in the detection result image to achieve the purpose of assisting the user to learn.

[0146] The second aspect is that when the recognition intention is the search intention, the following steps are performed:

[0147] According to the recognition result, the recognition intention of the user is determined.

[0148] When the recognition intention is the search intention, the target answer information corresponding to the to-be-recognized question type is obtained by reading the preset knowledge base based on the question information in the recognition result.

[0149] A search result image is generated based on the question information in the recognition result and the target answer information, and is displayed to the user.

[0150] Specifically, the search intention specifically refers to the intention of searching for correct answers of the questions in the to-be-recognized question type; correspondingly, the question information specifically refers to information composed of stems corresponding to the questions in the to-be-recognized question type; and the target answer information specifically refers to information corresponding to correct answers of the questions.

[0151] Based on this, after obtaining the recognition result corresponding to the to-be-recognized question type, the recognition intention of the user can be determined based on the recognition result. In the case where the recognition intention is a search intention, it indicates that the user needs to search for correct answers of the questions contained in the to-be-recognized question type. At this time, the preset knowledge base can be read based on the question information in the recognition result, so as to obtain the target answer information corresponding to the to-be-recognized question type. Then, by integrating the question information and the target answer information in the recognition result, a search result image can be generated and displayed to the user, so as to facilitate the user to understand the correct answer content of each question.

[0152] Following the above example, it is determined that the write-by-demand question type contains 10 questions, and each question is not answered. At this time, the preset knowledge base is read based on the question information of each question in the recognition result, and the target answer information corresponding to each question is obtained. Then, by integrating the target answer information of each question and the question information of each question, a search result image as shown in FIG. 10B can be obtained and displayed to the user, so that the user can determine the correct answer of each question. Figure 6

[0153] In summary, in the case where the user intention is a search intention, in order to facilitate the user to understand the correct answer of each question, the target answer information can be added in the generated search result image, so as to achieve the purpose of assisting the user in learning.

[0154] In actual application, the determination of the recognition intention can be determined by detecting whether the recognition result contains user handwriting content. If the recognition result contains user handwriting content, it indicates that the probability of the user performing correctness detection on the to-be-recognized question type is relatively high. At this time, it can be determined that the recognition intention of the user is a detection intention. If the recognition result does not contain user handwriting content, it indicates that the probability of the user searching for answers of the to-be-recognized question type is relatively high. At this time, it can be determined that the recognition intention of the user is a search intention.

[0155] ​The recognition method provided in the specification can perform recognition processing on the target image after obtaining the target image input by the user and containing a to-be-recognized question type, to generate a target character box set corresponding to the to-be-recognized question type according to the recognition result, then determine the attribute information of each character box based on the category information corresponding to each character box in the target character box set, and finally splice the character boxes contained in the target character box set by traversing the attribute information of each character box, so as to obtain the recognition result corresponding to the to-be-recognized question type according to the splicing result, which realizes splicing of the text content corresponding to the to-be-recognized question type based on the attribute information of the character box, can guarantee the accuracy of recognizing the to-be-recognized question type, improve the recognition efficiency of the question type, and thus can quickly feed back the recognition result to the user and improve the user experience.

[0156] The following describes the recognition method provided in the specification in combination with the accompanying Figure 7 The recognition method provided in the specification in the application of the answer search scene is taken as an example to further illustrate the recognition method. Among them, Figure 7 FIG. 1 shows a processing flowchart of the recognition method provided in an embodiment of the specification and applied in the answer search scene, which specifically includes the following steps:

[0157] In step S702, a target image containing a to-be-recognized question type input by a user is obtained.

[0158] In step S704, the target image is input to a question type recognition module for processing to obtain an initial question type box corresponding to the question type contained in the target image.

[0159] In step S706, the target image is input to a character recognition module for processing to obtain an initial character box corresponding to a character contained in the target image.

[0160] In step S708, character box coordinates corresponding to the initial character box and question type box coordinates corresponding to the initial question type box are determined based on the target image.

[0161] In step S710, the initial character box is divided and processed based on the question type box coordinates and the character box coordinates to obtain a plurality of initial character box sets.

[0162] In step S712, a question type keyword corresponding to the to-be-recognized question type and a keyword corresponding to each initial character box set are determined.

[0163] In step S714, a target character box set corresponding to the to-be-recognized question type is selected by calculating the matching degree of the question type keyword and the keyword corresponding to each initial character box set.

[0164] In step S716, category information corresponding to each character box is determined based on the character corresponding to each character box in the target character box set.

[0165] In step S718, the state of each character box is adjusted according to the category information, and attribute information of each character box is determined according to the state adjustment result.

[0166] In step S720, the character boxes included in the target character box set are spliced by traversing the attribute information of each character box, and a recognition result corresponding to the to-be-recognized question type is obtained.

[0167] In step S722, target answer information corresponding to the to-be-recognized question type is obtained based on the question information in the recognition result and a preset knowledge base.

[0168] In step S724, a search result image is generated based on the question information in the recognition result and the target answer information, and is displayed to the user.

[0169] The recognition method provided in the specification realizes splicing of text content corresponding to the to-be-recognized question type based on attribute information of character boxes, which not only ensures the accuracy of recognizing the to-be-recognized question type, but also improves the recognition efficiency of the question type, so as to quickly feed back the recognition result to the user and improve the user experience.

[0170] Corresponding to the above method embodiment, the specification also provides a recognition device embodiment, Figure 8 A structure schematic diagram of a recognition device provided by an embodiment of the specification is shown. As shown in the figure, Figure 8 The device comprises:

[0171] The acquisition module 802 is configured to acquire a target image input by a user and containing a to-be-recognized question type;

[0172] The recognition module 804 is configured to perform recognition processing on the target image, and generate a target character box set corresponding to the to-be-recognized question type according to a recognition processing result;

[0173] The determination module 806 is configured to determine attribute information of each character box in the target character box set based on category information corresponding to each character box;

[0174] The splicing module 808 is configured to splice the character boxes included in the target character box set by traversing the attribute information of each character box, and obtain a recognition result corresponding to the to-be-recognized question type.

[0175] In an optional embodiment, the recognition module 804 is further configured to:

[0176] input the target image into a question type recognition module for processing to obtain an initial question type frame corresponding to a question type contained in the target image, and input the target image into a character recognition module for processing to obtain an initial character frame corresponding to a character contained in the target image; and generate the target character frame set corresponding to the to-be-recognized question type based on the initial question type frame and the initial character frame.

[0177] In an optional embodiment, the recognition module 804 is further configured to:

[0178] determine character frame coordinates corresponding to the initial character frame and question type frame coordinates corresponding to the initial question type frame based on the target image; perform division processing on the initial character frame based on the question type frame coordinates and the character frame coordinates; and obtain at least one initial character frame set according to a division processing result; and select an initial character frame set matching the to-be-recognized question type from the at least one initial character frame set as the target character frame set.

[0179] In an optional embodiment, the recognition module 804 is further configured to:

[0180] determine a question type keyword corresponding to the to-be-recognized question type and a keyword corresponding to each initial character frame set contained in the at least one initial character frame set; calculate a matching degree of the question type keyword and the keyword corresponding to each initial character frame set; and select an initial character frame set corresponding to a keyword with a highest matching degree as the target character frame set corresponding to the to-be-recognized question type.

[0181] In an optional embodiment, the determination module 806 is further configured to:

[0182] determine category information corresponding to each character frame based on a character contained in each character frame in the target character frame set; perform state adjustment on each character frame according to the category information; and determine attribute information of each character frame according to a state adjustment result.

[0183] In an optional embodiment, the category information corresponding to each character frame includes at least one of the following:

[0184] a word unit category, a condition category, an answer category, and an invalid category.

[0185] In an optional embodiment, the splicing module 808 is further configured to:

[0186] read a question recognition condition corresponding to the to-be-recognized question type; traverse attribute information of each character box based on the question recognition condition, divide the target character box set according to a traversal result, and obtain a plurality of sub-character box sets; and splice character boxes included in each sub-character box set to obtain a plurality of question subjects, and take the plurality of question subjects as the recognition result corresponding to the to-be-recognized question type.

[0187] In an optional embodiment, the recognition device further includes:

[0188] The detection module is configured to determine the recognition intention of the user according to the recognition result, read target answer information corresponding to the to-be-recognized question type from a preset knowledge base based on question information in the recognition result in a case where the recognition intention is a detection intention, compare the target answer information with user answer information in the recognition result, generate a detection result image according to a comparison result, and display the detection result image to the user.

[0189] In an optional embodiment, the detection module is further configured to:

[0190] adjust a color of each question box included in the recognition result according to a comparison result, obtain a correct question box and an incorrect question box, update the incorrect question box and the correct question box based on the target answer information, generate the detection result image according to an update result, and display the detection result image to the user.

[0191] In an optional embodiment, the recognition device further includes:

[0192] The search module is configured to determine the recognition intention of the user according to the recognition result, read target answer information corresponding to the to-be-recognized question type from a preset knowledge base based on question information in the recognition result in a case where the recognition intention is a search intention, and generate a search result image based on the question information in the recognition result and the target answer information, and display the search result image to the user.

[0193] The recognition device provided in the embodiment can perform recognition processing on the target image after obtaining the target image input by the user and containing a to-be-recognized question type, to generate a target character box set corresponding to the to-be-recognized question type according to a recognition result, determine attribute information of each character box in the target character box set based on category information corresponding to each character box, and finally splice the character boxes contained in the target character box set by traversing the attribute information of each character box, so as to obtain the recognition result corresponding to the to-be-recognized question type according to the splicing result, realize splicing of text content corresponding to the to-be-recognized question type based on the attribute information of the character box, and not only can guarantee the accuracy of recognizing the to-be-recognized question type, but also can improve the recognition efficiency of the question type, so that the recognition result can be quickly fed back to the user, and the use experience of the user is improved.

[0194] The above is a schematic scheme of the recognition device of the embodiment. It should be noted that the technical scheme of the recognition device belongs to the same concept as the technical scheme of the recognition method described above, and the details of the technical scheme of the recognition device that are not described in detail can be referred to the description of the technical scheme of the recognition method.

[0195] Figure 9 A structural block diagram of a computing device 900 according to an embodiment of the present specification is shown. The components of the computing device 900 include, but are not limited to, a memory 910 and a processor 920. The processor 920 is connected to the memory 910 through a bus 930, and a database 950 is used to save data.

[0196] The computing device 900 also includes an access device 940, which enables the computing device 900 to communicate via one or more networks 960. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 940 can include one or more of any type of network interface (e.g., network interface card (NIC)), such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a near-field communication (NFC) interface, and the like.

[0197] In an embodiment of the present specification, the above-mentioned components of the computing device 900 and other components not shown in the Figure 9 may be connected to each other, for example, through a bus. It should be understood that Figure 9 The structural block diagram of the computing device shown is only for the purpose of example, and is not a limitation on the scope of the present specification. Those skilled in the art can add or replace other components as needed.

[0198] The computing device 900 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. The computing device 900 can also be a mobile or stationary server.

[0199] The processor 920 is configured to execute computer-executable instructions, including:

[0200] obtain a target image containing a to-be-recognized question type input by a user;

[0201] perform recognition processing on the target image, and generate a target character box set corresponding to the to-be-recognized question type according to a recognition processing result;

[0202] determine attribute information of each character box in the target character box set based on category information corresponding to each character box;

[0203] splice character boxes contained in the target character box set by traversing the attribute information of each character box, to obtain a recognition result corresponding to the to-be-recognized question type.

[0204] The above is a schematic scheme of the computing device of the embodiment. It should be noted that the technical scheme of the computing device belongs to the same concept as the technical scheme of the above-mentioned recognition method, and the details of the technical scheme of the computing device that are not described in detail can be referred to the description of the technical scheme of the above-mentioned recognition method.

[0205] An embodiment of the present specification also provides a computer readable storage medium storing computer instructions, which are executed by a processor to:

[0206] obtain a target image containing a to-be-recognized question type input by a user;

[0207] perform recognition processing on the target image, and generate a target character box set corresponding to the to-be-recognized question type according to a recognition processing result;

[0208] determine attribute information of each character box in the target character box set based on category information corresponding to each character box;

[0209] splice character boxes contained in the target character box set by traversing the attribute information of each character box, to obtain a recognition result corresponding to the to-be-recognized question type.

[0210] The above is a schematic solution of the computer readable storage medium of the embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the identification method described above belong to the same concept, and the details of the technical solution of the storage medium that are not described in detail can be referred to the description of the technical solution of the identification method.

[0211] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in an order other than that described in the embodiments and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous or possible.

[0212] The computer instructions include computer program code, which can be in the form of source code, object code, executable code, or some intermediate form. The computer readable medium can include any entity or apparatus capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content contained in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0213] It should be noted that for the foregoing method embodiments, in order to facilitate description, they are all expressed as a combination of a series of acts, but those skilled in the art should know that the present specification is not limited by the order of the described acts, because according to the present specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the acts and modules involved are not necessarily all necessary for the present specification.

[0214] In the above embodiments, the description of each embodiment is focused on, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0215] The preferred embodiments of the present specification disclosed above are only used to help illustrate the present specification. Alternative embodiments do not describe all the details and limit the present application to the specific embodiments described. Obviously, many modifications and changes can be made according to the content of the present specification. The present specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present specification, so that those skilled in the art can well understand and utilize the present specification. The present specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method of identification, characterized in that, The method comprises the following steps: obtaining a target image input by a user, the target image containing a to-be-recognized question type and other questions belonging to the same page as the to-be-recognized question type; performing recognition processing on the target image, and generating a target character box set corresponding to the to-be-recognized question type according to a recognition processing result; generating a target character box set corresponding to the to-be-recognized question type according to a recognition processing result, comprising: dividing initial character boxes based on positions of the initial character boxes and positions of initial question type boxes, obtaining at least one initial character box set, and screening a target character box set from the at least one initial character box set in combination with a keyword of the to-be-recognized question type, wherein the initial question type box is a rectangular box corresponding to each question type contained in the target image, and the initial character box is a rectangular box corresponding to each character contained in the target image; determining attribute information of each character box based on category information corresponding to each character box in the target character box set; splicing character boxes contained in the target character box set by traversing the attribute information of each character box, and obtaining a recognition result corresponding to the to-be-recognized question type.

2. The identification method according to claim 1, characterized in that, The recognition processing on the target image comprises: inputting the target image into a question type recognition module for processing, and obtaining an initial question type box corresponding to a question type contained in the target image; and inputting the target image into a character recognition module for processing, and obtaining an initial character box corresponding to a character contained in the target image.

3. The identification method according to claim 2, characterized in that, The dividing of the initial character boxes based on the positions of the initial character boxes and the positions of the initial question type boxes, the obtaining of at least one initial character box set, and the screening of a target character box set from the at least one initial character box set in combination with a keyword of the to-be-recognized question type comprise: determining character box coordinates corresponding to the initial character boxes and question type box coordinates corresponding to the initial question type boxes based on the target image; performing division processing on the initial character boxes based on the question type box coordinates and the character box coordinates, and obtaining at least one initial character box set according to a division processing result; screening an initial character box set matching the to-be-recognized question type from the at least one initial character box set in combination with a keyword of the to-be-recognized question type, and taking the initial character box set as the target character box set.

4. The identification method according to claim 3, characterized in that, The screening of an initial character box set matching the to-be-recognized question type from the at least one initial character box set in combination with a keyword of the to-be-recognized question type, and taking the initial character box set as the target character box set, comprises: determining a question type keyword corresponding to the to-be-recognized question type and keywords corresponding to each initial character box set contained in the at least one initial character box set; calculating matching degrees of the question type keyword and the keywords corresponding to each initial character box set; selecting an initial character box set corresponding to a keyword with the highest matching degree, and taking the initial character box set as the target character box set corresponding to the to-be-recognized question type.

5. The identification method according to claim 1, characterized in that, The determination of attribute information of each character box based on category information corresponding to each character box in the target character box set comprises: determining category information corresponding to each character box based on characters contained in each character box in the target character box set. According to the category information, the state of each character frame is adjusted, and attribute information of each character frame is determined according to a state adjustment result.

6. The identification method according to any one of claims 1 to 5, characterized in that, The category information corresponding to each character frame includes at least one of: a word unit category, a condition category, an answer category, and an invalid category.

7. The identification method of claim 1, wherein, The character frames included in the target character frame set are spliced by traversing the attribute information of each character frame, and an identification result corresponding to the to-be-identified question type is obtained, including: reading a question identification condition corresponding to the to-be-identified question type; traversing the attribute information of each character frame based on the question identification condition, dividing the target character frame set according to a traversal result, and obtaining a plurality of sub-character frame sets; splicing the character frames included in each sub-character frame set to obtain a plurality of question titles, and taking the plurality of question titles as the identification result corresponding to the to-be-identified question type.

8. The identification method of claim 1, wherein, After the step of splicing the character frames included in the target character frame set by traversing the attribute information of each character frame to obtain the identification result corresponding to the to-be-identified question type is executed, the method further includes: determining an identification intention of the user according to the identification result; in a case where the identification intention is a detection intention, reading target answer information corresponding to the to-be-identified question type from a preset knowledge base based on question information in the identification result; comparing the target answer information with user answer information in the identification result, generating a detection result image according to a comparison result, and displaying the detection result image to the user.

9. The identification method according to claim 8, characterized in that, The method further includes: adjusting a color of each question frame included in the identification result according to the comparison result to obtain a correct question frame and an incorrect question frame; updating the incorrect question frame and the correct question frame based on the target answer information, generating the detection result image according to an update result, and displaying the detection result image to the user.

10. The identification method of claim 1, wherein, After the step of splicing the character frames included in the target character frame set by traversing the attribute information of each character frame to obtain the identification result corresponding to the to-be-identified question type is executed, the method further includes: determining an identification intention of the user according to the identification result; in a case where the identification intention is a search intention, reading target answer information corresponding to the to-be-identified question type from a preset knowledge base based on question information in the identification result; generating a search result image based on the question information in the identification result and the target answer information, and displaying the search result image to the user.

11. An identification device, characterized in that The method includes: an acquisition module configured to acquire a target image including a to-be-identified question type input by a user; an identification module configured to perform identification processing on the target image, and generate a target character frame set corresponding to the to-be-identified question type according to an identification processing result. The recognition module is further configured to generate a set of target character boxes corresponding to the question type to be recognized based on the recognition processing result, including: dividing the initial character boxes based on the position of each initial character box and the position of each initial question type box to obtain at least one set of initial character boxes; and selecting a set of target character boxes from the at least one set of initial character boxes in combination with the keywords of the question type to be recognized, wherein the initial question type box is a rectangle corresponding to each question type contained in the target image, and the initial character box is a rectangle corresponding to each character contained in the target image; The determination module is configured to determine the attribute information of each character box based on the category information corresponding to each character box in the target character box set; The splicing module is configured to splice the character boxes contained in the target character box set by traversing the attribute information of each character box to obtain the recognition result corresponding to the question type to be recognized.

12. A computing device, comprising: include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium storing computer instructions, wherein, When executed by a processor, this instruction implements the steps of the method according to any one of claims 1 to 10.

14. A computer program product, characterised in that, Includes computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 10.

Images