A robot operation batch correction image recognition method based on double-engine cascading and layered subgraph splicing

Abstract

The application discloses a robot operation batch correction image recognition method based on double-engine cascading and layered subgraph splicing, which comprises the following steps: receiving an image to be recognized and performing validity inspection; calling a first optical character recognition engine to perform recognition; if the first engine returns non-empty content, directly outputting and marking the source; if the first engine content is empty, performing deep traversal splicing on the layered subgraph structure of the returned result, extracting text and marking the source; if the first engine fails or there is no effective text after the layered subgraph splicing, automatically triggering a second multi-modal visual recognition engine; if the second engine succeeds, outputting the text and marking the source, and if both the engines fail, returning error information. Through the hierarchical degradation strategy, the application effectively controls the calling frequency and recognition cost of the multi-modal visual recognition engine while ensuring the recognition success rate.

Description

Technical Field

[0001] This invention relates to the field of robot data processing technology, specifically to a robot homework correction image recognition method based on dual-engine cascaded recognition and hierarchical subgraph text splicing, applicable to educational auxiliary robots, automatic homework correction systems, intelligent image and text recognition and other scenarios. Background Technology

[0002] In the fields of intelligent education and robot-assisted teaching, homework correction is a core component. After acquiring images of student assignments, robots need to accurately extract the written text content from the images for subsequent semantic understanding, answer comparison, and scoring feedback. Currently, image text recognition mainly relies on optical character recognition (OCR) technology. However, in homework correction scenarios, students' handwriting styles vary, the degree of illegibility differs, and image shooting conditions are variable, including differences in lighting, angle, and resolution. A single OCR engine experiences significant fluctuations in recognition rate when faced with complex handwriting and blurry printed fonts, easily resulting in empty recognition content or insufficient confidence, causing interruptions in the homework correction process.

[0003] Some solutions attempt to incorporate the visual recognition capabilities of multimodal visual recognition engines as a supplement. However, the high cost and response latency of calling large language models make them suitable only when the optical character recognition engine returns empty results, balancing efficiency and cost. Furthermore, some optical character recognition engines return results organized in a hierarchical subgraph structure, where the top-level Content field may be empty, while the underlying SubImages, BlockInfo, and ParagraphInfo still contain valid text information. Failure to traverse and extract these hierarchical structures will result in the loss of valid text content, triggering unnecessary calls to secondary engines.

[0004] Therefore, there is an urgent need for a robot-based image recognition method that can perform depth-first traversal and splicing extraction of the hierarchical subgraph structure returned by the optical character recognition engine, and automatically cascade and enable a multimodal visual recognition engine when the main engine output is empty, so as to improve the completeness of text extraction and recognition success rate while controlling the calling cost and response latency. Summary of the Invention

[0005] The purpose of this invention is to provide a robot task correction image recognition method based on dual-engine cascading and hierarchical subgraph stitching, which solves the problems of text loss due to empty content returned by a single optical character recognition engine in the task handwriting recognition scenario, and unnecessary cost and response delay caused by direct cascading calls to multimodal visual recognition engines, so as to achieve an intelligent image recognition solution that balances recognition integrity, response speed and calling cost.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A robot task grading image recognition method based on dual-engine cascading and hierarchical subgraph stitching, characterized by comprising at least the following steps: S1. Receive image data to be identified, wherein the image data is a captured image of the work to be corrected obtained by the robot through the image acquisition module; S2. Perform a validity check on the received image data to determine whether the image data meets the preset minimum resolution threshold and minimum total pixel requirements. If it does not meet the requirements, refuse to process it and return a prompt message that the image quality does not meet the standards. S3. After the image data validity check is passed, the first optical character recognition engine is called to perform text recognition processing on the image data. The first optical character recognition engine is a general optical character recognition engine. S4. Analyze the return result of the first optical character recognition engine. If the return status is successful and the top-level Content field is not empty, then directly use the text content in the Content field as the recognition output and mark the text source as the first engine. S5. If the return status is successful but the top-level Content field is empty, then the SubImages array in the returned result is traversed hierarchically, and the text line content in BlockInfo.BlockDetails and the paragraph text content in ParagraphInfo.ParagraphDetails inside each SubImages element are extracted in turn. The text blocks are sorted and spliced ​​according to the reading order indicated by BlockInfo to generate complete recognition output text, and the text source is marked as the first engine sub-image splicing. S6. If the first optical character recognition engine returns a failure state, or if no valid text is extracted after traversing the layered sub-images in S5, the second recognition engine is automatically triggered to encode the image data into a preset image format and send it to the visual recognition interface of the multimodal visual recognition engine for visual text recognition. S7. Analyze the return result of the second recognition engine. If the return status is successful and the recognition result contains text content, extract the text content as the recognition output and mark the source of the text as the second engine. S8. If the second recognition engine also returns a failure status or the recognition result does not contain any valid text, an error message containing error codes and error descriptions is generated, and this round of recognition is terminated. The method is executed by the robot end-side processor.

[0007] Compared with the prior art, the present invention has the following advantages: First, a dual-engine cascaded architecture is adopted, with an optical character recognition engine as the main engine and a multimodal visual recognition engine as the backup engine. The secondary engine is only activated when the main engine completely fails, effectively controlling the frequency of multimodal visual recognition engine calls and overall recognition costs, while ensuring a high recognition success rate. This technical effect is not something that those skilled in the art could readily deduce from the prior art before understanding the technical solution of this invention.

[0008] Second, when the optical character recognition engine returns an empty content, a depth-first traversal and concatenation extraction of the hierarchical subgraph structure is prioritized. This fully utilizes the structured intermediate results returned by the recognition engine, avoiding the triggering of more expensive secondary engine calls due to the failure of the top-level field being empty, thus reducing unnecessary scheduling overhead. This technical effect is not something that those skilled in the art could readily deduce from the prior art before knowing the technical solution of this invention.

[0009] Third, clearly labeling the source of the extracted data facilitates differentiated confidence assessments and manual verification strategies in subsequent business processes based on the source type. This technical effect is not something that those skilled in the art could readily derive from existing technologies before understanding the technical solution of this invention. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating the overall process of the robot task grading image recognition method based on dual-engine cascading and hierarchical subgraph stitching in this invention. The meanings of the markings in the diagram are as follows: 100 - Receive image data to be recognized; 101 - Verify the validity of the image data; 102 - Call the first engine OCR recognition; 103 - Parse the result returned by the first engine; 104 - Output text and label the source as the first engine; 105 - Perform layered sub-image text splicing; 106 - Output spliced ​​text and label the source as the first engine sub-image splicing; 107 - Automatically trigger the second engine visual recognition; 108 - Parse the result returned by the second engine; 109 - Output text and label the source as the second engine; 110 - Return error information; 111 - Determine if the first engine is successful and has content; 112 - Determine if the second engine is successful; 113 - Determine if the layered sub-image splicing has valid text; 114 - Prompt if the image quality is substandard. Figure 2This is a schematic diagram of the first engine's hierarchical subgraph structure traversal and text concatenation in an embodiment of the present invention. The meanings of the markings in the diagram are as follows: 200 - First engine returns the result object; 201 - Top-level Content field is empty; 202 - SubImages array is expanded; 203 - BlockInfo field extraction; 204 - BlockDetails text line traversal; 205 - ParagraphInfo field extraction; 206 - ParagraphDetails paragraph text traversal; 207 - Sorting and concatenating according to reading order; 208 - Deduplication module; 209 - Output of complete concatenated text.

[0011] Figure 3 This is a comparative diagram of the dual-engine cascaded degradation strategy and the automatic routing mechanism in an embodiment of the present invention. The meanings of the markings in the diagram are as follows: 300 - Image data input, 301 - First engine OCR recognition result, 302 - Successful path with non-empty content (direct output), 303 - Successful path with empty content (sub-image stitching), 304 - Failure or no valid text path (triggers the second engine), 305 - Second engine visual recognition result, 306 - Successful output path, 307 - Failure returns an error path, 308 - Continuous failure counting routing module. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. The following embodiments are for illustrative purposes only and do not constitute a limitation thereof. Example

[0013] This embodiment provides a robot-based image recognition method for grading student assignments, based on dual-engine cascading and hierarchical subgraph stitching, executed by an edge processor deployed on an educational auxiliary robot. The method is automatically triggered after the robot acquires an image of the student's assignment to be graded via a camera or scanning module.

[0014] In S1, the robot acquires images of the work to be graded through an image acquisition module, which can be a camera on the robot's head, a document scanner, or an external document scanner. The image data is transmitted to the recognition pipeline as a byte stream with a preset encoding format, carrying metadata information from the time of image acquisition, including the acquisition timestamp, image resolution, and acquisition device identifier. The receiving module temporarily stores the image data in a memory buffer, awaiting subsequent validity verification.

[0015] In S2, the received image data undergoes a validity check. This check includes: verifying whether the image data is complete and decodeable; verifying whether the image resolution is not lower than a preset minimum resolution threshold; and verifying whether the total number of pixels in the image is not lower than a preset minimum total number of pixels. If any check fails, the current recognition process terminates, and a message containing the specific failure details is generated and returned to the caller, prompting the user to retake a clear, complete image with sufficient resolution.

[0016] In S3, once the image data passes the validity check, it is encoded according to the format required by the general optical character recognition engine interface. An HTTP request is then constructed and sent to the general optical character recognition engine's server endpoint. The request carries the image data, preset recognition parameters, including the language type to be recognized, whether handwriting recognition mode is enabled, etc., and waits for the recognition result to be returned synchronously. Internally, the engine service uses a deep learning model for text detection and recognition, returning a structured recognition result object.

[0017] In S4, the structured result object returned by the first optical character recognition engine is parsed. First, the returned HTTP status code and business status code are checked. If the status code indicates a successful request, and the top-level Content field of the returned result object is not an empty string or null, then the text content in the top-level Content field is directly used as the final output text for this round of recognition. A source annotation field is appended to the output text, indicating that its source is the first engine. The source annotation uses a preset tag format, adding a `source` field to the output JSON structure with a value of `engine1_direct`.

[0018] In S5, when the return status is success but the top-level Content field is an empty string or null, such as Figure 2As shown in marks 200 to 209, the SubImages array 202 in the returned result object 200 is expanded and traversed. The BlockInfo field 203 and ParagraphInfo field 205 within each SubImages element are extracted sequentially. Text line content is obtained from BlockDetails 204, and paragraph text content is obtained from ParagraphDetails 206. The total number of elements in the SubImages array in the returned result is counted. If the total number is greater than zero, each SubImages element is traversed one by one. For each SubImages element, its BlockInfo field is extracted, and then the BlockDetails array is extracted from the BlockInfo. The text line content contained in the Text property of each BlockDetails element is then obtained. Simultaneously, the ParagraphInfo field is extracted, and then the ParagraphDetails array is extracted from the ParagraphInfo. The paragraph text content contained in the Text property of each ParagraphDetails element is then obtained. The reading order of each text line and paragraph is determined by the preset sorting field in BlockInfo. All extracted text content is concatenated in reading order to generate complete recognition output text, and the source is labeled as the first engine subimage concatenation, with the source field value being engine1_subimage. If no non-empty text is extracted after traversing SubImages, the concatenation result is marked as empty.

[0019] In S6, if the first optical character recognition engine returns a failure status, or if no valid text content is obtained after the hierarchical subgraph traversal and splicing in S5, the second recognition engine is automatically triggered. For example... Figure 3 As shown in marks 300 to 307, when the first engine fails or the sub-image stitching has no valid text, the second recognition engine 305 is automatically triggered via the failed path 304. The dual-engine cascading degradation strategy attempts the following steps in order of priority: first, the direct output path 302; second, the sub-image stitching path 303; and finally, the second engine path 304. The triggering process includes: re-encoding or converting the original image data according to the preset image format required by the multimodal visual recognition engine's visual recognition interface; sending the encoded image data, along with the preset recognition prompt text, as part of the request body to the multimodal visual recognition engine's API endpoint. The preset prompt text explicitly instructs the model to target text recognition and requires the model to output all visible text content in the image in the original layout order.

[0020] In S7, the response returned by the second recognition engine is parsed. If the returned HTTP status code is successful and the text content field in the response body is not empty, the value of the text content field is extracted as the final output text for this round of recognition, and the source is labeled as the second engine, with the source field value being engine2_mllm. The text content returned by the second engine is arranged according to the layout and reading order in the original image.

[0021] In S8, if the second recognition engine also returns a failure status, or returns a success status but the text content field is empty, then this round of dual-engine recognition has failed. At this point, an error message object is generated, containing an error code and an error description. The error code is taken from a preset error classification table, and the error description is human-readable prompt text, clearly informing the caller that the current image cannot be recognized by either engine. The error message object also includes a thumbnail reference or data summary of the original image data for the caller to manually troubleshoot. At this point, the recognition process for this round terminates.

[0022] In the actual operation of this embodiment, the robot's end-side processor is responsible for scheduling and executing the entire process. The image acquisition module transmits image data to the entry point of the recognition pipeline via an internal bus. Each sub-module of the recognition pipeline is executed sequentially in different threads or coroutines on the processor. Data transfer between steps is accomplished through shared data structures in memory, avoiding unnecessary disk read / write operations and reducing end-to-end recognition latency.

[0023] The layered sub-image text splicing mechanism described in this embodiment fully utilizes the intermediate results generated by the optical character recognition engine during the layout analysis and region recognition stages to retrieve text information that was originally discarded due to the top-level Content being empty. This mechanism has practical value in scenarios such as recognizing table-based assignments and column-formatted exam paper images. The layout structure of such images is complex, and the optical character recognition engine will store the text of each region separately in the sub-layer level instead of merging it into the top-level Content field.

[0024] The second recognition engine employs a multimodal visual recognition engine. Its visual recognition capability is based on a large-scale image and text pre-trained model, demonstrating good generalization ability for content that is difficult to recognize in traditional optical character recognition engines, such as extremely illegible handwriting, artistic fonts, and seal characters. However, the computational resources and network bandwidth required for a single call are greater than those of the optical character recognition engine, so it is only invoked when necessary. This invention achieves a good balance between recognition success rate and computational cost through a refined two-level degradation strategy: first, layered sub-image stitching, and then a large language model as a backup. Example

[0025] This embodiment, based on Embodiment 1, further refines the hierarchical sub-image text concatenation strategy in S5. When traversing the SubImages array, each SubImages element is pre-checked to ensure its BlockInfo field exists and is not empty. If BlockInfo is empty, the SubImages element is skipped, and the process continues to the next element to avoid null pointer exceptions. Simultaneously, before concatenating the text, each text block is deduplicated: if two adjacent text blocks are identical after normalization (i.e., identical after removing spaces and punctuation differences), only the first text block is retained, and subsequent duplicate blocks are discarded. The destination threshold can be determined using a preset similarity threshold for fuzzy matching to improve the cleanliness of the concatenation result. Example

[0026] This embodiment, based on Embodiment 1, further accumulates the count of failures or empty results returned by the first engine in S6. If, in the same recognition session, the number of consecutive failures or empty results returned by the first engine exceeds a preset consecutive failure threshold, the subsequent image recognition request is automatically routed directly to the second engine, skipping S3 to S5, to shorten the average response time in the scenario of consecutive recognition failures. When a subsequent frame of image is successfully recognized by the second engine, the counter is reset to zero, and the default dual-engine cascade process is restored.

[0027] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A robot task grading image recognition method based on dual-engine cascading and hierarchical subgraph stitching, characterized in that, The process includes at least the following steps: S1, receiving image data to be identified, wherein the image data is a captured image of the work to be corrected obtained by the robot through the image acquisition module; S2, performing a validity check on the received image data to determine whether the image data meets the preset minimum resolution threshold and minimum total pixel requirement. If it does not meet the requirements, the processing is rejected and a prompt message indicating that the image quality is substandard is returned. S3. After the image data validity check is passed, the first optical character recognition engine is called to perform text recognition processing on the image data; S4. Analyze the return result of the first optical character recognition engine. If the return status is successful and the top-level Content field is not empty, the text content in the Content field is directly used as the recognition output, and the text source is marked as the first engine. S5. If the return status is successful but the top-level Content field is empty, the SubImages array in the return result is traversed hierarchically. The text line content in BlockInfo.BlockDetails and the paragraph text content in ParagraphInfo.ParagraphDetails inside each SubImages element are extracted in turn. The text is then spliced ​​into a complete text output according to the sorting information in BlockInfo, and the text source is marked as the first engine sub-image splicing. S6. If the first optical character recognition engine returns a failure status, or if no valid text is extracted after traversing the hierarchical sub-images in S5, the second recognition engine is automatically triggered. The image data is encoded into a preset image format and sent to the visual recognition interface of the multimodal visual recognition engine for visual text recognition. S7. Analyze the return result of the second recognition engine. If the return status is successful and the recognition result contains text content, extract the text content as the recognition output and mark the source of the text as the second engine. S8. If the second recognition engine also returns a failure status or the recognition result does not contain any valid text, generate an error message containing error codes and error descriptions, and terminate this round of recognition. The method is executed by the robot end-side processor.

2. The method according to claim 1, characterized in that, The hierarchical traversal described in S5 further includes: counting the total number of each element in the SubImages array; if the total number is greater than zero, traversing each element one by one; for each SubImages element, extracting its internal BlockInfo field, obtaining the Text attribute and sorting information of each text block from the BlockDetails array, and obtaining the paragraph text of ParagraphDetails from ParagraphInfo; performing deduplication on each extracted text block; if the contents of two adjacent text blocks are completely identical after normalization by removing spaces and preset punctuation marks, only the first text block is retained, and subsequent duplicate text blocks are discarded; concatenating all deduplicated text blocks into a complete text in ascending order according to the sorting information, which is then executed by the robot's end-side processor.

3. The method according to claim 1, characterized in that, In S6, the first optical character recognition engine is a general optical character recognition engine, and the second recognition engine is a multimodal visual recognition engine. Before triggering the second engine, the original image data is encoded according to the preset image format required by the second engine interface, and sent together with the preset recognition prompt text, which is then executed by the robot end-side processor.

4. The method according to claim 1, characterized in that, Also includes: The number of times the first optical character recognition engine returns a failure or an empty result is counted and accumulated; If the first engine continues to return failures or empty results within the preset consecutive failure threshold, subsequent image recognition requests will be directly routed to the second engine, skipping the calling steps of the first engine, until a certain frame of image is successfully recognized by the second engine and the counter is reset, and then executed by the robot end-side processor.

5. The method according to claim 1, characterized in that, The source labeling described in S4, S5 and S7 is achieved by adding a preset source tag field to the output result. The value of the source tag field includes the direct output of the first engine, the stitching of the first engine sub-graph, and the output of the second engine, which is executed by the robot end-side processor.

6. The method according to claim 1, characterized in that, The validity check described in S2 further includes: checking whether the image data can be completely decoded, checking whether the image resolution is not lower than the preset horizontal resolution threshold and vertical resolution threshold, and checking whether the total number of pixels in the image is not lower than the preset minimum total number of pixels; if any check fails, the recognition process is terminated and a prompt message is returned, which is then executed by the robot end-side processor.

7. A robot task grading image recognition system based on dual-engine cascading and hierarchical subgraph stitching, characterized in that, include: The first module is used to receive image data to be identified, wherein the image data is a captured image of the work to be corrected obtained by the robot through the image acquisition module; The second module is used to verify the validity of the received image data; The third module is used to call the first optical character recognition engine to perform text recognition processing on the image data after the image data validity check is passed; The fourth module is used to parse the return results of the first optical character recognition engine. If the top-level Content field is not empty, it will be output directly. The fifth module is used to perform hierarchical traversal and text concatenation on the SubImages array in the returned result when the top-level Content field is empty; The sixth module is used to automatically trigger the second recognition engine to perform visual text recognition when the first engine fails or fails to extract valid text. The seventh module is used to parse the return results of the second recognition engine; The eighth module is used to generate error messages and terminate recognition when the second engine also fails.

8. The system according to claim 7, characterized in that, The hierarchical traversal in the fifth module includes counting the total number of elements in the SubImages array, traversing each element to extract text blocks from BlockInfo and ParagraphInfo, removing duplicates from adjacent text blocks, and concatenating them according to sorting information; the first engine and the second engine in the sixth module are a general optical character recognition engine and a multimodal visual recognition engine, respectively; the fourth, fifth, and seventh modules add a source tag field to the output to indicate the source of the text.

9. An industrial robot, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the system of claim 7 or 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 6.

Images