OCR text detection post-processing method based on adaptive minimum rotation rectangle segmentation

Abstract

The invention discloses an OCR (Optical Character Recognition) text detection post-processing method based on adaptive minimum rotation rectangle segmentation, and belongs to the technical field of computer vision and OCR. In order to solve the problem of inaccurate fitting of a traditional minimum rotation rectangle (MRR) in curved text detection, the method provides a dynamic optimization mechanism: firstly, obtaining a text pixel-level mask through a segmentation model and extracting a connected component; after the initial MRR is fitted for each component, a priority queue is constructed for iterative processing; based on double-condition judgment of an intersection-to-parallel ratio (IoU) and a size, when the IoU is less than or equal to a dynamic threshold value tau and the short side is greater than or equal to the short side, generating sub-rectangles by bisecting along a perpendicular bisector of the long side; finally, the overlapping rate gt is calculated; performing fusion operation on 70% of adjacent frames; wherein the dynamic threshold value tau is equal to tau + k.C (curvature), and the curvature value C is calculated through the average distance between the contour point and the fitting ellipse.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision and optical character recognition (OCR) technology, and in particular, relates to a text detection post-processing method for an optical character recognition (OCR) system, which is particularly suitable for processing segmentation optimization problems in curved, folded or inclined text scenarios. BACKGROUND

[0002] In existing OCR systems, text detection usually adopts a two-stage method: first, a segmentation model (such as DBNet) is used to detect text regions, and then these regions are sent to a recognition model for character recognition.

[0003] However, in scenarios involving folded or curved paper text, the segmentation model may output connected regions containing multiple text instances. Traditional methods usually use Minimum Rotated Rectangle (MRR) to fit these regions, but due to the curvature of the text, the MRR may cover multiple text lines, causing the recognition model to fail to accurately recognize, thus reducing the overall recognition effect.

[0004] In addition, existing methods often rely on fixed rules (such as threshold or connected component analysis) to extract text regions when dealing with complex text shapes, lacking adaptability to curved text structures, making it difficult to accurately separate text instances. SUMMARY

[0005] 1. Technical problems to be solved: In view of the problems existing in the prior art, the purpose of the present application is to provide an OCR text detection post-processing method based on adaptive minimum rotated rectangle segmentation, which solves the recognition error problem caused by inaccurate MRR fitting in curved text scenarios, dynamically optimizes the detection box through an adaptive bisection splitting mechanism, and improves the robustness and accuracy of the OCR system.

[0006] 2. Technical solutions: To solve the above problems, the present application adopts the following technical solutions.

[0007] An OCR text detection post-processing method based on adaptive minimum rotated rectangle segmentation, comprising the following steps: S1, obtaining the text region pixel-level mask output by the segmentation model, and extracting a connected component set; S2, fitting an initial minimum rotated rectangle (MRR) for each connected component; S3, construct a dynamic processing queue and load initial MRR, the queue adopts a priority scheduling strategy: according to the IoU calculated by the MRR obtained by each connected component and the corresponding text region pixel-level mask (since part of the text region may have non-connectivity or boundary noise, direct pixel set is difficult to form an effective region, so the pixel set needs to be operated first Convex Hull operation, obtain the minimum convex polygon describing the region); arrange the rectangles to be processed in ascending order of IoU value, and process the rectangle box with the lowest IoU value first; S4, iteratively execute the following operations until the queue is empty: Take out the current rectangle box B, calculate its intersection over union IoU (B) with the corresponding connected component; When the following conditions are met simultaneously: IoU (B) ≤ τ; min (width (B), height (B)) ≥ When: (i) generate a unique ID for the connected component; (ii) perform binary cutting to generate sub-rectangles B1 and B2; (iii) add the unique ID and B1, B2 information to the queue; Otherwise, output B as a detection box; S5, since there may be overlapping parts in the newly generated minimum rotating rectangle group, perform scaling operation on the adjacent detection boxes that overlap in space (same ID) within the group; The specific algorithm is as follows: S51, adjust the boundaries of the text boxes in the same group; S52, input: boxes: contains multiple text box coordinates, each text box is composed of several two-dimensional points; group_ids: group numbers corresponding to boxes, used to represent boxes belonging to the same logical area; S53, execute the following steps: S531. Initialize an empty group mapping table group_map; S532. Traverse all text box indexes i: Get the corresponding group_id; Add index i to group_map[group_id]; S533. For each group group in group_map: If the number of boxes contained in the group is less than 2, skip the group; Otherwise, perform the following operations: S533.1 Sort all the boxes in the group by their average x coordinate in ascending order; S533.2 Traverse the sorted adjacent box pair (denoted as box_A and box_B): Get the maximum x coordinate of box_A, denoted as max_x_A; Get the minimum x coordinate of box_B, denoted as min_x_B; Calculate the midpoint of the two values, mid_x = (max_x_A + min_x_B) / 2; Limit the x coordinate of all points in box_A: If the x value of a point > mid_x, replace its x value with mid_x; Limit the x coordinate of all points in box_B: If the x value of a point < mid_x, replace its x value with mid_x; S54. Return the updated boxes Output: the list of boxes after adjusting the boundary position; S6, send the final detection frame into the OCR recognition model.

[0008] Further improvement is that the τ is dynamically configured as: τ = τ0 + k·C; Where τ0 is a basic threshold and τ0 ∈ [0.7, 0.9], k is a curvature coefficient and k ∈ [0.05, 0.2], and C is a curvature value calculated by the average distance between the contour point set and the fitted ellipse; The curvature value C is calculated according to the formula , Where P is the contour point, E is the minimum circumscribed ellipse, and N is the number of points.

[0009] Further improvement is that the bisection cutting of the S4 (ii) step includes: a) Sort the point set obtained by the cutting according to the axis points of the long side; b) Calculate the coordinates of the center point of the point set (geometric center point); c) Then calculate the nearest point to the (geometric center point); d) Divide the point set into left and right parts according to the center point; e) Generate the minimum rotating rectangle corresponding to the left and right parts respectively.

[0010] Further improvement is that the IoU calculation in the S4 step uses: , wherein |B∩C| represents the intersection area size, and |B∪C| represents the union area size; B is the point area of the minimum rotating rectangle, and C is the point area obtained by the original segmentation, and the intersection is obtained by cv2.intersectConvexConvex, and the area of the two is calculated by cv2.contourArea.

[0011] Further improvement lies in that the dynamic processing queue of S3 adopts a priority scheduling strategy. The rectangles to be processed are arranged in ascending order of the IoU value, and the rectangle frame with the lowest IoU value is preferentially processed.

[0012] An OCR text detection system comprises: A segmentation network module generates a text pixel-level mask. An adaptive post-processing engine comprises a connected domain analysis unit, a dynamic decision unit, and a frame fusion unit. An identification interface module.

[0013] Further improvement lies in that the dynamic decision unit comprises a hardware-accelerated geometric calculation module for accelerating IoU calculation, a vector operation unit (VPU) configured to calculate a cutting plane, and a curvature detection coprocessor for calculating a contour curvature value C.

[0014] A computer-readable storage medium, wherein the program further implements real-time visualization of the cutting process and an IoU change curve, and feedback of automatic optimization of τ and based on the identification result of S6. .

[0015] An industrial OCR identification terminal comprises an image collector, a processor, and a memory, wherein the memory stores the program according to any one of claims 8-9. The processor is configured to perform S1-S6 in a pipeline and support parallel processing between steps. The curvature detection coprocessor according to claim 7 is integrated.

[0016] 3. Beneficial effects Compared with the prior art, the technical scheme provided by the present application has the following beneficial effects: (1) Significantly improves the recognition accuracy of curved text Through the adaptive bisection cutting mechanism, the overfitting problem of the traditional minimum rotating rectangle (MRR) method in the curved text scene is effectively solved, and multiple text instances are accurately separated from the connected region. In the paper folding / curved text scene, the OCR recognition accuracy is improved by 32% (from 68% of the traditional method to 90%+).

[0017] (2) Enhance the ability to adapt to complex scenarios Dynamic threshold mechanism: adaptive threshold τ = τ0 + k · C based on contour curvature C, so that the system automatically adapts: Low curvature text (C < 1) uses a higher threshold (τ ≈ 0.85) to reduce invalid segmentation; High curvature text (C > 3) uses a lower threshold (τ ≈ 0.75) to increase the granularity of segmentation; Multi-morphology support: successfully handle complex morphologies such as inclined (> 45°), overlapping (IoU > 30%), broken text, etc.

[0018] (3) Optimize computing efficiency and resource utilization Optimization of algorithm complexity: based on O(n) connected component processing, only increase O(log n) binary segmentation complexity; Hardware acceleration innovation: Vector geometry engine based on SIMD instructions, support parallel processing of polygon intersection area calculation, realize 8 times efficiency improvement of IoU geometry calculation.

[0019] VPU unit accelerates cutting plane calculation, single cutting time <0.1ms; Priority queue scheduling: preferentially process low IoU regions, reduce the total iteration number by 30%.

[0020] (4) Simplify the OCR processing flow End-to-end integration: replace additional morphological processing (such as dilation and erosion) and manual threshold adjustment in traditional solutions; Reduce manual intervention: automatically optimize parameters (τ0, k, ) through recognition result feedback; Memory usage optimization: use AABB to limit the minimum processing area, avoid polygon intersection calculation in the full image range, reduce memory usage by about 45%.

[0021] It should be noted that the structures not introduced in the present application do not involve the design points and improvement direction of the present application, and are the same as the prior art or can be realized by using the prior art, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is the overall flowchart image of the present application; Figure 2 is the input original image; Figure 3 is the binary image obtained after the segmentation model is predicted; Figure 4 is the image output by the segmentation detection network after the non-optimized post-processing; Figure 5 is the final result obtained by sending the text box obtained by post-processing into the character recognition model without optimization; Figure 6 is the image output by the post-processing of the segmentation detection network optimized by the present application; Figure 7 is the final result obtained by sending the text box obtained by post-processing into the character recognition model after optimization by the present application. DETAILED DESCRIPTION

[0023] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the accompanying drawings, in which several embodiments of the present application are given. However, the present application can be realized in many different forms and is not limited to the embodiments described herein. On the contrary, these embodiments are provided to make the disclosure of the present application more thorough and comprehensive.

[0024] 1. System overall flow As shown in the overall flowchart, Figure 1 , the input original image example (such as a medical document), Figure 2 , the segmentation model output mask example, the present method comprises the following core steps: Figure 3 Input image preprocessing: denoising, normalization and resolution adjustment are performed on the original image to adapt to the input requirements of the segmentation model. Figure 2

[0025] Segmentation model prediction: using a segmentation network such as DBNet to process the image, outputting a binary mask M(x, y) Figure 3 : ; Connected region extraction: extracting a connected component set from the mask , each component corresponds to a text region.

[0026] Adaptive minimum rotating rectangle (Adaptive MRR) processing: performing recursive segmentation on each .

[0027] Detection box scaling: scaling processing is performed on the rectangular boxes overlapped in the group.

[0028] OCR recognition: sending the final detection box Figure 6 into the recognition model to output the result Figure 7 .

[0029] 2. Adaptive minimum rotating rectangle segmentation flow Input: connected component C, IoU threshold τ, minimum size ; ​Output: List of rectangles Blocks that meet the conditions.

[0030] Pseudo-code implementation: def AdaptiveSplit(C): R = MRR(C) # Fit initial Minimum Rotated Rectangle group = ID(C) Q = PriorityQueue(sort_by='IoU_asc') # Dynamic processing queue (ascending IoU) Q.push((R, group)) Blocks = [] while not Q.empty(): B, group = Q.pop() IoU_val = calculate_IoU(B, C) # Calculate the Intersection over Union of the current rectangle and C if IoU_val >= tau or min(B.width, B.height) < S_min: Blocks.append(B) # Termination condition: meet threshold or size too small else: B1, B2 = split_along_long_edge(B) # Split along the long edge Q.push((B1, group)) Q.push((B2, group)) return Blocks 2.1. IoU calculation optimization using parallel computing acceleration: For any rectangular frame BB and the original connected region CC, the Intersection over Union (IoU) is defined as: ; : represents the area of the polygon formed by the outline B; : represents the area of the polygon formed by the outline C; : represents the intersection area of the two polygons, i.e. their geometric overlapping part; : represents the union area of the two polygons.

[0031] : To prevent the introduction of a tiny positive number (such as 10 -6 ) for the denominator is zero, you can set according to the actual system precision.

[0032] Axis-Aligned Bounding Box (AABB) constraint: only traverse the minimum processing area of B; SIMD instruction parallel: through the vectorization of geometric operation framework, support multiple polygon concurrent intersection calculation, improve the calculation efficiency.

[0033] 2.2. In this scheme, for the cutting step along the long side: a) The point set obtained by the cutting is sorted according to the axis point of the long side; b) Calculate the coordinates of the point center point (geometric center point) of the point set; c) Then calculate the nearest point from (geometric center point); d) According to the center point, the point set is divided into left and right parts; e) Generate the minimum rotating rectangle corresponding to the left and right parts respectively.

[0034] 2.3. The threshold value τ in this scheme is adaptively adjusted according to the text curvature: τ = τ0 + k · C; τ0 ∈ [0.7, 0.9], k ∈ [0.05, 0.2] The curvature calculation formula is: , For the contour point, E is the minimum circumscribed ellipse, and N is the number of points; 2.4. Recursion and termination condition Termination condition 1: IoU (B, C) ≥ τ, which means that the rectangle B is accurate enough and does not need to be further divided.

[0035] Termination condition 2: , to avoid excessive division into invalid small blocks.

[0036] 2.5. Algorithm complexity analysis a) Connected component extraction It is still a pixel-based operation, and the time complexity is O (n), where n is the total number of image pixels.

[0037] b) Convex hull calculation (for each connected component) For a connected component with k boundary points, the convex hull calculation complexity is O (klog k).

[0038] c) Minimum rotating rectangle (MRR) fitting Based on the convex hull point set, the time complexity is O (k).

[0039] d) Bisecting cutting process Let the maximum cutting depth be d, and the IoU of the current rectangle and the corresponding region convex hull needs to be calculated each time the cutting is performed.

[0040] Each IoU calculation is based on polygon intersection, with a complexity of about O(k + m), where m is the number of MRR vertices (fixed at 4).

[0041] The maximum number of cuts is 2 d - 1 (number of binary tree nodes), but in practice, it is much smaller due to early stopping or conditional restrictions.

[0042] Overall post-processing complexity For each connected component, convex hull + MRR + multiple IoU calculations, the overall complexity is: , which can meet the real-time or near real-time processing requirements 2.6. Key points of specific implementation (1) Selection of segmentation model: DBNet based on ResNet- or HRNet- backbone can be used to output high-precision text segmentation mask.

[0043] (2) Connected component and contour extraction: OpenCV `findContours` can be used to extract the connected component contour of the text region, and OpenCV `convexHull` can be used to obtain the corresponding minimum convex hull contour.

[0044] (3) MRR fitting: Use OpenCV `minAreaRect` interface to obtain the parameters of the rotated rectangle.

[0045] (4) IoU calculation: Use OpenCV `intersectConvexConvex` to get the intersection area of the minimum rotated rectangle and its corresponding convex hull contour, and use OpenCV `contourArea` to get the area of each. Finally, calculate the IoU value.

[0046] (5) Parallelization: multiple rectangles in different connected components or queues are processed in parallel to improve throughput.

[0047] (6) Parameter tuning: Threshold τ can be searched on the validation set; The minimum size is determined according to the input requirements of the downstream recognition model 3. Detection box fusion Post-processing is performed on adjacent detection boxes: Scaling condition: there are more than 2 rectangular boxes in the group and there is an overlapping part.

[0048] Operation: Zoom-in bounding box; Purpose: Avoid the case of calculating the minimum rotating rectangle of adjacent text regions.

[0049] 4. Hardware implementation System architecture includes: Segmentation network module: generate pixel-level mask; Adaptive post-processing engine: Connected component analysis unit: extract ; Dynamic decision unit: integrate hardware accelerator: VPU (vector processor): real-time calculation of cutting plane; Curvature coprocessor: parallel calculation of C value; SIMD geometry acceleration unit: accelerate IoU calculation; Frame fusion unit: perform overlap detection; OCR recognition interface: connect to recognition model.

[0050] 5. Parameter tuning and visualization Parameter optimization: Determine the optimal value of τ based on the validation set grid search; According to the feedback of OCR recognition results, automatically adjust parameters (such as reduce τ0, k, when the recognition error rate increases).

[0051] Real-time monitoring: Dynamically display in GUI: (a) Rectangle box evolution in segmentation process; (b) IoU value change curve of each rectangle; (c) Heat map of curvature C.

[0052] Comparison experiment data (medical document dataset)

[0053] This scheme is verified in medical documents ( Figure 5 → Figure 7 ), industrial labels, ancient document scenes, etc. It solves the problem of recognition confusion caused by curved text and provides key support for the landing of OCR technology in practical applications.

[0054] The above-described embodiments only express some implementation manners of the present application, which are described in a more specific and detailed manner, but should not be understood as a limitation on the patent scope of the present application; it should be noted that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, which all belong to the protection scope of the present application; therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A post-processing method for OCR text detection based on adaptive minimum rotated rectangle segmentation, characterized in that: The following steps are involved: S1. Obtain the pixel-level mask of the text area output by the segmentation model and extract the connected component set; S2, fitting an initial minimum rotation rectangle (MRR) for each connected component; S3. Build a dynamic processing queue and load the initial MRR. The queue adopts a priority scheduling strategy: calculate the IoU based on the MRR obtained for each connected component and the pixel-level mask of the corresponding text area; sort the rectangles to be processed in ascending order of IoU value, and give priority to the rectangles with the lowest IoU value; S4. Iterate and perform the following operations until the queue is empty: Take out the current rectangular box B and calculate its intersection over union (IoU) (B) with the corresponding connected component. When both: IoU(B)≤τ; min(width(B),height(B))≥ hour: (i) Generate a unique ID for the connected component; (ii) Perform a binary cut to generate sub-rectangles B1 and B2; (iii) add the unique ID and B1 and B2 information to the queue; Otherwise, output B as the detection box; S5. Since there may be overlapping parts within the newly generated minimum rotated rectangle group, a scaling operation is performed on the spatially overlapping adjacent detection boxes (with the same ID) within the group. The specific algorithm is as follows: S51. Adjust the boundaries of text boxes in the same group; S52. Input: boxes: contains multiple text box coordinates, each text box is composed of several two-dimensional points; group_ids: group numbers corresponding to boxes, used to indicate boxes belonging to the same logical area; S53, the execution steps are as follows: S531. Initialize an empty group mapping table group_map; S532. Traverse all text box indexes i: Get the corresponding group_id; Add index i to group_map[group_id]; S533. For each group in group_map: If the number of boxes in the group is less than 2, skip the group; Otherwise, do the following: S533.1 sort all box indices in the group by their average x-coordinate from smallest to largest; S533.2 Traverse the sorted adjacent box pairs (denoted as box_A and box_B): Get the maximum x coordinate of box_A, denoted as max_x_A; Get the minimum x coordinate of box_B, denoted as min_x_B; Calculate the midpoint of these two values ​​mid_x = (max_x_A+min_x_B) / 2; Limit the x coordinates of all points in box_A: If the x value of a point is > mid_x, then its x value is replaced by mid_x; Limit the x coordinates of all points in box_B: If the x value of a point is less than mid_x, its x value is replaced by mid_x; S54. Return updated boxes Output: List of boxes after adjusting the border position; S6. Send the final detection frame to the OCR recognition model.

2. The OCR text detection post-processing method based on adaptive minimum rotated rectangle segmentation according to claim 1, characterized in that: The dynamic configuration of τ is: τ=τ0+k·C; The dynamic threshold τ is used for the IoU(B)≤τ judgment condition in step S4, where τ0 is the basic threshold and τ0∈[0.7,0.9], k is the curvature coefficient and k∈[0.05,0.2], and C is the curvature value calculated by the average distance between the contour point set and the fitted ellipse; The curvature value C is calculated according to the formula , is the contour point, E is the minimum circumscribed ellipse, and N is the number of points.

3. The OCR text detection post-processing method based on adaptive minimum rotated rectangle segmentation according to claim 1, characterized in that: The bisection in step S4(ii) comprises: a) Sort the point set obtained by the segmentation according to the direction axis points of the long side; b) Calculate the coordinates of the center point of the point set (geometric center point); c) Then calculate the point closest to (the geometric center point); d) Divide the point set into two parts, left and right, based on the center point; e) Generate the minimum rotated rectangles corresponding to the left and right parts respectively.

4. The OCR text detection post-processing method based on adaptive minimum rotated rectangle segmentation according to claim 1, characterized in that: The IoU calculation in step S4 is: , where |B∩C| represents the size of the intersection area, and |B∪C| represents the size of the union area; B is the point area of ​​the minimum rotated rectangle and its area is obtained by the following algorithm. C is the point area obtained by the original segmentation and its area is obtained by the following algorithm. The intersection is obtained by cv2.intersectConvexConvex; cv2.contourArea calculates the area of ​​the two.

5. The OCR text detection post-processing method based on adaptive minimum rotated rectangle segmentation according to claim 1, characterized in that: The S3 dynamic processing queue adopts a priority scheduling strategy: Arrange the rectangles to be processed in ascending order of IoU value; Prioritize the rectangle with the lowest IoU value.

6. An OCR text detection system, characterized by: include: Segmentation network module to generate pixel-level text masks; Adaptive post-processing engine, including: connected domain analysis unit; Dynamic decision-making unit; Frame fusion unit; Identify the interface module.

7. An OCR text detection system according to claim 6, characterized in that: The dynamic decision-making unit includes: Hardware-accelerated geometric calculation module, used to accelerate the calculation of the intersection area between convex polygons and the minimum rotated rectangle, and accelerate the IoU calculation; a vector processing unit (VPU) configured to calculate cutting planes; Curvature detection coprocessor, calculates the contour curvature value C.

8. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, steps S1 to S6 of claim 1 are implemented.

9. The computer-readable storage medium according to claim 8, wherein: The program also implements: Real-time visualization of the cutting process and IoU change curve; Automatically optimize τ and .

10. An industrial OCR recognition terminal, comprising an image collector, a processor and a memory, characterized in that: The memory stores the program according to any one of claims 8 to 9; The processor is configured to execute steps S1-S6 in a pipeline and support parallel processing between steps; Integrate the curvature detection coprocessor as described in claim 7.

Images