Super-large format document intelligent identification method and system based on block parallelism

Abstract

The invention provides an intelligent recognition method and system for an ultra-large format document based on block parallel. The method comprises the following steps: acquiring image data of a to-be-recognized document; a self-adaptive blocking strategy is adopted, dynamic blocking is conducted according to an available memory of a system or image features, a certain overlapping area is kept between every two adjacent blocks, and then blocking tasks are converted into an asynchronous task queue; using a thread pool to process each block in parallel, or adopting a distributed computing architecture to distribute tasks to a plurality of computing nodes for execution; detecting a text region and a table structure, and performing text recognition on the detected text region through a text recognition model; processing results of all the blocks are collected and combined according to the recorded position information; and performing confidence scoring according to an identification result output by the text identification model to optimize a text region, and performing visualization processing and layout analysis on the optimized text to generate an output result in a standard format.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method and system for intelligent recognition of ultra-large format documents based on block parallelism. Background Art

[0002] Documents play a vital role in various industries as key information carriers. Regardless of government affairs, finance, manufacturing and other industries, various types of documents play an indispensable role as digital materials in business processes. Discovering and extracting valuable content from original documents is one of the most core technologies. This involves comprehensive analysis of document formats and in-depth understanding of document content. Intelligent document understanding can accurately extract core business information by comprehensively analyzing document formats and deeply understanding document content. In practical applications, automatic processing and understanding of document images has a wide range of uses, including but not limited to document image classification, document image information extraction, and document visual question and answer.

[0003] At present, document intelligence has become an indispensable part of information-based office. Manual operations are inefficient in image analysis, text processing, and data recognition, and are difficult to meet the office needs of the big data era. Therefore, OCR technology has become an indispensable technology in the fields of information collection, processing, and transmission due to its advantages such as fast recognition speed, high recognition rate, and the ability to directly process images in various formats.

[0004] With the deepening of digital transformation, the demand for intelligent processing of ultra-large format documents is growing. Application scenarios include intelligent recognition and analysis of ultra-large format documents such as engineering drawings, architectural design drawings, and medical images. Current document recognition methods generally face problems such as memory overflow and low processing efficiency when processing such ultra-large format documents. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method for intelligent recognition of ultra-large format documents based on block parallelism, which breaks through the memory limitation and performance bottleneck of traditional document recognition systems when processing ultra-large format documents, and realizes efficient recognition of documents of any size.

[0006] In order to achieve the purpose of the present invention, the technical solution adopted by the present invention is: A method for intelligent recognition of ultra-large format documents based on block parallelism includes the following steps: A. Image Data Preprocessing Obtain image data of the document to be recognized and adopt image enhancement strategies to improve the quality of the original image; B. Image Block Processing Adopt an adaptive block strategy to dynamically block according to the available system memory or image features, maintain a certain overlap area between adjacent blocks to ensure the integrity of the text, and then convert the block task into an asynchronous task queue; C. Parallel text detection and recognition Use thread pools to process each block in parallel, or use a distributed computing architecture to distribute tasks to multiple computing nodes for execution; detect text areas and table structures, record their relative position information, and perform text recognition on the detected text areas through a text recognition model; D. Result merging and structural reconstruction Collect the processing results of each block and merge them according to the recorded location information. For texts spanning multiple blocks, reorganize the text by analyzing the contents of the overlapping areas. E. Post-processing optimization and output The confidence score is given according to the recognition results output by the text recognition model to optimize the text area, and the optimized text is visualized and analyzed to generate output results in a standard format.

[0007] In step A of the present invention, after format verification of the input data, the data is converted into a numpy array, and then the numpy array is decoded into an image format.

[0008] In step B of the present invention, if the width and height of the image exceed the preset values, the image is grid-blocked, and an overlapping area of ​​a fixed size is set between adjacent blocks, and the boundary coordinates and index information of each block are recorded for subsequent image processing; The calculation formula of the adaptive block strategy is: k=[(Mm) / (mo)]+1, v=[(Nn) / (no)]+1, k is the number of blocks in the vertical direction, v is the number of blocks in the horizontal direction, and k and v are rounded down; M and N represent the length and width of the image, respectively, and m and n represent the length and width of the block, respectively; o represents the side length of the overlapping area, which is a preset value; The values ​​of m and n are dynamically assigned according to the available memory of the system or the image features.

[0009] In step C of the present invention, the detected text areas are processed in parallel, the tilt angle is calculated for each text area, image rotation correction is performed according to the tilt angle, and the corrected image is used for text recognition.

[0010] The table structure detection in step C of the present invention comprises the following steps: C1. Use the erosion and dilation method to extract the table frame structure of the table area; C2. Detect the table outline. According to the table outline, each outline corresponds to a potential cell. The outline coordinates are sorted from small to large, and the column center point is calculated to adjust the horizontal alignment of the cell. C3, remove the outlines that are too small or too large, and filter out the non-cell areas; C4, check whether there are enough white pixels on the four edges of each cell, and only keep the cells that pass the edge check; C5. Return the final confirmed cell coordinate list and generate a table structure.

[0011] The step D of the present invention comprises: D1. Use R-tree spatial index to build a fast query structure for table cells; D2, traverse all detected text blocks, convert their coordinates from the local coordinate system of the blocks to the global coordinate system of the original image, and merge overlapping text blocks in the vertical direction; D3. Use the OpenCV library to check the legitimacy of the merged text blocks, fix possible geometric problems, and calculate the coordinates of the center point of each text block; legitimacy means that the merged cell is a polygon rather than multiple connected polygons.

[0012] D4. Use the spatial index to query the table cell where the center point of each text block is located, establish the correspondence between the text block and the cell, merge adjacent text blocks in the horizontal direction, and assign the text block coordinate array as a merge mark; D5. According to the merge markers, the merged text blocks are collected to form the final text area list. The OpenCV library is used again to ensure that each text area polygon is legal and retains its association information with the table cell. D6. Output the merged text block coordinate array and the corresponding table cell index, which are used to fill the text recognition results back into the corresponding position of the cell.

[0013] Preferably, the overlapping text blocks are merged in the vertical direction: for the adjacent blocks in the vertical direction, the text blocks in their overlapping areas are checked to determine whether the text blocks in the upper and lower blocks overlap or are close to each other in the overlapping area, and if so, these text blocks are merged; Merge adjacent text blocks horizontally, sort them from left to right according to their horizontal positions, traverse the sorted text block list, compare adjacent text blocks, use geometric union operations on eligible text blocks to merge them into a new polygon, and update the merge mark to avoid repeated processing.

[0014] Further preferably, the condition is met, which means that the following conditions are met at the same time: the two text blocks belong to the same table cell; they are adjacent in the horizontal direction and have overlapping areas in the vertical direction; and the geometric intersection of the two text blocks meets a certain area or ratio threshold.

[0015] The step E of the present invention comprises: E1. Use the average probability of each character output by the text recognition model as the confidence score, filter the results with confidence lower than the threshold, remove the blank text area, and mark the area with confidence lower than the threshold but higher than the lower limit as the area to be manually verified; E2. Visualization processing: Create a mask image of the same size as the original image, use polygon filling to mark the text area, enhance the brightness of the text area, and generate the processed visualization result; E3. Layout analysis to generate output results in standard format.

[0016] A system for implementing the ultra-large format document intelligent recognition method of the present invention comprises: A data receiving module is used to obtain the document image to be recognized and pre-process the image data; The image segmentation module uses an adaptive block strategy to dynamically segment images based on the available system memory or image features; The task scheduling module converts the block tasks into asynchronous task queues and prioritizes the tasks according to the complexity and importance of the blocks; The text recognition module pre-configures multiple candidate text recognition models, and each candidate text recognition model corresponds to a different recognition scenario. The block to be recognized is input into the corresponding candidate text recognition model to obtain the recognition result; The result merging module is used to collect the recognition results of each block and intelligently merge them according to the recorded location information; The post-processing optimization module is used to optimize the text area, perform visualization and layout analysis on the optimized text, and generate output results in a standard format.

[0017] The beneficial effects of the present invention are: 1. The present invention has formulated an adaptive blocking strategy, which performs dynamic blocking according to the system's available memory or image features, realizes the processing capability of documents of any size, optimizes the efficiency of memory resource utilization, stabilizes system resource occupancy, and does not cause memory overflow even when processing very large documents, significantly improving system stability. The processing speed is increased by parallel computing, and multi-task concurrent execution is achieved, processing efficiency is improved, and processing delay is reduced. Compared with the traditional serial processing method, the processing speed is increased by 3-5 times. The combination of intelligent blocking and parallel processing of the present invention breaks through the memory limitations of traditional OCR systems, can process large-format documents of any size, and the processing efficiency increases linearly with hardware resources.

[0018] 2. When existing OCR methods perform text recognition on very large images, the system processing usually fails due to insufficient memory or video memory. The present invention uses adaptive blocking to divide the large image, recognizes each sub-image separately, and then reconstructs the sub-image information on the large image, reducing the pressure on the memory and video memory, and ensuring that the system can process very large images normally and stably.

[0019] 3. The present invention divides a large image into multiple sub-images, and each sub-image does not undergo scaling or other adjustments during text detection, ensuring that image information is not lost and that text detection and recognition effects are completely consistent. Post-processing optimization removes duplicate text and text with low recognition confidence, ensuring the accuracy of the recognition results and high recognition quality under high-speed parallel computing. The present invention's adaptive block strategy and overlapping area processing mechanism solve the problem of text cross-block breaks, ensuring the integrity of the document, and improving recognition accuracy by more than 30% compared to traditional methods.

[0020] 4. The present invention performs OCR recognition on each block separately, which can ensure the integrity of the document and will not lose the information in the picture. Then the results are processed using the position information of text detection, the information of table line detection and layout analysis technology to complete the reconstruction of the entire layout, restore the simple paragraph structure and table structure, and further maintain the integrity and coherence of the document.

[0021] 5. The present invention records the position information of each block when dividing the blocks, obtains the information of each text line when detecting the text, and obtains the information of each cell by table line detection. The results of each sub-image are merged using this information, and then a simple layout analysis method is used to accurately reconstruct the logical structure of the document.

[0022] 6. Currently, all files can be transmitted in the form of binary streams and converted into numpy arrays using the numpy tool library. Binary stream data and data in npy format can be accepted, and these data are converted into a unified multi-dimensional array image format. After the data format is unified, it is suitable for the recognition method of the present invention, thereby realizing the processing of various document types.

[0023] 7. The table line detection method of the present invention, after extracting the table frame line and outline, also filters out non-cell areas and cells that fail edge inspection, thereby ensuring accurate extraction of the table and having the ability to process different layouts.

[0024] 8. The intelligent recognition method of the present invention can be used in conjunction with OCR algorithms of all languages, thereby supporting multi-language documents. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The present invention is a flow chart of the method for intelligently identifying ultra-large-format documents based on block parallelism.

[0026] Figure 2 Flow chart of the results merging and structural reconstruction of the present invention.

[0027] Figure 3 This is a flow chart of table structure detection of the present invention. DETAILED DESCRIPTION

[0028] In order to more clearly and in detail illustrate the technical solution of the present invention, the present invention is further described below through relevant embodiments. The following embodiments are only for specific explanation of the implementation method of the present invention, and do not limit the protection scope of the present invention.

[0029] Example 1

[0030] A method for intelligent recognition of ultra-large format documents based on block parallelism includes the following steps: A. Image Data Preprocessing Obtain image data of the document to be recognized and adopt image enhancement strategies to improve the quality of the original image; B. Image Block Processing Adopt an adaptive block strategy to dynamically block according to the available system memory or image features, maintain a certain overlap area between adjacent blocks to ensure the integrity of the text, and then convert the block task into an asynchronous task queue; C. Parallel text detection and recognition Use thread pools to process each block in parallel, or use a distributed computing architecture to distribute tasks to multiple computing nodes for execution; detect text areas and table structures, record their relative position information, and perform text recognition on the detected text areas through a text recognition model; D. Result merging and structural reconstruction Collect the processing results of each block and merge them according to the recorded location information. For texts spanning multiple blocks, reorganize the text by analyzing the contents of the overlapping areas. E. Post-processing optimization and output The confidence score is given according to the recognition results output by the text recognition model to optimize the text area, and the optimized text is visualized and analyzed to generate output results in a standard format.

[0031] Example 2

[0032] This embodiment is based on embodiment 1: In step A, after format verification of the input data, the data is converted into a numpy array, and then the numpy array is decoded into an image format.

[0033] The system receives input image data and supports two formats: binary stream and numpy array. It can also be extended to support multiple input methods such as base64 encoding and image URL links. The system will verify the format of the input data to ensure the integrity and validity of the data, and uniformly convert it into a standard processing format.

[0034] For binary stream formats, the system uses BytesIO for memory optimization processing; for numpy array formats, the system directly performs subsequent processing. Data is written to the buffer in a streaming manner to avoid loading the entire file at one time. Use the numpy library to convert binary data into a numpy array, which is usually a one-dimensional array with each element representing a byte value. The one-dimensional array is reorganized into a multidimensional array based on the format of the image data (such as height, width, and number of channels). Finally, the reshaped multidimensional array is decoded into an image format. The decoding process converts the numerical array into visible image data based on image encoding standards (such as JPEG, PNG, etc.).

[0035] At the same time, different image enhancement strategies can be used in the preprocessing process, such as adaptive histogram equalization, image sharpening and other technologies to improve the quality of the original image. In addition, image compression technology can also be introduced to dynamically adjust the compression rate according to the system resource status to optimize memory usage while ensuring recognition quality.

[0036] Example 3

[0037] This embodiment is based on embodiment 1: In step B, if the width and height of the image exceed the preset values, the image is divided into grid blocks, and a fixed-size overlapping area is set between adjacent blocks, and the boundary coordinates and index information of each block are recorded for subsequent image processing; The calculation formula of the adaptive block strategy is: k=[(Mm) / (mo)]+1, v=[(Nn) / (no)]+1, k is the number of blocks in the vertical direction, v is the number of blocks in the horizontal direction, and k and v are rounded down; M and N represent the length and width of the image, respectively, and m and n represent the length and width of the block, respectively; o represents the side length of the overlapping area, which is a preset value; The values ​​of m and n are dynamically assigned according to the available memory of the system or the image features.

[0038] Taking an image with three channels and a bit depth of 8 bits as an example, assuming that the system available memory is Mem, then the memory occupied by each block is m*n*3 bytes. Just set m*n<=Mem / 3.

[0039] The specific dynamic assignment method is as follows: 1. Initialize variables: Define two variables m and n, both with initial values ​​of 0.

[0040] 2. Nested loops: The outer loop iterates over a list [1024, 512, 256], assigning the current value to the variable mt each time.

[0041] The inner loop also traverses the list [1024, 512, 256], assigning the current value to the variable nt each time.

[0042] 3. Conditional judgment: In the inner loop, check whether mt * nt is less than Mem / 3.

[0043] If the condition is met, mt is assigned to m, nt is assigned to n, and the inner loop is immediately jumped out.

[0044] 4. Verification of the validity of m and n: If m>0 and n>0, it means m and n are valid. If m=0 or n=0, it means that the system has insufficient available memory and cannot support the system to continue processing, so the task is terminated.

[0045] The maximum value of m,n is 1024, and the minimum value is 256, that is, the maximum memory occupied by each block is 3Mb, and the minimum memory occupied is 192Kb. This memory usage can support multiple blocks in parallel processing for most scenarios. The list data used for dynamic assignment can be set as needed.

[0046] The present invention can also perform dynamic segmentation based on image edge information, text density, texture and other features.

[0047] Here, we take the texture features of the image as an example and use the LBP (local binary pattern) histogram to describe the local texture features of the image. The image complexity is determined based on the LBP value, and m and n are dynamically assigned values.

[0048] The specific dynamic assignment method is as follows: 1. Basic principles of LBP histogram LBP calculation: For each pixel of the image, compare its grayscale value with that of its surrounding pixels.

[0049] If the gray value of the neighborhood pixel is greater than that of the central pixel, it is marked as 1, otherwise it is marked as 0.

[0050] Arrange these binary values ​​in sequence to form a binary number, and convert it into decimal to obtain the LBP value of the pixel.

[0051] 2. Relationship between LBP and texture complexity A uniform pattern is a pattern in which the number of binary transitions in the LBP value does not exceed 2 (for example, 00000000, 11111111, 00011100, etc.). A uniform pattern usually corresponds to a simple texture (such as a smooth area or a regular texture).

[0052] A non-uniform pattern is one in which the number of binary transitions in the LBP value exceeds 2. Non-uniform patterns usually correspond to complex textures (such as detailed areas or irregular textures).

[0053] 3. The assignment method of m and n is as follows: ① Initialize variables: Define two variables m and n, both with initial values ​​of 0.

[0054] ②Calculate the LBP value of each pixel in the original image ③Conditional judgment and assignment: The number of LBP values ​​that meet the definition of uniform mode is LBP_n, the total number of pixels is LBP_N, and the proportion of uniform mode in all pixel LBP values ​​is calculated as p = LBP_n / LBP_N If p>=0.6, it means the image content is relatively simple, and the block size is set to m=n=1024; If 0.3<= p<0.6, it means the image content is slightly complex, and the block size is set to m=n=512; If p<0.3, it means that the image content is complex and the block size is set to m=n=256.

[0055] The conditional values ​​used to determine the p value and the data values ​​used to assign values ​​to m and n can be set as needed.

[0056] The present invention can also use multi-level priority queues for task scheduling, and prioritize tasks according to the complexity and importance of the blocks. In addition, GPU parallel computing can be introduced to accelerate the image processing and text detection process through technologies such as CUDA.

[0057] This adaptive block strategy aims to improve the processing efficiency of large-size images and solve the problem of cross-block text recognition. The system first obtains the width and height of the image. For images that exceed the preset maximum length (the default value is 1024 pixels), the system will determine the appropriate block size based on the maximum length. The image is divided into several small grid-like blocks in the horizontal and vertical directions according to the calculated number of blocks. The width and height of each block do not exceed the set maximum length, ensuring that the amount of data in each block is within a controllable range. When blocking, the system will record the boundary information of each block, including the coordinates of the upper left corner and the lower right corner, for subsequent processing.

[0058] Overlapping area processing: In order to solve the recognition difficulties caused by text spanning multiple blocks, the system sets a fixed-size overlapping area between adjacent blocks (the default value is 128 pixels). Calculate the boundary of each block: For each block, the system calculates the coordinates of its upper left corner and lower right corner: Horizontal: The position of the left border is determined by subtracting the size of the overlapped area from the tile width, multiplied by the horizontal tile index. The position of the right border is determined by adding the tile width to the left border. If the right border exceeds the total width of the image, it is clamped to the image width.

[0059] Vertical: The position of the upper boundary is determined by subtracting the size of the overlapped area from the tile height, multiplied by the vertical tile index. The position of the lower boundary is determined by adding the tile height to the upper boundary. If the lower boundary exceeds the total height of the image, it is clamped to the image height.

[0060] These overlapping areas ensure that the text across blocks can be fully captured and recognized. At the same time, the system records the relative position relationship and index information of each block, providing necessary reference for subsequent result merging and reconstruction.

[0061] Example 4

[0062] This embodiment is based on embodiment 1: In the step C, a thread pool is used to process the detected text areas in parallel, a tilt angle is calculated for each text area, image rotation correction is performed according to the tilt angle, and the corrected image is used for text recognition.

[0063] You can also use a distributed computing architecture to assign tasks to multiple computing nodes for execution.

[0064] The size of the thread pool can be adjusted according to the number of CPU cores or the amount of tasks. Submit the processing task of each text area to the thread pool and perform the following operations in parallel: Calculate the tilt angle: Calculate the minimum bounding rectangle of the text area. Extract the center point, size (width and height), and tilt angle from the rectangle.

[0065] Adjust the rectangle direction: If the absolute value of the tilt angle is greater than 45 degrees, adjust the angle so that the long side of the rectangle is as horizontal as possible. Calculate the number of rotations based on the adjusted angle and update the width and height of the rectangle.

[0066] Image rotation correction: According to the adjusted angle, the text area is rotated and corrected. Affine transformation is used to calculate the rotation matrix and rotate the image.

[0067] Text detection: To detect text line areas, you can combine multiple detection algorithms, such as text detection based on segmentation algorithms such as the DBNet++ algorithm, or EAST, DB, etc., to improve detection accuracy through model integration.

[0068] Text recognition: To recognize single-line text content, the most suitable recognition model can be dynamically selected according to different languages ​​and scenarios, such as using a special recognition model for handwriting. The recognition model can use the Tesseract text recognition model or the CRNN model for recognition.

[0069] Example 5

[0070] This embodiment is based on embodiment 1: The table structure detection in step C is used to detect the position of table cells in the image and return the coordinate information of each cell. It includes the following steps: C1. Use the erosion and dilation method to extract the table frame structure of the table area; C2. Detect the table outline. According to the table outline, each outline corresponds to a potential cell. The outline coordinates are sorted from small to large, and the column center point is calculated to adjust the horizontal alignment of the cell. C3, remove the outlines that are too small or too large, and filter out the non-cell areas; C4, check whether there are enough white pixels on the four edges of each cell, and only keep the cells that pass the edge check; C5. Return the final confirmed cell coordinate list and generate a table structure.

[0071] Specific implementation steps

[0072] C11. Image preprocessing: Grayscale: Convert a color image to a grayscale image to simplify the data structure.

[0073] Binarization: Perform threshold processing on the grayscale image to generate a black and white binary image and highlight the lines of the table.

[0074] Image inversion: Invert the binary image so that the table lines are white and the background is black, which is convenient for line detection.

[0075] C12, table line detection: Define Structural Elements: Create vertical and horizontal rectangular structural elements whose dimensions are determined by the preset kernel lengths.

[0076] C13. Extract vertical lines: Erosion: Use vertical structure elements to erode the inverted binary image to remove non-vertical lines and noise.

[0077] Dilation: Dilate the erosion result to strengthen the continuity of vertical lines.

[0078] C14. Extract horizontal lines: Erosion: Use horizontal structuring elements to erode and eliminate non-horizontal lines.

[0079] Dilation: Dilate the erosion result to strengthen the horizontal lines.

[0080] Line merging: Merge the extracted vertical and horizontal line images by weight to obtain complete table grid lines.

[0081] C21, contour detection and sorting: Contour Detection: Contours are detected in the merged image, each of which corresponds to a potential cell.

[0082] C22, contour sorting: Sort by row: Sort from top to bottom based on the y coordinate of the contour to determine the row position of the cell.

[0083] Sort by column: Within each row, sort by x-coordinate from left to right to determine the column position.

[0084] C23, Cell grouping and alignment: Grouping: Group the sorted contours into a two-dimensional structure of rows and columns for easy subsequent processing.

[0085] Alignment: Calculate the center point of the column and adjust the horizontal alignment of the cells to ensure the correctness of the column.

[0086] C3. Cell filtering and verification: Size filtering: Remove outlines that are too small or too large, and filter out non-cell areas.

[0087] C4. Legality Verification: Edge Check: Verifies that enough white pixels exist on all four edges of each cell (table lines).

[0088] Legal cells: Only cells that pass the edge check are retained to ensure the accuracy of the results.

[0089] Result output: Returns the final confirmed cell coordinate list, each element contains the upper left corner coordinates (x, y) and width and height (width, height) of the cell.

[0090] Table structure reconstruction can adopt a combination of rule-based and deep learning methods to improve the recognition accuracy of complex tables.

[0091] Example 6

[0092] This embodiment is based on embodiment 1: The step D comprises: D1. Use R-tree spatial index to build a fast query structure for table cells; this can quickly determine the table cell to which each text block belongs, avoiding cross-cell merging.

[0093] D2, traverse all detected text blocks, convert their coordinates from the local coordinate system of the blocks to the global coordinate system of the original image, and merge overlapping text blocks in the vertical direction; D3. Use the OpenCV library to check the legitimacy of the merged text blocks and fix possible geometric problems (such as invalid polygons); calculate the coordinates of the center point of each text block to facilitate subsequent spatial query and sorting.

[0094] D4. Use the spatial index to query the table cell where the center point of each text block is located, establish the correspondence between the text block and the cell, ensure that the merging process is only performed within the same cell, avoid erroneous merging across cells, merge adjacent text blocks in the horizontal direction, and assign the text block coordinate array as a merge mark; D5. According to the merge markers, the merged text blocks are collected to form the final text area list. The OpenCV library is used again to ensure that each text block polygon is legal and retains its association information with the table cells. D6. Output the merged text block coordinate array and the corresponding table cell index, which are used to fill the text recognition results back into the corresponding position of the cell.

[0095] Text block coordinate array: Use an array to mark all texts. Each element in the array corresponds to a text block. Fill the corresponding positions of the two text blocks that need to be merged with the same value, indicating that they need to be merged into one.

[0096] For example, when the array is created, all elements are 0. If the first text block and the second text block are merged, the first and second position elements of the array are replaced with 1, indicating that the two text blocks have been merged. If the third text block and the fourth text block are merged, the third and fourth elements are marked as 2 to distinguish them from the previous 1, because they are not merged with the text block marked as 1.

[0097] Example 7

[0098] This embodiment is based on Embodiment 6: Merging overlapping text blocks in the vertical direction: for adjacent blocks in the vertical direction, checking the text blocks in their overlapping areas, determining whether the text blocks in the upper and lower blocks overlap or are close to each other in the overlapping areas, and if so, merging these text blocks; Merge adjacent text blocks horizontally, sort them from left to right according to their horizontal positions, traverse the sorted text block list, compare adjacent text blocks, use geometric union operations on eligible text blocks to merge them into a new polygon, and update the merge mark to avoid repeated processing.

[0099] The condition is met, which means that the following conditions are met at the same time: the two text blocks belong to the same table cell; they are adjacent in the horizontal direction and have overlapping areas in the vertical direction; and the geometric intersection of the two text blocks meets a certain area or ratio threshold.

[0100] Example 8

[0101] This embodiment is based on embodiment 1: The step E comprises: E1. Use the average probability of each character output by the text recognition model as the confidence score, filter the results with confidence lower than the threshold, remove the blank text area, and mark the area with confidence lower than the threshold but higher than the lower limit as the area to be manually verified; E2. Visualization processing: Create a mask image of the same size as the original image, use polygon filling to mark the text area, enhance the brightness of the text area, and generate the processed visualization result.

[0102] E3. Layout analysis to generate output results in standard format.

[0103] In addition to the standard text output format, it can also support multiple output formats, such as PDF, Word, Excel, etc. Layout analysis can use a layout segmentation model based on deep learning to more accurately identify the logical structure of the document. In addition, OCR post-processing technologies such as dictionary matching and contextual error correction can be introduced to further improve recognition accuracy.

[0104] After completing all processing, the system will automatically release the used memory resources and clean up temporary files. If an exception occurs during the processing (such as insufficient memory, recognition failure, etc.), the system will trigger the corresponding exception handling mechanism, including task retry, result rollback, etc. The system will also record detailed error logs to facilitate subsequent problem analysis and optimization.

[0105] Specific implementation steps of E1: E11. Calculate the confidence of the recognition result Definition of confidence: Confidence is the degree of trust that a text recognition model has in its recognition results, and is usually a value between 0 and 1. The higher the confidence, the more reliable the recognition result.

[0106] Get confidence: The output of the text recognition model usually contains a confidence score for each recognition result. If the model does not directly provide confidence, you can calculate the confidence through the output probability distribution of the model.

[0107] E12. Filter low confidence results Set confidence threshold: Set a confidence threshold of 0.7 according to actual needs. Results below this threshold are considered unreliable.

[0108] Filtering logic: Traverse all recognition results and check whether their confidence is lower than the threshold. If the confidence is lower than the threshold, remove the result from the final result list.

[0109] E13. Remove blank text areas A blank text area refers to an area where the recognition result is an empty string or contains only blank characters, such as spaces and line breaks.

[0110] Detect blank text: Traverse all recognition results and check whether the recognized text is empty or contains only blank characters. If it is blank text, remove the result from the final result list.

[0111] E14. Mark areas that require manual verification Conditions requiring manual verification: The confidence level is lower than the threshold of 0.7 but higher than the lower limit of 0.5. This indicates that the recognition result contains special characters or content that does not meet expectations.

[0112] Tag logic: For areas that require manual verification, add a tag to the results. Save or record these areas separately for subsequent manual review.

[0113] Example 9

[0114] The system for implementing the ultra-large format document intelligent recognition method of the present invention comprises: A data receiving module is used to obtain the document image to be recognized and pre-process the image data; The image segmentation module uses an adaptive block strategy to dynamically segment images based on the available system memory or image features; The task scheduling module converts the block tasks into asynchronous task queues and prioritizes the tasks according to the complexity and importance of the blocks; The text recognition module pre-configures multiple candidate text recognition models, and each candidate text recognition model corresponds to a different recognition scenario. The block to be recognized is input into the corresponding candidate text recognition model to obtain the recognition result; The result merging module is used to collect the recognition results of each block and intelligently merge them according to the recorded location information; The post-processing optimization module is used to optimize the text area, perform visualization and layout analysis on the optimized text, and generate output results in a standard format.

[0115] The above-mentioned embodiments only express the specific implementation of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A method for intelligent recognition of ultra-large format documents based on block parallelism, characterized in that: The following steps are involved: A. Image Data Preprocessing Obtain image data of the document to be recognized and adopt image enhancement strategies to improve the quality of the original image; B. Image Block Processing Adopt an adaptive block strategy to dynamically block according to the available system memory or image features, maintain a certain overlap area between adjacent blocks to ensure the integrity of the text, and then convert the block task into an asynchronous task queue; C. Parallel text detection and recognition Use thread pools to process each block in parallel, or use a distributed computing architecture to distribute tasks to multiple computing nodes for execution; detect text areas and table structures, record their relative position information, and perform text recognition on the detected text areas through a text recognition model; D. Result merging and structural reconstruction Collect the processing results of each block and merge them according to the recorded location information. For texts spanning multiple blocks, reorganize the text by analyzing the contents of the overlapping areas. E. Post-processing optimization and output The confidence score is given according to the recognition results output by the text recognition model to optimize the text area, and the optimized text is visualized and analyzed to generate output results in a standard format.

2. The method for intelligent recognition of ultra-large format documents based on block parallelism according to claim 1 is characterized in that: In step A, after format verification of the input data, the data is converted into a numpy array, and then the numpy array is decoded into an image format.

3. The method for intelligent recognition of ultra-large format documents based on block parallelism according to claim 1 is characterized in that: In step B, if the width and height of the image exceed the preset values, the image is divided into grid blocks, and a fixed-size overlapping area is set between adjacent blocks, and the boundary coordinates and index information of each block are recorded for subsequent image processing; The calculation formula of the adaptive block strategy is: k=[(Mm) / (mo)]+1, v=[(Nn) / (no)]+1, k is the number of blocks in the vertical direction, v is the number of blocks in the horizontal direction, and k and v are rounded down; M and N represent the length and width of the image, respectively, and m and n represent the length and width of the block, respectively; o represents the side length of the overlapping area, which is a preset value; The values ​​of m and n are dynamically assigned according to the available memory of the system or the image features.

4. The method for intelligent recognition of ultra-large format documents based on block parallelism according to claim 1 is characterized in that: In the step C, the detected text regions are processed in parallel, the tilt angle is calculated for each text region, image rotation correction is performed according to the tilt angle, and the corrected image is used for text recognition.

5. The method for intelligent recognition of ultra-large format documents based on block parallelism according to claim 1 is characterized in that: The table structure detection in step C includes the following steps: C1. Use the erosion and dilation method to extract the table frame structure of the table area; C2. Detect the table outline. According to the table outline, each outline corresponds to a potential cell. Sort the outline coordinates from small to large, calculate the column center point, and adjust the horizontal alignment of the cell. C3, remove the outlines that are too small or too large, and filter out the non-cell areas; C4, check whether there are enough white pixels on the four edges of each cell, and only keep the cells that pass the edge check; C5. Return the final confirmed cell coordinate list and generate a table structure.

6. The method for intelligent recognition of ultra-large format documents based on block parallelism according to claim 1 is characterized in that: The step D comprises: D1. Use R-tree spatial index to build a fast query structure for table cells; D2, traverse all detected text blocks, convert their coordinates from the local coordinate system of the blocks to the global coordinate system of the original image, and merge overlapping text blocks in the vertical direction; D3. Use the OpenCV library to check the legitimacy of the merged text blocks, fix possible geometric problems, and calculate the coordinates of the center point of each text block; D4. Use the spatial index to query the table cell where the center point of each text block is located, establish the correspondence between the text block and the cell, merge adjacent text blocks in the horizontal direction, and assign the text block coordinate array as a merge mark; D5. According to the merge mark, the merged text blocks are collected to form the final text area list. The OpenCV library is used again to ensure that the polygons of each text block are legal and retain their association information with the table cells. D6. Output the merged text block coordinate array and the corresponding table cell index, which are used to fill the text recognition results back into the corresponding position of the cell.

7. The method for intelligent recognition of ultra-large format documents based on block parallelism according to claim 6 is characterized in that: Merging overlapping text blocks in the vertical direction: for adjacent blocks in the vertical direction, checking the text blocks in their overlapping areas, determining whether the text blocks in the upper and lower blocks overlap or are close to each other in the overlapping areas, and if so, merging these text blocks; Merge adjacent text blocks horizontally, sort them from left to right according to their horizontal positions, traverse the sorted text block list, compare adjacent text blocks, use geometric union operations on eligible text blocks to merge them into a new polygon, and update the merge mark to avoid repeated processing.

8. The method for intelligent recognition of ultra-large format documents based on block parallelism according to claim 7 is characterized in that: The condition is met, which means that the following conditions are met at the same time: the two text blocks belong to the same table cell; they are adjacent in the horizontal direction and have overlapping areas in the vertical direction; and the geometric intersection of the two text blocks meets a certain area or ratio threshold.

9. The method for intelligent recognition of ultra-large format documents based on block parallelism according to claim 1 is characterized in that: The step E comprises: E1. Use the average probability of each character output by the text recognition model as the confidence score, filter the results with confidence lower than the threshold, remove the blank text area, and mark the area with confidence lower than the threshold but higher than the lower limit as the area to be manually verified; E2. Visualization processing: Create a mask image of the same size as the original image, use polygon filling to mark the text area, enhance the brightness of the text area, and generate the processed visualization result; E3. Layout analysis to generate output results in standard format.

10. A system for implementing the method for intelligent recognition of ultra-large format documents according to any one of claims 1 to 9, characterized in that: include: A data receiving module is used to obtain the document image to be recognized and pre-process the image data; The image segmentation module uses an adaptive block strategy to dynamically segment images based on the available system memory or image features; The task scheduling module converts the block tasks into asynchronous task queues and prioritizes the tasks according to the complexity and importance of the blocks; The text recognition module pre-configures multiple candidate text recognition models, and each candidate text recognition model corresponds to a different recognition scenario. The block to be recognized is input into the corresponding candidate text recognition model to obtain the recognition result; The result merging module is used to collect the recognition results of each block and intelligently merge them according to the recorded location information; The post-processing optimization module is used to optimize the text area, perform visualization and layout analysis on the optimized text, and generate output results in a standard format.

Images