Text recognition method, system, and storage medium

Abstract

The present application relates to the technical fields of image processing and character recognition, and particularly relates to a text recognition method, a system and a storage medium, wherein the text recognition method comprises: obtaining position coordinate information of a global text block in an input image; performing horizontal slicing on the global text block to cut it into at least one horizontal slice along a vertical direction, each horizontal slice covering one or more continuous lines of the global text block; performing vertical slicing on the global text block to cut it into at least one vertical slice along a horizontal direction, each vertical slice covering one or more continuous columns of the global text block; performing text recognition on the horizontal slices and the vertical slices respectively to obtain a first set of recognition results and a second set of recognition results; performing spatial coordinate alignment and comparison on the results corresponding to the same text unit in the first set of recognition results and the second set of recognition results, and outputting the recognition text of the text unit if the results are consistent.

Description

Technical Field

[0001] This invention relates to the fields of image processing and character recognition technology. Specifically, it relates to a text recognition method, system, and storage medium that improves recognition accuracy by performing horizontal and vertical slicing of global text blocks in an image and comparing and verifying the recognition results of the same text in the two types of slices based on coordinate information. This invention is particularly applicable to, but not limited to, high-precision applications such as automated processing of receiving and inspection documents in semiconductor manufacturing plants, digitization of financial statements, and structured extraction of medical records. Background Technology

[0002] In the information age, automated information extraction from paper documents has become a key determinant of operational efficiency for modern enterprises. Taking the semiconductor industry as an example, the daily flow of purchase invoices, delivery notes, and receiving slips within its supply chain carries core data such as invoice numbers, purchase order numbers, material codes, and amounts. The accurate entry of this data directly impacts inventory accuracy, financial settlement efficiency, and production scheduling response speed. Currently, the industry's prevalent operational model still relies heavily on manual data entry and verification. This model is not only labor-intensive but also fails to meet the industry benchmark of 99.99% error rate.

[0003] In existing technologies, Optical Character Recognition (OCR) technology has been widely used in document digitization. A typical OCR workflow includes: image preprocessing (e.g., grayscale conversion, denoising, binarization), text detection (e.g., object detection networks based on EAST, DBnet, or CRAFT), and text recognition (e.g., Convolutional Recurrent Neural Network (CRNN) + Connectionist Temporal Classification (CTC), Transformer-based OCR, or attention-based encoder-decoder. Although the development of deep learning has improved end-to-end recognition performance, existing technologies still suffer from the following systemic technical shortcomings, which have long troubled those skilled in the art.

[0004] (i) Lack of cross-validation mechanism in single recognition: Whether it is traditional OCR or deep learning-based models, recognition is usually performed only once for each detected text block. When the input image has local blur (e.g., motion blur or defocus blur), uneven lighting (e.g., specular reflection or shadow occlusion), character adhesion (e.g., stroke merging caused by low-resolution scanning), defects (e.g., creases or stains), or complex backgrounds (e.g., table lines overlapping with characters), single recognition is prone to producing erroneous outputs, and the system cannot automatically detect its own errors. Although subsequent manual review or business rule verification can capture some errors, it increases processing latency and operational costs. There is no existing technology that can automatically generate redundant information and perform self-comparison during the recognition process, thereby forming an error detection and correction mechanism.

[0005] (ii) Lack of Redundancy Verification Based on Spatial Layout: Each text unit is located at the intersection of a row (horizontal) and a column (vertical). In other words, each text unit simultaneously belongs to a unique row context and a unique column context. Those skilled in the art should recognize that horizontal slices (i.e., sub-images extending horizontally) preserve complete horizontal neighborhood information, which is beneficial for correcting isolated character recognition errors using intra-row semantic relevance; vertical slices (i.e., sub-images extending vertically) preserve complete vertical neighborhood information, which is beneficial for distinguishing similar strings using intra-column format consistency (e.g., thousands separator alignment in numerical columns). However, no existing technology provides a method for dual recognition and comparison of the same text unit from both "row perspective" and "column perspective." This prevents existing systems from utilizing the redundant evidence provided by these two orthogonal directions, thus fundamentally limiting recognition robustness.

[0006] (iii) Limited image preprocessing capabilities, unable to adapt to diverse image quality: In practical applications, input images vary significantly in quality. For example, some images possess high resolution and contrast, while others suffer from low illumination, high noise, large tilt angles, creases, fading, and other degradations. Traditional methods lack the ability to adaptively enhance the same image using multiple versions and fuse the recognition results. Therefore, when image quality deviates from the training distribution, recognition performance drops sharply.

[0007] (iv) Single recognition model and insufficient information source: Most systems rely on only one OCR engine (such as Tesseract, PaddleOCR or commercial software development kits (SDK)). Different engines have different strengths and weaknesses in adapting to different fonts, languages ​​and layouts. The recognition errors of a single engine are correlated and difficult to correct through internal mechanisms.

[0008] In summary, there is an urgent need for a method and system that can acquire text location coordinates, generate horizontal and vertical slices based on the coordinates, achieve accurate spatial correlation through coordinate mapping, and use horizontal and vertical dual-view cross-validation to detect and correct single recognition errors, so as to significantly improve recognition accuracy and robustness. Summary of the Invention

[0009] In view of the shortcomings of the prior art, the technical problem to be solved by the present invention is to provide a text recognition method, system and storage medium that can solve the problems of lack of verification in single recognition, lack of spatial redundancy verification and missing location association, and achieve high accuracy and high robustness in text recognition.

[0010] To solve the above-mentioned technical problems, the present invention provides a text recognition method, comprising the following steps: Obtain the position coordinate information of a global text block in the input image, wherein the global text block contains at least one text unit; Based on the location coordinate information, the global text block is horizontally sliced, dividing it into at least one horizontal slice along the vertical direction. Each horizontal slice covers one or more consecutive lines in the global text block. Based on the location coordinate information, the global text block is sliced ​​vertically to divide it into at least one vertical slice along the horizontal direction. Each vertical slice covers one or more consecutive columns of the global text block. Text recognition is performed on the horizontal slices and vertical slices respectively to obtain a first recognition result set and a second recognition result set; Based on the location coordinate information, the spatial coordinates of the results corresponding to the same text unit in the first recognition result set and the second recognition result set are aligned and compared. If they are consistent, the recognized text of the text unit is output.

[0011] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is: providing a text recognition system, comprising: One or more processors; A memory, coupled to the processor, stores instructions that, when executed by the processor, cause the system to perform the methods described above.

[0012] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is to provide a storage medium on which a computer program is stored, wherein the program is executed by a processor to implement the above-mentioned method.

[0013] In summary, the text recognition method, system, and storage medium of the present invention have the following unexpected technical effects: (i) By independently generating horizontal and vertical slices on the same input image, this invention constructs two orthogonal redundant recognition channels for each text unit. Each text unit simultaneously receives two independent recognitions from its row context and its column context. When the two recognition results are consistent, the reliability of the output is higher than either single recognition; when they are inconsistent, the system automatically detects potential errors without external triggering, thus improving the accuracy and efficiency of text recognition.

[0014] (ii) An overlapping region is set between adjacent slices, so that text units located near row or column boundaries are covered by multiple slices. Even if a slice is misidentified due to local degradation, other overlapping slices may still provide the correct result and correct it through weighted voting within the group. This feature enhances the system's robustness to local image defects, which is not possible with non-overlapping uniform slices.

[0015] (III) This invention maps the local bounding box of each slice recognition result back to the original image coordinate system through coordinate transformation, and uses the intersection-union ratio (IUGR) as a quantitative indicator of spatial overlap. This method solves the problem that slice recognition results cannot accurately correspond to the original text position. Since the transformation is deterministic, as long as the slice contains a sufficient portion of the target text, an accurate spatial correspondence can be established regardless of the slice's size and position. Furthermore, the IUGR threshold can be flexibly adjusted according to business needs—the threshold can be tightened for scenarios with extremely high accuracy requirements, and appropriately relaxed for scenarios with poor image quality. More importantly, one original text unit can be associated with multiple slice results, which lays a data foundation for subsequent multi-source information fusion. Compared with existing technologies that rely on string matching or coarse position estimation, the spatial association mechanism of this invention is more reliable and quantifiable.

[0016] (iv) When the recognition results in the horizontal and vertical directions conflict, this invention will not simply discard either result, but will execute a hierarchical retry mechanism. The retry strategy can be dynamically selected according to the specific circumstances of the inconsistency: for example, the expanded margin or overlap rate of the slice can be adjusted and the slice can be re-sliced ​​for recognition; a recognition engine more suitable for the current image features (such as font and layout) can be used; when automated means cannot solve the problem, the questionable text unit will be handed over to manual review. Users can also configure the maximum number of retries and the priority of the retry strategy according to the business's trade-off between latency and accuracy. This hierarchical design, from lightweight parameter adjustment to heavy-duty model invocation and then to manual backup, allows the system to maintain a high degree of automation while also reserving a reliable exit for the most difficult samples.

[0017] (V) This invention allows for the application of multiple preprocessing methods to the same input image, generating multiple derived images. Each image independently completes the entire recognition process, and then the multiple recognition results are fused together. Different preprocessing methods have their own focuses: histogram equalization excels at improving low-contrast images, adaptive threshold binarization is beneficial for separating foreground and background, super-resolution reconstruction can restore edge details in low-resolution images, and morphological operations can effectively remove noise. These methods are complementary in their adaptability to image degradation. The fused output is generally superior to any single preprocessing path, and the system does not need to know in advance what type of degradation exists in the image, nor does it need to tune parameters individually for each degradation, further improving the accuracy of text recognition.

[0018] (vi) This invention can utilize multiple heterogeneous recognition engines (e.g., CRNN-based OCR engines, Transformer-based OCR engines, CNN-based OCR engines, multimodal models incorporating layout information, and even large language models). Each engine independently completes horizontal and vertical slice recognition and cross-validation, and then the outputs of each engine are cross-compared and fused. The error patterns of different engines often have low correlation—some are sensitive to digit strings, some are more accurate with English words, and some are good at utilizing document structure information. By using their consensus as the final output through voting or confidence weighting, the systematic bias of a single engine can be effectively eliminated, allowing the overall performance to exceed the upper limit of any single model.

[0019] (vii) Before slicing, this invention performs tilt correction on each line of text individually, rather than performing a global rotation on the entire image. This is because in actual scanning or photography, especially with thick books or folded documents, the tilt angles of different lines are often inconsistent, and a global rotation can only cover a portion of the area. The line-by-line correction method of this invention extracts the bounding box corners of all text units within a line, calculates the direction vector of each text unit based on these corners, and then performs a weighted average of the vectors based on factors such as character size and recognition confidence to obtain the overall tilt angle of the line. Then, an affine transformation is performed only on the image strip corresponding to that line. In this way, each line can be independently rotated to a horizontal state, thereby significantly improving the accuracy of subsequent slicing alignment and the success rate of spatial matching. For complex distortions such as page curvature, this line-by-line processing method has irreplaceable advantages.

[0020] In summary, this invention constructs a complete and highly robust text recognition scheme through a series of techniques, including cross-validation of horizontal and vertical slices, overlapping redundant coverage, precise correlation between coordinate mapping and intersection-union ratio, adaptive retries, multi-path preprocessing fusion, multi-engine collaboration, and line-by-line tilt correction. These features are not isolated combinations, but rather work together in a progressive manner to jointly solve several interrelated problems that have long existed in existing technologies. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of some embodiments of the text recognition method of the present invention.

[0022] Figure 2 An exemplary image is shown, depicting a global text block and its multiple text units.

[0023] Figure 3 Exemplary examples show the relevant information. Figure 2 It is a portion of the baseline data set of global text blocks in the dataset.

[0024] Figure 4 An example was shown regarding Figure 2 One of the horizontal slices of the global text block.

[0025] Figure 5 Exemplary examples show the relevant information. Figure 2 One of the vertical slices of the global text block.

[0026] Figure 6 A flowchart of a spatial coordinate alignment and comparison method is shown as an example in some embodiments of the text recognition method of the present invention.

[0027] Figure 7 Flowcharts of some embodiments of the text recognition method of the present invention are shown as examples.

[0028] Figure 8 Flowcharts of some embodiments of the text recognition method of the present invention are shown as examples.

[0029] Figure 9 A flowchart of the text tilt correction step is shown as an example in some embodiments of the text recognition method of the present invention.

[0030] The diagrams in the instruction manual are labeled as follows: Global text block 100; text unit 101. Detailed Implementation

[0031] The following disclosure provides various embodiments or examples for implementing different features of the invention. Specific examples of components and arrangements will be described below to simplify the invention. Of course, these are merely examples and are not intended to limit the invention. For example, in the following description, forming a first component above or on a second component may include embodiments where the first and second components are in direct contact, or embodiments where other components may be formed between the first and second components such that the first and second components are not in direct contact. Furthermore, reference numerals and / or characters may be repeated in various instances of the invention. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or configurations.

[0032] Furthermore, spatial relation terms such as "below," "under," "below," "above," and "above" may be used herein to readily describe the relationship between one element or component and another element (or component) or component (or component) as shown in the figure. In addition to the orientations shown in the figure, spatial relation terms will encompass various different orientations of the device in use or operation. The device may be positioned in other ways (rotated 90 degrees or in other orientations) and will be interpreted accordingly through the spatial relation descriptors used herein.

[0033] Although the numerical ranges and parameter settings presented in this invention are approximations, the numerical settings in specific instances are reported as precisely as possible. Any numerical value, however, inherently contains certain inevitable errors arising from the standard deviation found in the respective test measurements. Similarly, as used herein, the term "about" generally refers to within 10%, 5%, 1%, or 0.5% of a given value or range. Alternatively, the term "about" means within an acceptable average standard error that can be conceived by one of ordinary skill in the art. Except in instances of operation / work, or unless expressly stated otherwise, all numerical ranges, totals, values, and percentages, such as those for material quantities, durations, temperatures, operating conditions, amounts, and other similarities disclosed herein, should be understood to be modified by the term "about" in all cases. Therefore, unless otherwise stated, the numerical parameter settings set forth in this invention and the appended claims are approximations that can be changed upon request. At a minimum, each numerical parameter should be interpreted as being based on the number of significant figures reported and applying ordinary rounding techniques. A range herein may be expressed as from one endpoint to another or between two endpoints. All scopes disclosed herein include endpoints unless otherwise stated.

[0034] Furthermore, the technical parts described in this invention and the appended claims are primarily the improved technical parts of this invention, and do not limit the object protected by this invention to only having these technical parts. Other known essential components (structures and / or methods) and / or non-essential components of the object protected, besides the technical parts described in this invention and the appended claims, are not included in this invention and the appended claims because they do not fall within the scope of improvements of this invention; however, this does not mean that the object protected by this invention does not possess these known components.

[0035] Please see Figure 1 and Figure 2 , Figure 1 A flowchart of a text recognition method in some embodiments of the present invention is shown as an example. Figure 2 An exemplary image of a global text block 100 and its internal text units is shown. In the illustrated embodiment, the text recognition method includes the following steps: S110. Obtain the position coordinate information of the global text block 100 in the input image.

[0036] The global text block 100 can be defined as the total set of all text to be recognized in the input image. In simpler terms, it is the largest area from which the user wants to extract all text information from the image. Typically, this area corresponds to one or more tables, one or more paragraphs, or the main content area of ​​the image. The global text block 100 emphasizes wholeness and globality, encompassing the entire text layout to be subsequently sliced ​​horizontally and vertically. For example, in a semiconductor purchase invoice, the entire table area consisting of all rows and columns can be considered a global text block 100. This block contains multiple text units 101. Similarly, in an image containing text paragraphs and tables, the entire text paragraph and table can be considered a global text block 100. From a coordinate perspective, the global text block 100 is typically approximated by a bounding rectangle that can encompass all text units 101 within the block. Of course, for irregularly shaped text blocks (such as tables with gaps), a combination of multiple bounding boxes or a convex hull can be used to describe them, but in most industrial scenarios, a rectangular bounding box is sufficient.

[0037] The global text block 100 includes at least one text unit 101. In this embodiment, the global text block 100 includes a plurality of text units 101. Each text unit 101 can be configured as an independent string. When determining whether a string is an independent text unit 101, it can be defined based on the separator between it and surrounding strings. The separator includes, but is not limited to, spacing, separator lines, or whitespace. For example, if a continuous string has a predetermined separator before (on the left) and also a predetermined separator after (on the right) of the current line, then this continuous string can be defined as a text unit 101. In other words, a detectable separator should exist between a text unit 101 and its adjacent text units 101.

[0038] In real-world scenarios, text units 101 typically correspond to the smallest segments of information in natural reading or business processing. For example, in a semiconductor purchase invoice form, the individual content within each cell—such as "global semiconductor sales," "6,210," or "PO-123456"—can be considered a text unit 101. These strings are usually separated from adjacent cells by cell borders or obvious whitespace, thus conforming to the aforementioned delimiter rules. As another example, in an English paragraph, each word separated by spaces (such as "The," "semiconductor," "industry") constitutes a text unit 101, as the spaces between words provide a measurable distance. For Chinese text, since there are usually no spaces between characters, various strategies can be used to divide text units 101. For example, punctuation marks (such as commas, periods, semicolons, etc.) can be considered delimiters, with the strings before and after the punctuation marks treated as independent text units 101. As yet another example, an entire line can be considered a text unit 101.

[0039] The detection of the separator markers can be achieved using various known image processing techniques. For example, the spacing or blank space can be determined by analyzing the horizontal distance between the bounding boxes of adjacent characters: if this distance is greater than a preset threshold (e.g., a preset multiple of the average character width), then a separator marker is determined to exist. Separator lines or borders can be identified using Hough transform, edge detection, or deep learning-based segmentation models.

[0040] It should be noted that the definition of text unit 101 does not require absolute precision. It is sufficient to divide the text information within the global text block 100 into several independent units that facilitate subsequent horizontal and vertical slicing cross-validation. The core of this invention lies in using the positional coordinate information of these text units 101 to guide slice generation and result alignment, rather than pursuing perfect semantic word segmentation. Therefore, even if some strings are mistakenly divided into multiple text units 101 (e.g., splitting an English word into two character blocks), subsequent horizontal and vertical slicing recognition and cross-validation can still be corrected through coordinate matching and confidence levels. For Chinese text, adopting a strategy of "treating an entire line as a text unit 101," although coarser in granularity, is often more practical in paragraphs without obvious separators and does not affect the effectiveness of subsequent horizontal and vertical slicing, because horizontal and vertical slicing still perform cross-validation within rows and columns.

[0041] Please see Figure 2 This image contains a title, a continuous narrative text, and a borderless table. Based on this example, the global text block 100 is defined as the set of all text to be recognized in the entire image, specifically the entire rectangular area from the top title "Semiconductor Industry Technology Evolution and Market Landscape (2025-2030)" to the last row of the bottom table. The text units 101 within this global text block 100 employ different segmentation strategies based on typographical features. Figure 2 The boundary of a portion of the text unit 101 is marked by a dashed box.

[0042] Figure 2The continuous narrative text area in the document comprises a title and a continuous paragraph consisting of multiple sentences. Since Chinese narrative text typically lacks clear word breaks, dividing the text into units 101 by character or phrase would hinder subsequent cross-validation of horizontal and vertical slices. Therefore, in this area, each line of text is treated as an independent text unit 101. For example, the first line, titled "Semiconductor Industry Technology Evolution and Market Landscape (2025-2030)," is enclosed in a dashed box, indicating it is a text unit 101; the following line, containing the narrative text "The global semiconductor industry is undergoing profound changes driven by artificial intelligence (AI), high-performance computing (HPC), and automotive electrification / intelligentization. According to WSTS and IC Insights," is also marked with a dashed box, similarly constituting a text unit 101. Each subsequent line of narrative text (e.g., "Latest data: The global semiconductor market size is projected to reach $715 billion in 2025, a year-on-year increase of 10.3%. Logic chips (LogicIC) account for 31.2%, memory chips, etc.") constitutes a separate text unit 101. The bounding box of each text unit 101 is defined by its upper left corner coordinates (X0, Y0) and lower right corner coordinates (X1, Y1), and contains the entire content of the line from the first character to the last character (including punctuation marks).

[0043] Figure 2The borderless table area is located below the narrative text, with the header "Key Data Summary Table (Unit: US$ Billion, unless otherwise noted)". The table's row and column structure is indicated only by horizontal and vertical text alignment; there are no explicit table lines or borders. The table contains multiple indicator rows and several data columns. For example, the first column includes rows such as "Global Semiconductor Sales", "Logic Chips", "Memory Chips", "HBM", "Analog Chips", "Automotive Chips", "Optoelectronics / Sensors", "Wafer Foundry Revenue", "Advanced Packaging Market Size", "Semiconductor Equipment Sales (WFE)", "Global Capital Expenditure (CapEx)", "Automotive Semiconductor Value per Vehicle", "SiC Power Device Penetration (%)", and "HBM Bandwidth (TB / s)". The top row includes columns such as "Indicators", "Actual 2024", "Forecast 2025", "Forecast 2026", "CAGR (24-26)", and "Key Supporting Factors". In this borderless table, each independent string within a cell is defined as a text unit 101 (some text units 101 are exemplarily marked with dashed boxes). Examples include "Indicator," "Actual in 2024," and "Forecast in 2025" in the header, and "Global Semiconductor Sales," "6,210," "7,150," "8,050," "+13.9%," and "AI Computing Power Demand, Automotive Electronics Penetration" in the data rows. Whether a string constitutes an independent text unit 101 can be determined based on the separator between it and adjacent strings. Specifically, in a row, there is a clear white space between two adjacent cells (e.g., "Global Semiconductor Sales" and "6,210"), and this white space is typically greater than a preset multiple of the average character width. Based on this, they can be divided into different text units 101. Similarly, vertically, there is also a row spacing between two cells in the same column (e.g., "6,210" and "2,850"), making them belong to different text units 101.

[0044] The bounding boxes of these text units 101 can be automatically obtained through initial global text recognition (such as well-known techniques like detection models based on EAST or DBnet). Since the table has no borders, the detection model needs to rely on the spatial distribution between texts to infer the bounding box of each string, which is feasible in borderless tables because there are measurable blank areas on the left, right, top, and bottom sides of each individual string. Figure 2The position and size of the dashed box visually represent the bounding box of each text unit 101: for short text in the header, the dashed box is close to the text; for longer text such as "AI computing power demand, automotive electronics penetration", the dashed box covers the entire phrase; for a whole line of text in a narrative paragraph, the dashed box extends from the first character of the line to the last character.

[0045] Whether the entire line is treated as a text unit 101 (e.g., a descriptive text area) or each string within a cell is treated as a text unit 101 (e.g., a table area), the subsequent steps of this invention (horizontal slicing, vertical slicing, coordinate mapping, and horizontal / vertical comparison) function correctly. For an entire line of text units 101, horizontal slicing can be further divided by line (in fact, each horizontal slice will contain the entire line of text units 101), while vertical slicing can generate virtual column slices based on the horizontal positions of different areas within the line, thereby achieving cross-validation of different semantic segments within the line of text. For fine-grained text units 101 in a table, horizontal and vertical slicing will directly correspond to each cell, making cross-validation more direct and effective.

[0046] The location coordinate information of the global text block 100 refers to a data set (e.g., a reference data set) describing the spatial position of each text unit 101 within the block in the original image. Specifically, the location coordinate information can be configured to include the bounding box coordinates of each text unit 101, typically represented by the coordinates of the top-left corner (X0, Y0) and bottom-right corner (X1, Y1) of the bounding box; the recognized text content of each text unit 101; and the recognition confidence of each text unit 101. Furthermore, preferably or optionally, for tilted text units 101, the location coordinate information may also include the rotation angle of the bounding box or the coordinates of the four vertices. This information collectively constitutes the reference data set, providing spatial reference and initial recognition results for subsequent horizontal slicing, vertical slicing, and spatial coordinate alignment and comparison steps. The location coordinate information can be obtained through known technical means, such as through OCR-based detection methods (e.g., using text detection networks such as EAST and DBnet combined with a CRNN+CTC recognition engine).

[0047] Please see Figure 3 , Figure 3 Exemplary examples show the relevant information. Figure 2 This is a portion of the baseline data set of global text block 100. To more clearly illustrate the composition and acquisition method of the above-mentioned location coordinate information, the following continues to combine... Figure 2 The specific images shown are examples of OCR-based detection methods.

[0048] In the OCR-based detection method, taking the text unit 101 with the title "Semiconductor Industry Technology Evolution and Market Landscape (2025-2030)" as an example, its bounding box coordinates are top left corner (39,6), top right corner (393,7), bottom right corner (393,25), bottom left corner (39,24), and the extreme values ​​of the axis-aligned rectangle are xmin=39, ymin=6, xmax=393, ymax=25. The text is identified as the title mentioned above, with a confidence level of approximately 0.9948.

[0049] Taking a line from a narrative paragraph, such as “The global semiconductor industry is undergoing profound changes driven by artificial intelligence (AI), high-performance computing (HPC), and the electrification / intelligentization of automobiles,” as an example (text unit 101), its bounding box covers the entire line width (xmin=35, xmax=762), with a confidence level of approximately 0.9688.

[0050] Taking cells in a borderless table, such as “Global Semiconductor Sales” and “6,210”, as examples, their bounding boxes are clearly separated in the horizontal direction (the former xmax=115, the latter xmin=173), with an interval of about 58 pixels, indicating that they belong to different text units 101.

[0051] S120, Horizontal slice: Please refer to Figure 4 Based on the location coordinate information, the global text block 100 is horizontally sliced, dividing it into at least one horizontal slice along the vertical direction. Each horizontal slice covers one or more consecutive lines in the global text block 100.

[0052] According to one or more embodiments, during the process of performing horizontal slicing on the global text block 100, the slice is performed in units of each line or several consecutive lines, based on the vertical coordinate range of each line of text unit 101 within the global text block 100, and there may be overlapping areas between adjacent horizontal slices.

[0053] By performing cluster analysis on the ordinates (ymin, ymax) of all text units 101 obtained in step S110, the vertical range of each row can be determined.

[0054] By analyzing the ymin and ymax values ​​of each text unit 101, multiple distinct line bands can be identified. For example, the title line has ymin=6 and ymax=25; the first line of a narrative paragraph has ymin=77 and ymax=91; the header "Key Data Summary Table (Unit: US$ billion, unless otherwise specified)" has ymin=243 and ymax=257; the table header row has ymin ≈273~292; and so on.

[0055] For each line of text, a horizontal slice is generated. According to one or more embodiments, the vertical range extends downward from the minimum ymin of the line by a first preset margin (e.g., 5 pixels) to the maximum ymax of the line by an upward extension by a second preset margin (e.g., 5 pixels). The downward extension (ymin minus the first preset margin) and upward extension (ymax plus the second preset margin) of the horizontal slice can be set according to the needs of different embodiments, for example, the height of each horizontal slice can be dynamically adjusted according to the line spacing (Y1-Y0) in each embodiment.

[0056] The horizontal range covers the entire global text block 100 from the leftmost xmin to the rightmost xmax (e.g., from xmin=9 of the leftmost text unit 101 "Indicator" to xmax=738 of the rightmost text unit 101 "Key Supporting Factor"). This rectangular region is cut out from the input image as a horizontal slice for that row. To increase redundancy verification capability, an overlapping area can be set between adjacent horizontal slices, with an overlap rate preferably between 0.1 and 0.9, for example, 20%. In this case, the vertical lower boundary of the horizontal slice of the i-th row overlaps with the vertical upper boundary of the horizontal slice of the (i+1)-th row by a certain number of pixels, so that text units 101 located near the row boundary appear in both slices simultaneously. For example, if the first data row (Y≈306~329) and the second data row (Y≈339~362) are set to an overlap rate of 20%, the lower boundary of the first data row slice will be extended to about 335 pixels, and the upper boundary of the second data row slice will be extended to about 330 pixels. The two overlap in the area of ​​about Y=330~335, thereby ensuring that the text unit 101 located at the line boundary (e.g., the blank area between "6,210" and "2,850") is not missed.

[0057] S130, Longitudinal section: Please refer to Figure 5 Based on the location coordinate information, the global text block 100 is sliced ​​vertically, dividing it into at least one vertical slice along the horizontal direction. Each vertical slice covers one or more consecutive columns of the global text block 100.

[0058] According to one or more embodiments, during the process of performing vertical slicing on the global text block 100, the slice can be performed on a per-column or multi-column basis according to the horizontal coordinate range of each column of text units 101 within the global text block 100, and there may be overlapping areas between adjacent vertical slices. The horizontal range of each column can be determined by performing cluster analysis on the horizontal coordinates (xmin, xmax) of all text units 101 obtained in step S110.

[0059] According to one or more embodiments, the vertical slice can also be based on the widest text unit 101 in the column as the segmentation reference to ensure the integrity of each text unit 101 in the column is preserved. For example, the widest text unit 101 in the frameless table can be used as the segmentation reference, so that the horizontal range of the vertical slice covers the widest text unit 101. According to one or more embodiments, the width of the vertical slice can also be a fixed value.

[0060] Please continue reading Figure 2 and Figure 3 This step involves cluster analysis of the x-coordinates (xmin and xmax) of all text units 101 to determine the horizontal range of each column. Based on the baseline dataset, several columns can be identified: the first column (indicator column) has xmin ≈ 9–115; the second column (actual 2024) has xmin ≈ 160–234; the third column (2025 forecast) has xmin ≈ 267–340; the fourth column (2026 forecast) has xmin ≈ 381–453; the fifth column (compound growth rate) has xmin ≈ 488–609; and the sixth column (key supporting factors) has xmin ≈ 659–790.

[0061] For each column of text, a vertical slice is generated. According to one or more embodiments, the horizontal range extends downward from the column's minimum xmin by a third preset margin (e.g., 8 pixels) to the column's maximum xmax by a fourth preset margin (e.g., 8 pixels); the vertical range covers the entire global text block 100 from the top ymin to the bottom ymax (e.g., from the header's ymin=6 to the table's special symbol description's ymax=789). An overlap area (e.g., 10%) can also be set between adjacent vertical slices to enhance redundant coverage of text units 101 at column boundaries. For example, the horizontal range of the vertical slice in the second column (actual in 2024) can be extended to approximately 152–242, and the third column to approximately 259–348, with the two overlapping in the X≈259–242 region, ensuring that text units 101 near column boundaries (such as the numbers to the right of "6,210") are covered by multiple vertical slices. According to one or more embodiments, the width of the vertical slice can also be dynamically adjusted according to the character width and the preset number of vertical slices. For example, the character width can be obtained by (X1-X0) / number of text characters, thereby dynamically adjusting the width of the vertically cut image, where (X1-X0) represents the width of the text unit 101.

[0062] S140. Perform text recognition on the horizontal slice and the vertical slice respectively to obtain a first recognition result set and a second recognition result set.

[0063] Specifically, each horizontal slice image and each vertical slice image are fed into one or more text recognition engines (e.g., a CRNN+CTC-based OCR engine, a Transformer-based recognizer or multimodal model, a CNN-based OCR engine, a multimodal model, or a large language model). Each recognition engine outputs all detected text units 101 within that slice, along with their local bounding box coordinates and confidence scores in the slice image. Figure 2 Taking the first horizontal slice of the data row (covering Y≈301~334) as an example, the text unit 101 contained in this slice includes "global semiconductor sales", "6,210", "7,150", "8,050", "+13.9%", "AI computing power demand, automotive electronics penetration". In the first set of recognition results returned by the recognition engine, each result contains a text string, local bounding box coordinates (for example, the local box of "6,210" is (164,5,204,28)) and confidence (for example, 0.999). Similarly, for the vertical slice of the second column (actual 2024) (covering X≈152~242), the text units 101 contained within this slice include the header "actual 2024" (local box (8,0,82,21)), the first data row "6,210" (local box (8,35,48,58)), the second data row "2,850" (local box (8,68,48,91)), etc. These recognition results are collected into a second recognition result set.

[0064] S150. Based on the location coordinate information, the spatial coordinates of the results corresponding to the same text unit 101 in the first recognition result set and the second recognition result set are aligned and compared. If they are consistent, the recognized text of the text unit 101 is output.

[0065] In one or more embodiments, when multiple text recognition engines are used for text recognition, spatial coordinate alignment and comparison are performed independently for each engine. Then, the recognition results from different engines are cross-compared and fused (e.g., using majority voting or weighted voting), and different weights are assigned according to the historical accuracy or current confidence of each engine.

[0066] According to one or more embodiments, please refer to Figure 6 The S150 step can be configured to include the following sub-steps: S151. Obtain the first recognition result set and the second recognition result set. As mentioned above, the first recognition result set contains the local bounding boxes of all text units 101 in the horizontal slice of the first data row and their recognized text; the second recognition result set contains the local bounding boxes of all text units 101 in the vertical slice of the second column and their recognized text.

[0067] S152. For each slice recognition result, its local bounding box coordinates are mapped back to the original image coordinate system through coordinate transformation to obtain the mapped bounding box. Let the original offset of a horizontal slice be (offX_H, offY_H), and the local bounding box of a text unit 101 within the horizontal slice be (lx_min, ly_min, lx_max, ly_max). Then the original coordinates after mapping are (offX_H+lx_min, offY_H+ly_min, offX_H+lx_max, offY_H+ly_max). Taking text unit 101 "6,210" as an example, its local bounding box in the horizontal slice of the first data row is (164,5,204,31). The original offset of this horizontal slice can be determined by the leftmost xmin (taken as 9) of the global text block 100 and the value of the topmost ymin of the row after expanding the margin upwards (taken as 298), for example, the offset is (9,298). After mapping, the original coordinates (173,303,213,329) are obtained, which are completely consistent with the original bounding box (173,303,213,329) of "6,210" in the baseline dataset. Similarly, for the result "6,210" in the second column of vertical slices, its local bounding box is (18,297,58,323), and the vertical slice offset is (155,6). After mapping, the original coordinates (173,303,213,329) are obtained, which are also completely consistent with the original bounding box.

[0068] S153. Calculate the intersection-union ratio (IoU) of the original position coordinates of the mapped bounding box and each text unit 101 in the global text block 100. If the IoU is greater than a preset threshold, then associate the slice recognition result with the text unit 101.

[0069] Continuing with the example of "6,210", its original bounding box is (173,303,213,329). The IoU between the horizontally mapped box (173,303,213,329) and the original box is 1.0; the IoU between the vertically mapped box (173,303,213,329) and the original box is also 1.0. If the preset threshold is 0.5, both are greater than the threshold, therefore, the horizontal and vertical slice recognition results are associated with text unit 101 "6,210". If the mapped box is slightly offset from the original box due to detection or rounding errors, the IoU may still be greater than the threshold (e.g., 0.63 or 0.85), but as long as the preset conditions are met, the association will be successful.

[0070] S154. For each text unit 101, collect all slice recognition results associated with that text unit 101, and classify them into a horizontal matching result subset and a vertical matching result subset according to the slice source.

[0071] Continuing with the example of "6,210", the horizontal matching result subset contains one result from the horizontal slice (text "6,210", confidence 0.999), and the vertical matching result subset contains one result from the vertical slice (text "6,210", confidence 0.999).

[0072] S155. Compare the identified text with the highest confidence in the horizontal matching result subset with the identified text with the highest confidence in the vertical matching result subset (i.e., perform spatial coordinate alignment comparison corresponding to the same text unit 101). S156. If the two are consistent, the recognized text of the text unit 101 is output.

[0073] Continuing with “6,210” as an example, we compare the identified text “6,210” with the highest confidence in the horizontal matching result subset with the identified text “6,210” with the highest confidence in the vertical matching result subset. Since they are the same, we output the identified text “6,210”.

[0074] S157. If the two are inconsistent, the text can be re-identified according to different retry mechanisms.

[0075] For example, suppose the text corresponding to the highest confidence score of the horizontal subset is "62l0", while the text corresponding to the highest confidence score of the vertical subset is "6,210". Since they are different, a retry mechanism can be triggered. The retry mechanism can include one or more of the following: 1. Adjusting the segmentation parameters of the horizontal or vertical slices (such as changing one or more of the extended margin, overlap rate, slice height, slice width, and segmentation start position) and then re-executing the slicing, recognition, and comparison steps; 2. Calling different text recognition engines to re-recognize the original image region or the regenerated slice of the text unit 101, for example, switching from PaddleOCR to a multimodal model, or calling multiple heterogeneous recognition engines for re-recognition; 3. Marking the text unit 101 as awaiting manual review and recording inconsistent horizontal and vertical recognition results. Users can configure the maximum number of retries and the priority of the retry strategy according to business needs.

[0076] Based on this embodiment, the text recognition method of the present invention has the following unexpected technical effects: (i) By independently generating horizontal and vertical slices on the same input image, this invention constructs two orthogonal redundant recognition channels for each text unit. Each text unit simultaneously receives two independent recognitions from its row context and its column context. When the two recognition results are consistent, the reliability of the output is higher than either single recognition; when they are inconsistent, the system automatically detects potential errors, improving recognition accuracy and the system's self-correction capability.

[0077] (ii) The local bounding boxes within the slices are mapped back to the original image coordinate system using coordinate transformation, and the degree of overlap between the mapped boxes and the original text units is quantized using IoU. This ensures that even if there are differences in slice size and position, as long as a sufficient portion of the target text is included, an accurate spatial correspondence can be established.

[0078] (iii) Since an original text unit can be associated with multiple recognition results from multiple slices (such as overlapping slices and multi-scale slices), the present invention can collect and organize these results to form horizontal matching subsets and vertical matching subsets, providing a rich data foundation for subsequent confidence comparison, retry mechanism, etc.

[0079] (iv) After obtaining subsets of horizontal and vertical matching results, the identified text with the highest confidence in each direction is compared. The result is only output when the best evidence in both directions matches; otherwise, a retry or manual review is triggered. This mechanism avoids misjudgment caused by high confidence errors in a single direction, achieving true cross-validation.

[0080] (v) An overlapping area is set between adjacent slices so that text units located near row or column boundaries can appear in multiple slices simultaneously. When a slice is misidentified due to local stains, creases, reflections, or other degradation, other overlapping slices may still provide correct identification results, which can be corrected through subsequent comparison and fusion.

[0081] In summary, this invention constructs a complete technology chain from coordinate acquisition, horizontal and vertical slicing, spatial association to cross-validation, which improves the accuracy, robustness and adaptability of text recognition, and effectively solves the long-standing problems in existing technologies such as lack of verification in single recognition, spatial redundancy, and inaccurate association.

[0082] Please see Figure 7 , Figure 7 A flowchart of a text recognition method according to some embodiments of the present invention is shown as an example. In the illustrated embodiments, the text recognition method includes the following steps: S210. Obtain the input image.

[0083] The input image can be a digital photograph captured by a camera module integrated into an electronic device, such as a handheld electronic device, a document image obtained from a scanner, or a page stored in Portable Document Format (PDF). The input image can be acquired via a wired link (e.g., USB, Ethernet) or a wireless link (e.g., Wi-Fi, Bluetooth). The input image can be in any known raster image format, including but not limited to JPEG, PNG, BMP, TIFF, or RAW formats.

[0084] S220. Perform at least one preprocessing step on the input image to generate at least one derived image.

[0085] The types of preprocessing operations include, but are not limited to: grayscale conversion (e.g., converting an RGB image to a single-channel luminance image), histogram equalization (e.g., contrast-limited adaptive histogram equalization, CLAHE), adaptive threshold binarization (e.g., local thresholding algorithms using Gaussian windows), morphological operations (e.g., opening operations to remove small noise points, closing operations to fill holes inside characters, and top-hat transformations to correct uneven lighting), super-resolution reconstruction (e.g., magnification algorithms based on ESPCN or SRGAN), and denoising filtering (e.g., Gaussian filtering, median filtering, or bilateral filtering). It should be understood that the above preprocessing operations can be performed individually or in any combination, either in series or in parallel.

[0086] In an exemplary embodiment, grayscale processing, denoising processing, and resolution enhancement processing are performed on the input image to generate a first derived image corresponding to grayscale processing, a second derived image corresponding to denoising processing, and a third derived image corresponding to resolution enhancement processing. For example, when the input image is color and contains Gaussian noise and low resolution, grayscale processing is used to eliminate interference from color information, denoising processing is used to smooth noisy pixels, and super-resolution reconstruction is used to restore the sharpness of character edges.

[0087] Those skilled in the art should understand that although the description uses three preprocessing operations and the output of three derived images as an example, the scope of protection of this invention is not limited thereto. Depending on different application scenarios or recognition requirements, one, two, four, or more preprocessing operations can be performed on the input image to generate a corresponding number of derived images, and these variations should all fall within the scope of protection of this invention.

[0088] S230. Obtain the position coordinate information of the global text block 100 in the input image and the derived image.

[0089] Based on this embodiment, the positional coordinate information of the global text block 100 in the aforementioned input image (original image), first derived image, second derived image, and third derived image is obtained. This positional coordinate information can be obtained using any technique known in the art. For example, a text detection network based on EAST or DBnet can be used to infer for each image, outputting the bounding box coordinates (typically represented by the coordinates of the upper left and lower right corners) of each text unit 101, along with its recognized text and confidence score. Alternatively, methods similar to those described in the foregoing embodiment can be used, which will not be repeated here. Through this step, each image (including the original image and derived images) is converted into a structured baseline data set containing the spatial locations and initial recognition results of all text units 101 in the image.

[0090] S240, Horizontal Slicing: Based on the position coordinate information corresponding to each image, horizontal slicing is performed on the global text block 100 of each image.

[0091] Based on this embodiment, horizontal slicing is performed on each of the global text blocks 100 based on the position coordinate information of the global text blocks 100 in the input image, the first derived image, the second derived image, and the third derived image, respectively. The specific implementation of horizontal slicing can employ the same or similar methods as in the aforementioned embodiment. For example, by performing cluster analysis on the ordinates (ymin and ymax) of all text units 101 in each image, the vertical range of each row is determined, and then horizontal segmentation is performed on a row-by-row basis to generate horizontal slices. These horizontal slices are stored independently for subsequent text recognition.

[0092] S250, Vertical Slicing: Based on the position coordinate information corresponding to each image, vertical slicing is performed on the global text block 100 of each image.

[0093] Based on this embodiment, vertical slicing is performed on each of the global text blocks 100 based on the position coordinate information of the global text blocks 100 in the input image, the first derived image, the second derived image, and the third derived image, respectively. The specific implementation of vertical slicing can employ the same or similar methods as in the aforementioned embodiment. For example, by performing cluster analysis on the horizontal coordinates (xmin and xmax) of all text units 101 in each image, the horizontal range of each column is determined, and then vertical segmentation is performed on each column to generate vertical slices. These vertical slices are stored independently for subsequent text recognition. This will not be described again here.

[0094] S260. Corresponding to the horizontal and vertical slices of each image, perform text recognition on the horizontal and vertical slices respectively to obtain the first recognition result set and the second recognition result set corresponding to each image.

[0095] Based on this embodiment, according to the horizontal and vertical slices of the input image, the first derived image, the second derived image, and the third derived image, corresponding first and second recognition result sets are generated respectively. Specifically, for the input image, all its horizontal slices are fed into one or more OCR engines to generate the first recognition result set of the input image; all its vertical slices are fed into the same or different OCR engines to generate the second recognition result set of the input image. Similarly, the above process is repeated for the first derived image, the second derived image, and the third derived image to generate their respective first and second recognition result sets. The specific implementation of text recognition can adopt the same or similar methods as the aforementioned embodiment, and will not be described again here. Each entry in each recognition result set contains at least: the recognized text string, the local bounding box coordinates of the string in the sliced ​​image (usually defined by the four corner points or extreme value coordinates), and a confidence score (e.g., a floating-point number between 0 and 1) representing the confidence level.

[0096] S270. Based on the location coordinate information, the spatial coordinates of the first recognition result set corresponding to each image and the results corresponding to the same text unit 101 in the second recognition result set are aligned and compared. If they are consistent, the recognized text of the text unit 101 is output.

[0097] Based on this embodiment, the following operations are performed based on the positional coordinate information of the input image, the first derived image, the second derived image, and the third derived image, as well as the first recognition result set and the second recognition result set: For the input image, spatial coordinate alignment and comparison are performed on the results corresponding to the same text unit 101 in the first recognition result set and the second recognition result set. If they match, the recognized text of the text unit 101 is output. Similarly, the same spatial coordinate alignment and comparison are performed on the first derived image, the second derived image, and the third derived image, and the recognized text is output for each. The specific implementation of spatial coordinate alignment and comparison can adopt the same or similar methods as those in the aforementioned embodiment. For example, for each image, the local bounding box of each slice recognition result is mapped back to the original coordinate system of the image through coordinate transformation, and then the IoU between the mapped bounding box and the bounding box of each original text unit 101 in the image is calculated. Slice results with IoU greater than a preset threshold (e.g., 0.5) are associated with the corresponding text unit 101. Subsequently, all associated results of each text unit 101 are collected and divided into a horizontal matching result subset and a vertical matching result subset according to the slice source. The text with the highest confidence in the horizontal subset is compared with the text with the highest confidence in the vertical subset. If they are the same, the text is output.

[0098] In some embodiments, if the spatial coordinate alignment of an image is inconsistent with the comparison result, a retry mechanism for that image can be triggered. For example, suppose that the highest confidence score of the horizontal subset of a text unit 101 in the first derived image is "6210" (incorrect), while the highest confidence score of the vertical subset is "6,210" (correct), then an inconsistency is detected. At this time, according to the preset retry strategy, one or more of the following operations can be performed: adjust the segmentation parameters of the horizontal or vertical slices (e.g., increase the extended margin, change the overlap rate) and then re-perform the slicing, recognition, and comparison steps for the derived image; or call a text recognition engine different from the one used in the initial recognition (e.g., switch from PaddleOCR to the LayoutLMv3 multimodal model) to re-recognize the corresponding original image region or the regenerated slice in the derived image; or mark the text unit 101 as awaiting manual review and record the inconsistent horizontal and vertical recognition results for subsequent analysis. If the image still fails to output a consistent result after reaching the preset maximum number of retries (e.g., 2 or 3 times), the output data of the derived image can be discarded or its fusion weight reduced, depending on the specific application scenario.

[0099] S280. The recognition results of multiple images are fused to output the final recognized text. The fusion can be performed using one or more methods from weighted voting, majority voting, Bayesian fusion, or DS evidence theory.

[0100] In an exemplary embodiment, for each text unit 101, after obtaining the respective recognition results from multiple images (including the original input image and derived images), weighted voting fusion is performed. Specifically, a basic weight value is pre-assigned to each image, and a recognition threshold is set. Simultaneously, an output inconsistency weight is configured for each image, which is lower than its basic weight value. When an image is determined to have the highest confidence recognition text in the horizontal matching result subset and the vertical matching result subset during the spatial coordinate alignment and comparison step (i.e., step S270), the image participates in fusion with its basic weight value; conversely, if the comparison results within the image are inconsistent, the image participates in fusion with its output inconsistency weight (or, the image does not output any recognition text, and its weight is set to zero). For each candidate recognition text, the corresponding weights of all images that output the text are summed to obtain the weighted total score of the text. The text with the highest weighted total score is selected as the candidate text. If the weighted total score of the candidate text is greater than or equal to the recognition threshold, it is output as the final recognition text.

[0101] The following explanation uses four images as an example. Assume there are an input image, a first derived image, a second derived image, and a third derived image. For simplicity, assume each image has a base weight of 0.3, an output inconsistency weight of 0.1, and a recognition threshold of 0.65. For a specific text unit 101 (e.g., the number "6,210"), after independently executing steps S210 to S270 on each image, the following results are obtained: The input image has consistent horizontal and vertical comparisons, and the output recognized text "6,210" is used in the fusion with a base weight of 0.3. The first derived image also has consistent horizontal and vertical comparisons, and the output recognized text "6,210" is used in the fusion with a base weight of 0.3. The second derived image also has consistent horizontal and vertical comparisons, and the output recognized text "6,210" is used in the fusion with a base weight of 0.3. The third derived image has inconsistent horizontal and vertical comparisons (e.g., horizontal output "6,210", vertical output "6,210"). According to the preset strategy, the third derived image is used in the fusion with its inconsistent output weight of 0.1, and the output recognized text uses the text "6,210" with higher confidence in its vertical matching result subset (or its horizontal result, depending on the configuration; in this example, the vertical result "6,210" is used).

[0102] At this point, the weighted total score of the candidate text "6,210" is 0.3 + 0.3 + 0.3 + 0.1 = 1.0, which is greater than the threshold of 0.65. Therefore, the final output is "6,210". If one of the input image, the first derived image, or the second derived image is internally inconsistent and uses a low weight, the total score may still reach the threshold. Conversely, if internal inconsistencies in most images cause the low weights to accumulate and fall below the threshold, manual review or a backup strategy will be triggered.

[0103] In another embodiment, the weight of each image can be dynamically determined solely based on its internal horizontal and vertical comparison results, without pre-assigning basic weights. For example, images with internal consistency receive a weight of 1.0, while images with internal inconsistency receive a weight of 0.2. During fusion, the weights of images that output the same text are summed, and the text with the highest score exceeding a threshold is output. According to this scheme, when an image fails to output consistent results in its internal spatial coordinate alignment and comparison steps, the weight assigned to that image in subsequent fusion processes is reduced accordingly, thereby reducing the interference of unreliable recognition results on the final output decision. In other words, images with higher internal consistency have a larger weight in the weighted voting, while the contribution of images with lower internal consistency to the fusion result is selectively suppressed, thereby improving the credibility of the final recognized text. Those skilled in the art should understand that the weighted fusion processing rules should not be limited to the rules described in the above examples, and can be adaptively varied under different scenarios or needs.

[0104] In another exemplary embodiment, a majority voting mechanism can be used for fusion. Specifically, the text with the highest frequency in the output recognition text of all images (including input images and derived images) is counted. If its frequency exceeds half of the total number of images (e.g., it appears at least 3 times in 4 images), then that text is used as the final output; otherwise, manual review is triggered or a weighted voting method is switched to re-determine the result. It should be noted that, in addition to the weighted voting and majority voting methods described above, this invention can also employ other known fusion methods, such as Bayesian fusion or DS evidence theory. Since these methods are mature in the field and their specific implementation details do not constitute essential technical features of this invention, they will not be described in detail here to maintain the brevity of the specification. The above fusion methods can be used individually or in combination according to actual needs. For example, majority voting can be performed first, and if a majority consensus cannot be reached, weighted voting can be switched, or Bayesian fusion can be further used for refined decision-making. Based on the disclosure of this invention, those skilled in the art can apply the above fusion methods to the recognition result fusion step of this invention without creative effort.

[0105] Based on this embodiment, the text recognition method of the present invention has the following unexpected technical effects: (i) Through the multi-preprocessing fusion scheme described in S210 to S280, the present invention can improve the robustness of recognition for low-quality and various degradation types of input images. Different preprocessing methods are used to complementarily enhance the same original input image. By independently executing the complete recognition process (including horizontal slicing, vertical slicing, coordinate mapping, and horizontal and vertical cross-validation) on the original image and its derived images generated by different preprocessing, and then fusing the multi-path recognition results, a high-accuracy recognition result can be adaptively obtained.

[0106] (II) The aforementioned multi-preprocessing fusion scheme is fully compatible with and can be seamlessly integrated with the cross-validation mechanism for horizontal and vertical slicing described in this invention. Since each derived image independently performs position coordinate acquisition, horizontal slicing, vertical slicing, coordinate mapping, and horizontal / vertical comparison, the recognized text output by each image has already undergone double verification of rows and columns. When the recognition results of multiple images are fused, the error patterns of each image have low correlation, further improving the reliability of the fused output. When the horizontal / vertical comparisons within one or more images are inconsistent, they are given lower weights or effectively suppressed during the fusion process, thereby reducing the negative impact of unreliable results on the final decision. Therefore, this invention constructs a complete technology chain from image enhancement, redundancy (horizontal / vertical orthogonal redundancy, overlapping slice redundancy, multi-preprocessing redundancy, multi-engine redundancy) information generation, spatial precise association, cross-validation to multi-source fusion, greatly improving recognition accuracy, robustness, and adaptability.

[0107] Please see Figure 8 , Figure 8 A flowchart of a text recognition method according to some embodiments of the present invention is shown as an example. In the illustrated embodiments, the text recognition method includes the following steps: S310. Obtain the position coordinate information of the global text block 100 in the input image, wherein the global text block 100 contains at least one text unit 101.

[0108] S320. Based on the position coordinate information, the text units 101 in the global text block 100 are tilted so that each line of text unit 101 is independently rotated to a horizontal state.

[0109] Please see Figure 9 , Figure 9 A flowchart of a text skew correction step is illustrated exemplarily in some embodiments of the present invention. In one or more embodiments, the text skew correction step may include the following sub-steps: S321. Obtain the coordinates of the bounding box corners of the text unit 101 in each line of the global text block 100, wherein each bounding box is defined by the upper left corner (X0,Y0), the upper right corner (X1,Y0), the lower right corner (X1,Y1), and the lower left corner (X0,Y1).

[0110] by Figure 2 Taking the header row as an example, this row contains only one text unit 101, "Semiconductor Industry Technology Evolution and Market Landscape (2025-2030)". The coordinates of its bounding box corners can be read from the baseline dataset as: top left (39,6), top right (393,7), bottom right (393,25), and bottom left (39,24). Since this row may actually have a slight tilt (e.g., top right Y=7, top left Y=6, indicating a slight downward tilt to the right), its tilt angle can be calculated in subsequent steps. Taking the first data row in the table as another example, this row contains multiple text units 101, such as "Global Semiconductor Sales" (bounding box (11,306,115,323)) and "6,210" (bounding box (173,303,213,329)). Each text unit 101 provides its four corner coordinates.

[0111] S322. For each text unit 101 in the line, calculate at least one direction vector based on its bounding box corner points, the direction vector including a horizontal direction vector (X1→X0, Y0→Y0) and / or a vertical direction vector (X0→X0, Y1→Y0).

[0112] In practical calculations, a horizontal vector is typically used to estimate the slant angle of the text line, as this vector reflects the direction of the top edge of text unit 101. For the header text unit 101, its horizontal vector is calculated as (393-39,0)=(354,0). However, since the Y-coordinate of the top right corner is 7 and the Y-coordinate of the top left corner is 6, the top edge is not perfectly horizontal. To capture the slant, the vector connecting the top right and top left corners can be used: (X1-X0,Y1_top-Y0_top)=(354,7-6)=(354,1). Similarly, for text unit 101, which displays "Global Semiconductor Sales," the top left corner (11,306), top right corner (115,306), and the Y-axis of the top right corner being equal to that of the top left corner indicate that the top edge of text unit 101 is horizontal. However, other text units 101 in the same line (such as the corner point (173,303,213,329) of "6,210") may have slight offsets. By combining the vectors of multiple text units 101 within the line, the overall tilt of the line can be determined.

[0113] S323. Assign a weight value to each directional vector based on at least one of the following factors: the spacing between adjacent points of the text unit 101, the character size, and the recognition confidence level.

[0114] For example, text units 101 with larger character heights (Y1-Y0) generally have higher positioning accuracy and can therefore be assigned higher weights. In the header row, the character height is approximately 19 pixels, so a weight of 1.0 can be set. In the table data rows, the character height for "Global Semiconductor Sales" is 17 pixels, while the character height for "6,210" is 26 pixels (due to the vertical splay of the numbers), and the latter can be assigned a higher weight, such as 1.2. Furthermore, text units 101 with higher recognition confidence are also assigned higher weights; for example, "6,210" has a confidence level of approximately 0.999, and an additional weighting factor can be added. Text units 101 with uniform spacing between adjacent points (i.e., the average spacing between characters within a text unit 101) are considered more reliable, and their weights are increased accordingly. This weighting strategy reduces the impact of individual character recognition errors or bounding box positioning deviations on tilt estimation.

[0115] S324. Calculate the weighted average of the direction vectors of all text units 101 in the line to obtain the overall tilt angle of the line.

[0116] by Figure 2The header row and the first data row in the table are used as examples. The header row contains only one text unit 101, "Semiconductor Industry Technology Evolution and Market Landscape (2025-2030)". Its bounding box corners are read from the baseline data: top left (39,6), top right (393,7), bottom right (393,25), and bottom left (39,24). The upper edge of this text unit 101 is determined by the top left and top right corners, with a direction vector of (393-39,7-6)=(354,1). Since this row contains only one text unit 101, its weight is set to 1.0, and the weighted average direction vector is (354,1). Therefore, the tilt angle θ = arctan(1 / 354) ≈ 0.162 degrees is calculated. This slight tilt (approximately 0.16 degrees) may be a minor deviation during the scanning or detection process and can be eliminated through subsequent rotation correction.

[0117] The first data row contains six text units 101, from left to right: “Global Semiconductor Sales”, “6,210”, “7,150”, “8,050”, “+13.9%”, “AI Computing Power Demand”, “Automotive Electronics Penetration”. The bounding box corners of each text unit 101 are extracted from the baseline data, and their horizontal vectors (top edges) and weight assignments are shown in the table below:

[0118] Table 1 In Table 1, the bounding box of “6,210” shows a vertical offset of 23 pixels (ΔY=23) at its top edge due to differences in the height of the numeric characters or detection errors. This offset deviates significantly from the expected horizontal position. To avoid excessive influence of this outlier on the overall tilt estimation, its weight was reduced to 0.2; the top edges of the remaining text units 101 are basically horizontal, and their weights are all set to 1.0.

[0119] Weighted sum of X components = 104 × 1.0 + 40 × 0.2 + 36 × 1.0 + 36 × 1.0 + 46 × 1.0 + 160 × 1.0 = 104 + 8 + 36 + 36 + 46 + 160 = 390 Weighted sum of Y components = 0 × 1.0 + 23 × 0.2 + 0 + 0 + 0 + 0 = 4.6 Total weights = 1.0 + 0.2 + 1.0 + 1.0 + 1.0 + 1.0 = 5.2 Average direction vector = (390 / 5.2, 4.6 / 5.2) ≈ (75.0, 0.8846) Therefore, the overall tilt angle of the row is calculated as θ = arctan(0.8846 / 75.0) ≈ arctan(0.01179) ≈ 0.676 degrees. This angle reflects the slight tilt trend of the first data row as a whole, with long text units 101 (such as "Global Semiconductor Sales" and "AI Computing Power Demand, Automotive Electronics Penetration") providing the main horizontal reference, while the influence of the abnormal text unit 101 "6,210" is effectively suppressed.

[0120] S325. Based on the overall tilt angle of the line, rotate and correct the text unit 101 of the line so that the text of the line remains horizontal.

[0121] For example, for the header row, if the calculated tilt angle is 0.162 degrees, the system rotates the image strip within the Y range [1,30] by -0.162 degrees, using the center of that row area as the rotation center. After rotation, the top edge of the header row becomes horizontal. Similarly, the first data row of the table is rotated by 0.676 degrees, aligning the top edges of all text units 101 in that row to the same horizontal line. After line-by-line correction, each line of text in the entire image is independently corrected to a horizontal position, thus eliminating distortion caused by page curvature or local tilt. Similarly, the other rows are calculated and corrected independently. Through line-by-line processing, even if the image has page curvature or local tilt distortion, each line of text can be accurately corrected to a horizontal position, thus providing a high-quality alignment basis for subsequent horizontal slicing, vertical slicing, and spatial coordinate alignment and comparison.

[0122] S330. Perform a horizontal slice on the global text block 100, dividing it into at least one horizontal slice along the vertical direction. Each horizontal slice covers one or more consecutive lines in the global text block 100.

[0123] S340. Perform vertical slicing on the global text block 100, dividing it into at least one vertical slice along the horizontal direction. Each vertical slice covers one or more consecutive columns of the global text block 100.

[0124] S350. Perform text recognition on the horizontal slice and the vertical slice respectively to obtain a first recognition result set and a second recognition result set.

[0125] S360. Based on the location coordinate information, the spatial coordinates of the results corresponding to the same text unit 101 in the first recognition result set and the second recognition result set are aligned and compared. If they are consistent, the recognized text of the text unit 101 is output.

[0126] It should be noted that the method principles described in S310, S330 to S360 above are the same as or basically similar to the method principles described in S110 to S150 or S210 to S280 in the aforementioned embodiments, and therefore will not be repeated here. Those skilled in the art, based on the disclosure of this invention, can directly apply the details of horizontal slicing, vertical slicing, coordinate mapping, IoU calculation, and horizontal-vertical comparison in the aforementioned embodiments to this embodiment. Based on this embodiment, the text recognition method of this invention has the following unexpected technical effects: (i) The line-by-line tilt correction scheme can effectively handle common non-linear distortions during document scanning or photography. For example, when scanning thick books, the text lines in the middle of the page and the text lines at the edges often have different tilt angles; on folded documents or uneven paper, the tilt angles between lines may vary gradually or abruptly. By independently calculating the tilt angle for each line and performing rotation correction separately, this invention can accurately correct each line of text to a horizontal state.

[0127] (ii) The line-by-line correction method effectively improves the alignment accuracy of subsequent horizontal and vertical slices. After line-by-line correction, each line of text is basically horizontal, so the vertical range of the horizontal slice can be set more compactly, reducing the problem of inter-line character overlap or inaccurate cutting caused by tilting. At the same time, the vertical slice (cut by column) can also be more accurately aligned to different lines of text in the same column on the corrected image, thereby improving the success rate of IoU calculation when aligning spatial coordinates.

[0128] (III) The weighted averaging mechanism used in the tilt correction step further enhances the robustness of the tilt angle estimation. Specifically, by assigning higher weights to text units 101 with larger character sizes, higher recognition confidence, or uniform spacing between adjacent points, the system can effectively suppress the negative impact of small characters, low confidence, or abnormally formatted text units 101 on the tilt angle estimation. Even if there are a few character recognition errors or bounding box positioning deviations in a certain line, the weighted averaging can still ensure the accurate estimation of the overall tilt angle, thereby avoiding correction failures caused by individual outliers.

[0129] (iv) The tilt correction step is fully compatible with the cross-validation mechanism for horizontal and vertical slices and can be seamlessly integrated. Since tilt correction occurs before slicing, the corrected image provides a more regular input for subsequent horizontal and vertical slices, enabling row and column slices to more accurately correspond to the row and column structure of the original text. As a result, the accuracy and robustness of the entire recognition system are substantially improved.

[0130] In summary, this embodiment solves the problem that global rotation cannot handle nonlinear distortion in the prior art by introducing row-by-row tilt correction before performing horizontal and vertical slicing and using a weighted average mechanism to calculate the tilt angle of each row. This provides a high-quality alignment foundation for subsequent horizontal and vertical cross-validation and further improves the recognition accuracy, robustness and automation level of the entire text recognition method.

[0131] Finally, the present invention also discloses a text recognition system, including one or more processors and a memory. The memory is coupled to the processor and stores instructions. When the instructions are executed by the processor, the system performs the text recognition method described in any of the above embodiments. The processor may be one or more of a central processing unit (CPU), graphics processing unit (GPU), tensor processing unit (TPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), or field-programmable gate array (FPGA). The memory may be volatile memory (e.g., dynamic random access memory DRAM) or non-volatile memory (e.g., flash memory, read-only memory ROM).

[0132] The present invention also discloses a storage medium. This storage medium may be a solid-state drive (SSD), a hard disk drive (HDD), an optical disc (CD / DVD / Blu-ray), a universal serial bus (USB) flash drive, a secure digital card (SD card), or any other physical carrier capable of storing data in a non-transitory manner. A computer program is stored on the storage medium, and when the program is loaded and executed by a processor, it implements the text recognition method described in any of the above embodiments.

[0133] This invention also discloses a computer, including but not limited to mobile phones, tablets, desktop computers, laptops, servers, industrial control computers, or embedded devices. The computer internally includes a processor, a memory connected to the processor, a display screen, and a camera. The computer may also be equipped with external devices such as speakers, microphones, keyboards, mice, touchpads, and network interfaces as needed. The memory can be used to store program code and data, and the processor is used to run programs stored in the memory to perform various computer functions and data processing. For example, by running the corresponding computer program stored in the memory, the text recognition method described above can be executed, thereby achieving automated information extraction from document images.

[0134] The above embodiments merely illustrate preferred implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention should be determined by the appended claims.

Claims

1. A text recognition method, characterized in that, Includes the following steps: Obtain the position coordinate information of a global text block in the input image, wherein the global text block contains at least one text unit; Based on the location coordinate information, the global text block is horizontally sliced, dividing it into at least one horizontal slice along the vertical direction. Each horizontal slice covers one or more consecutive lines in the global text block. Based on the location coordinate information, the global text block is sliced ​​vertically to divide it into at least one vertical slice along the horizontal direction. Each vertical slice covers one or more consecutive columns of the global text block. Text recognition is performed on the horizontal slices and vertical slices respectively to obtain a first recognition result set and a second recognition result set; Based on the location coordinate information, the spatial coordinates of the results corresponding to the same text unit in the first recognition result set and the second recognition result set are aligned and compared. If they are consistent, the recognized text of the text unit is output.

2. The text recognition method as described in claim 1, characterized in that: During the horizontal slicing of the global text block, the slice is performed on a per-line or consecutive-line basis, based on the vertical coordinate range of each line of text units within the global text block. Adjacent horizontal slices may have overlapping areas; and / or During the process of performing vertical slicing on the global text block, the slice is performed in units of each column or multiple consecutive columns, based on the horizontal coordinate range of each column of text units within the global text block. There may be overlapping areas between adjacent vertical slices.

3. The text recognition method according to claim 1, characterized in that, The spatial coordinate alignment and comparison step includes the following sub-steps: Obtain the first recognition result set and the second recognition result set, wherein the first recognition result set and the second recognition result set contain the text units recognized in each slice and their local bounding box coordinates in the slice image; The local bounding box coordinates of each text unit in each slice recognition result are mapped back to the original image coordinate system through coordinate transformation to obtain the mapped bounding box. Calculate the intersection-union ratio (IUU) of the mapped bounding box with the original position coordinates of each text unit within the global text block. If the IUU is greater than a preset threshold, then associate the slice recognition result with that text unit. Collect all slice recognition results associated with each text unit, and divide them into a horizontal matching result subset and a vertical matching result subset according to the slice source; The text with the highest confidence in the horizontal matching result subset is compared with the text with the highest confidence in the vertical matching result subset. If they are the same, the text with the highest confidence in the vertical matching result subset is output.

4. The text recognition method as described in claim 1, characterized in that: In the spatial coordinate alignment and comparison step, if the results of the spatial coordinate alignment and comparison are inconsistent, a retry mechanism is triggered; the retry mechanism includes any one or more of the following: After adjusting the segmentation parameters of the horizontal or vertical slices, the slicing, identification, and comparison steps are re-executed; wherein, the segmentation parameters include at least one of slice height, slice width, overlap rate, segmentation start position, and extension margin; Different text recognition engines are invoked to re-recognize the original image region or the regenerated slice of the text unit; The text units are marked as requiring manual review, and inconsistent horizontal and vertical recognition results are recorded.

5. The text recognition method as described in claim 1, characterized in that, It also includes the following steps: Perform at least one preprocessing step on the input image to generate at least one derived image; The steps of obtaining position coordinate information, horizontal slicing, vertical slicing, text recognition, and spatial coordinate alignment and comparison are performed on the input image and the derived image respectively to obtain the recognition result of the same text unit for each image; The recognition results of multiple images are fused together to output the final recognized text.

6. The text recognition method as described in claim 5, characterized in that: The types of preprocessing include grayscale conversion, histogram equalization, adaptive threshold binarization, morphological operations, super-resolution reconstruction, and denoising filtering.

7. The text recognition method as described in claim 1, characterized in that, The text recognition uses at least two heterogeneous recognition engines, and the recognition results output by each engine are cross-compared and fused; the heterogeneous recognition engines are selected from at least two of the following: CRNN-based OCR engine, Transformer-based OCR engine, convolutional neural network-based OCR engine, multimodal model, and large language model.

8. The text recognition method as described in claim 1, characterized in that, Before performing horizontal and vertical slicing, a text skew correction step is also included: Obtain the coordinates of the bounding box corners of each text unit in each line of the global text block, wherein the bounding box of each text unit is defined by the top left corner (X0,Y0), the top right corner (X1,Y0), the bottom right corner (X1,Y1), and the bottom left corner (X0,Y1). At least one direction vector is calculated based on the bounding box corner points of each text unit in the line, the direction vector including a horizontal direction vector (X1→X0, Y0→Y0) and / or a vertical direction vector (X0→X0, Y1→Y0); Each directional vector is assigned a weight value based on at least one of the following factors: the spacing between adjacent points of the text unit, the character size, and the recognition confidence. The overall tilt angle of the line is obtained by weighting the direction vectors of all text units in the line according to their weights. Based on the overall tilt angle of the line, the text units in that line are rotated and corrected to keep the text horizontal.

9. A text recognition system, characterized in that, include: One or more processors; A memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform the method of any one of claims 1 to 8.

10. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method of any one of claims 1 to 8.

Images