Method and system for text detection and recognition in complex layout images

Abstract

This invention relates to a method and system for text detection and recognition in complex layout images. The method includes: performing text detection on a high-resolution original image and generating multiple candidate text boxes containing text; post-processing the multiple candidate text boxes based on the layout structure characteristics of the text to obtain multiple effective text regions; cropping the original image according to the effective text regions to obtain text sub-images corresponding to the effective text regions; and performing text recognition on each of the multiple text sub-images to obtain text recognition results. The method and system of this invention improve the accuracy, stability, and efficiency of text recognition in complex layout image scenarios.

Description

Specification 1 A Method and System for Text Detection and Recognition of Complex Layout Images Technical Field This invention relates to the fields of computer vision and text recognition technology; specifically, it relates to a method and system for text detection and recognition of complex layout images. Background Art With the development of computer vision and text recognition technologies such as OCR technology, existing text recognition systems typically employ a processing method of detecting and directly recognizing text from the entire image. However, in practical applications, especially in scenarios involving complex layout images such as advertisements, packaging, and marketing posters, the original entire image often has characteristics such as complex layout, dense text, varying font sizes, multi-line arrangement, and the inclusion of footnotes, superscripts, or multi-column structures. Under such conditions of high image resolution and text density, the effective feature scale of the text in the image is relatively reduced. In this case, the direct recognition processing method is prone to problems such as text not being recognized, repeated detection of the same text area, text blocks being split into multiple small areas, or the recognition area containing a large amount of non-target background, thus leading to reduced recognition accuracy and hindering subsequent structured processing and utilization of the text content. Furthermore, the recognition performance of existing text recognition models primarily depends on the effective character height of the input text, such as requiring the text pixels to be greater than 15px, rather than on the original resolution of the input image. Therefore, when a high-resolution original image is input into a text recognition model, due to limitations on the model's maximum resolution or maximum input side length, normalization processing is required. This results in the high-resolution original image being proportionally reduced, causing the pixels of characters or symbols that would normally be recognizable by the human eye at their original size to be lower than the effective character height required by the model, thus making them unrecognizable or incorrectly recognized. In existing technologies, for high-resolution images containing complex text, a sliding window cropping method is typically used to evenly divide the original image into smaller sub-images before text recognition. However, in image scenarios with diverse layout features, this approach easily segments related text within the same paragraph, or even individual characters, into different sub-images, leading to problems such as lost or misidentified text, incomplete information extraction, and severely impacting the subsequent comprehensive and accurate understanding and processing of the text content in the image. In view of the above, the present invention provides a method and system for text detection and recognition of complex layout images, thereby solving or at least alleviating one or more of the above-mentioned problems and other problems existing in the prior art.To achieve the aforementioned objective, a first aspect of the present invention provides a method for text detection and recognition of complex layout images. The method includes: Step 1: Performing text detection on a high-resolution original image and generating multiple candidate text boxes containing text; Step 2: Post-processing the multiple candidate text boxes based on the layout structure characteristics of the text to obtain multiple effective text regions; Step 3: Cropping the original image according to the effective text regions to obtain text sub-images corresponding to the effective text regions; Step 4: Performing text recognition on each of the multiple text sub-images to obtain text recognition results, wherein Step 2 includes, but is not limited to: Step 2-1: Performing non-maximum suppression processing on the candidate text boxes to eliminate overlapping duplicate text boxes; Step 2-2: Differentiating the candidate text boxes into superscript text boxes and regular text boxes, and merging the superscript text boxes with the nearest regular text box; Step 2-3: Determining the paragraph relationship between the multiple candidate text boxes, and merging candidate text boxes belonging to the same paragraph region; Step 2-4: Determining the nesting relationship between the multiple candidate text boxes, and merging candidate text boxes nested in the same paragraph region. 30134896 A Instruction Manual 3 Delete candidate text boxes inside other candidate text boxes; Step 2-5: Expand the borders of the candidate text boxes. In the text detection and recognition method described above, optionally, in step 2-2: Set area threshold ratio coefficient βA, height threshold βh, and width threshold βw. When the area A of the candidate text box satisfies A ≤ βA × W × H, and the height h and width w of the candidate text box satisfy h ≤ βh, w ≤ βw, the candidate text box is determined to be a subscript text box s, and other candidate text boxes that do not meet the conditions are determined to be text text boxes m, where W is the total width of the original image and H is the total height of the original image.In the text detection and recognition method described above, optionally, in step 2-2: the positional relationship between the superscript text box s and the text box m is determined. When the following conditions are met: Sx1 ≥ mx2 − γ × (mx2 − mx1) Sx1 ≤ mx2 + βx ×W Sy2 ≥ my1 − βy × H Sy1 ≤ my2 + δ × (my2 − my1), the text box m is determined to be the related text box of the superscript text box s. Where Sx1, Sy1, and Sy2 are the left boundary x-coordinate, upper boundary y-coordinate, and lower boundary y-coordinate of the subscript text box s, respectively; mx1, mx2, my1, and my2 are the left boundary x-coordinate, right boundary x-coordinate, upper boundary y-coordinate, and lower boundary y-coordinate of the text box m, respectively; γ is the horizontal position tolerance coefficient; βx is the horizontal expansion tolerance coefficient; βy is the upper vertical tolerance coefficient; δ is the lower vertical tolerance coefficient; W is the total width of the original image; and H is the total height of the original image. In the relevant text boxes, the text box m* closest to the subscript text box s is selected, satisfying m* = argminCost(s,m). Cost(s,m) = max(0, Sx1 −mx2) + |Sy2 −my1|. The subscript text box s is then merged with the closest text box m*. The merged new text box is the bounding rectangle of the superscript text box s and the nearest text box m∗. In the text detection and recognition method described above, optionally, in steps 2-3: a horizontal overlap threshold θx and a line spacing threshold coefficient θy are set. When the horizontal overlap ratio Rx and line spacing V between two candidate text boxes satisfy Rx ≥ θx and V ≤ θy × H, it is determined that the two candidate text boxes belong to the same paragraph region, where Rx = xoverlap xmin, xoverlap is the horizontal overlap width of the two candidate text boxes, xmin is the total width of the narrower text box in the two candidate text boxes, and H is the total height of the original image.In the text detection and recognition method described above, optionally, in steps 2-3: when the candidate text box C and multiple candidate text boxes B are determined to belong to the same paragraph region, the candidate text box C and the candidate text box B* closest to it are merged. The merged new text box is the bounding rectangle of the candidate text box C and the candidate text box B* closest to it, where B* = argminCost(C, B) Cost(C, B) = V + λ × |Cx1 − Bx1| V is the line spacing between the candidate text box C and the candidate text box B, λ is the left boundary difference weight coefficient, and Cx1 and Bx1 are the left boundary horizontal coordinates of the candidate text boxes C and B, respectively. In the text detection and recognition method described above, optionally, in steps 2-4: HK 30134896 A Specification 5 sets a position tolerance coefficient ε. When the candidate text box D and the candidate text box E satisfy Dx1 − ε ≤ Ex1 ≤ Ex2 ≤ Dx2 + ε and Dy1 − ε ≤ Ey1 ≤ Ey2 ≤ Dy2 + ε, it is determined that the candidate text box E is nested inside the candidate text box D, and the candidate text box E is deleted. Wherein, ε = η × max⁡(W,H), Dx1, Dx2, Dy1, Dy2 are the left boundary x-coordinate, right boundary x-coordinate, upper boundary y-coordinate, and lower boundary y-coordinate of the candidate text box D, respectively; Ex1, Ex2, Ey1, Ey2 are the left boundary x-coordinate, right boundary x-coordinate, upper boundary y-coordinate, and lower boundary y-coordinate of the candidate text box E, respectively; η is the tolerance ratio coefficient; and W is the total width of the original image. H is the total height of the original image.In the text detection and recognition method described above, optionally, in steps 2-5: the horizontal expansion amount px, the vertical upward expansion amount pytop, and the vertical downward expansion amount pybot of the border expansion satisfy px = αx × W pytop = αytop × H pybot = αybot × H where αx is the horizontal expansion coefficient, αytop is the upward expansion coefficient, αybot is the downward expansion coefficient, satisfying αytop < αybot, W is the total width of the original image, H is the total height of the original image, and the new text box after border expansion satisfies x1′ = max⁡(0, x1 − px) x2′ = min⁡(W, x2 + px) y1′ = max⁡(0, y1 − pytop) y2′ = min⁡(H, y2 + pybot) Where x1, x2, y1, and y2 are the left, right, top, and bottom horizontal coordinates of the original text box, and x1', x2', y1', and y2' are the left, right, top, and bottom horizontal coordinates of the new text box. In the text detection and recognition method described above, optionally, in step 4, before performing text recognition on the text sub-image: Calculate the scaling ratio scale, scale = ⁡min⁡( Htarget Hblock , Wmax Wblock ) where ⁡Hblock is the height of the text sub-image, ⁡Wblock is the width of the text sub-image, ⁡Htarget is the target input height of the text recognition model, and Wmax is the maximum limited width of the text recognition model; Scale the text sub-image proportionally, I' = Resize(I, scale) where I is the text sub-image, I' is the scaled text sub-image; Perform text recognition on the scaled text sub-image I'.To achieve the aforementioned objectives, a second aspect of the present invention provides a text detection and recognition system for complex layout images, capable of implementing the text detection and recognition method described above. The text detection and recognition system includes: a text detection module for performing text detection on the original image and generating multiple candidate text boxes containing text; and a post-processing module for performing post-processing on the multiple candidate text boxes based on the layout structure characteristics of the text, and obtaining multiple effective text regions. The post-processing includes: performing non-maximum suppression processing on the candidate text boxes to eliminate overlapping duplicate text boxes; distinguishing the candidate text boxes into superscript text boxes and regular text boxes, merging the superscript text box and the nearest regular text box; determining the paragraph relationship between the multiple candidate text boxes, merging candidate text boxes belonging to the same paragraph region; determining the nesting relationship between the multiple candidate text boxes, deleting candidate text boxes nested within other candidate text boxes; and expanding the borders of the candidate text boxes. The image cropping module is used to crop the original image according to the effective text region and obtain a text sub-image corresponding to the effective text region; the text recognition module is used to perform text recognition on multiple text sub-images respectively and obtain text recognition results. The text detection and recognition method for complex layout images of the present invention, through preliminary text detection and post-processing based on the layout structure, crops the high-resolution original image into multiple low-resolution text sub-images containing effective text regions, and then performs text recognition on the text sub-images. This can improve the text features in the original image to a scale range that is easy to recognize, making text information that is unrecognizable in traditional methods recognizable while preserving text details, thus improving the clarity and accuracy of text recognition. Furthermore, the post-processing method based on the layout structure not only reduces redundant recognition areas and reduces background interference, improving the stability and efficiency of text recognition, but also preserves the structural information of the text, improving the completeness and accuracy of text recognition, which is beneficial for subsequent comprehensive and unbiased text information extraction and understanding. In optional embodiments, post-processing methods such as non-maximum suppression, merging superscript and text boxes, merging paragraph-related text boxes, deleting nested text boxes, and expanding text box borders can further preserve the complete semantic information and logical structure of the text content. By eliminating redundancy, repeated cropping and recognition are avoided, and the integrity of the text is prevented from being destroyed during subsequent cropping, thereby improving the efficiency, accuracy, stability, and integrity of text recognition. In a further optional embodiment, when expanding the text box, the downward expansion range is greater than the upward expansion range, which better matches the asymmetry of Chinese characters and English letters in the vertical structure, protecting the descending part of the character from being cropped and destroying it, and improving the recognizability of the text.This invention further provides a text detection and recognition system for complex layout images (HK 30134896 A Specification 8), thus this text detection and recognition system also has the aforementioned advantages. Brief Description of the Drawings The disclosure of this invention will become more apparent with reference to the accompanying drawings. It should be understood that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings: Figure 1 is a flowchart of an embodiment of the text detection and recognition method of this invention. Figure 2 is a schematic diagram of an embodiment of the original image to which this invention is applicable. Figure 3 is a schematic diagram of the effective text region obtained from the original image in Figure 2 by the post-processing method of this invention. Detailed Description Referring to the accompanying drawings and specific embodiments, the structure, composition, features, and advantages of the text detection and recognition method and system of this invention will be described below by way of example; however, all descriptions should not be used to limit this invention in any way. Furthermore, for any single technical feature described or implied in the embodiments mentioned herein, or any single technical feature shown or implied in the accompanying drawings, the present invention still allows for any combination or deletion of these technical features (or their equivalents) without any technical obstacle. Therefore, these further embodiments according to the present invention should also be considered within the scope of this description. Figure 1 is a flowchart of an embodiment of the text detection and recognition method of the present invention. The text detection and recognition method of the present invention is designed for high-resolution complex page images with diverse text layouts, high text density, and rich graphic layers, such as advertisements, product packaging, and poster images. Compared to form and ticket images with structured text content and containing key positioning information, and magazine and newspaper images with regular paragraph structures and clear reading order, if such complex page images are simply cropped into uniform sections for text recognition, it is very easy to lose text information, damage the integrity and semantic coherence of the text content, and produce incorrect recognition results. Especially when these complex layout images contain text content of a marketing nature, they often include superscript text in addition to regular text. This superscript text is either placed in the upper right corner of the text for annotation or forms footnotes with the following text. During text recognition, it is particularly important to preserve the structural relevance of superscript text within paragraphs to ensure complete and unbiased matching of multiple paragraphs corresponding to the same superscript during subsequent information extraction. This is beneficial for applications in various practical scenarios such as advertising copy review. Optionally, the text detection and recognition method used in this invention can be based on OCR (Opical Character Recognition) technology, or on other text recognition technologies; this invention does not impose any limitations on this.As shown in Figure 1, for the high-resolution original image to be recognized, this invention first performs region-level text detection in step 1 and generates multiple candidate text boxes containing text. Region-level text detection is a technique for locating and defining text regions in an image, using bounding boxes to provide regions of interest for subsequent text recognition. This invention uses rectangular bounding boxes as an example for detailed explanation later, but it should be noted that the text detection and recognition method and system of this invention are also applicable to bounding boxes of other shapes after simple deformation. Under high-resolution, complex layout image conditions, region-level text detection inevitably encounters various problems such as over-segmentation, insufficient segmentation, nesting, and / or overlap. Therefore, in step 2, based on the layout structure characteristics of the text, this invention performs post-processing on the multiple candidate text boxes obtained in step 1. In optional embodiments, the post-processing method may include, but is not limited to, the method shown in Figure 1: using non-maximum suppression (NMS) to eliminate overlapping duplicate text boxes; distinguishing candidate text boxes into subscript text boxes and regular text boxes, and merging the subscript text boxes with their nearest regular text boxes; determining the paragraph relationship between multiple candidate text boxes and merging candidate text boxes belonging to the same paragraph region; determining the nesting relationship between multiple candidate text boxes and deleting candidate text boxes nested within other candidate text boxes; and expanding the borders of the candidate text boxes. Specifically, the non-maximum suppression (NMS) process in step 2-1 is a method used to remove duplicate bounding boxes after target detection, retaining only the bounding box with the highest confidence for the same target. NMS is a common post-processing method in the prior art, and will not be elaborated upon here. In step 2-2, to incorporate superscript symbols such as ¹²³, *, etc., into their corresponding text areas and maintain the correct labeling relationship between text and superscripts, and between superscripts and footnotes in the image, thus preventing them from being cropped out and making it impossible to accurately distinguish between text and footnotes used for annotation, it is first necessary to distinguish between superscript text boxes and text text boxes. In an optional embodiment, an area threshold ratio coefficient βA, a height threshold βh, and a width threshold βw are set. When the area A, height h, and width w of a candidate text box satisfy A ≤ βA × W × H, h ≤ βh, and w ≤ βw, the candidate text box is determined to be a superscript text box s, and other candidate text boxes that do not meet the conditions are determined to be text text boxes m. Here, W is the total width of the original image, and H is the total height of the original image. Before merging the superscript text box and the text text box, it is necessary to confirm the relationship between the superscript text box s and the text text box m.Optionally, a horizontal position tolerance coefficient γ, a horizontal expansion tolerance coefficient βx, an upper vertical tolerance coefficient βy, and a lower vertical tolerance coefficient δ are set. When the following conditions are met: Sx1 ≥ mx2 − γ × (mx2 − mx1) Sx1 ≤ mx2 + βx ×W Sy2 ≥ my1 − βy × H Sy1 ≤ my2 + δ × (my2 − my1), it can be determined that the position of the subscript text box s is above and to the right of the text box m, thus determining that the text box m is the related text box of the subscript text box s. Here, Sx1, Sy1, and Sy2 are the left boundary x-coordinate, upper boundary y-coordinate, and lower boundary y-coordinate of the subscript text box s, respectively; and mx1, mx2, my1, and my2 are the left boundary x-coordinate, right boundary x-coordinate, upper boundary y-coordinate, and lower boundary y-coordinate of the text box m, respectively. A single superscript text box *s* may correspond to multiple related text boxes. In this case, the text box *m* closest to the superscript text box *s* is selected for merging. A cost function *Cost(s,m) = max(0, Sx1 −mx2) + |Sy2 −my1| is constructed. The nearest text box *m* satisfies *m* = argminCost(s,m). The merged new text box is the rectangle enclosing the superscript text box *s* and the nearest text box *m*. Thus, the superscript text with annotation function and its associated body text are merged. Furthermore, there are superscript texts constituting footnote content. The merging method for this type of superscript text is a conventional technique in this field, and will not be elaborated upon here. Incorporating isolated superscript characters into the text box most relevant to them avoids semantic breaks caused by superscripts being individually cropped and recognized later. This ensures accurate differentiation between the main text and footnotes used for annotation, as well as correct correspondence between the main text and footnotes that provide explanations of the main text. This improves the completeness and accuracy of the text recognition results in terms of understanding, making it more suitable for practical work scenarios such as information matching and document review. In steps 2-3, when determining the paragraph relationship between multiple candidate text boxes, optionally, a horizontal overlap threshold θx and a line spacing threshold coefficient θy can be set. When the horizontal overlap ratio Rx between candidate text box C and candidate text box B satisfies Rx ≥ θx, these two candidate text boxes can be considered to be in the same column. When the line spacing V between candidate text box C and candidate text box B satisfies V ≤ θy × H, these two candidate text boxes can be considered to be vertically adjacent. Therefore, when the aforementioned two conditions are met, these two candidate text boxes are determined to belong to the same paragraph region.Wherein, the horizontal overlap ratio Rx is defined as Rx = xoverlap xmin, where xoverlap is the horizontal overlap width of the two candidate text boxes, and xmin is the total width of the narrower text box in the two candidate text boxes described in HK 30134896 A Specification 12. This judgment method based on the degree of relative overlap can avoid the influence of factors such as the font size of the text, the absolute width of the candidate text boxes, and the text alignment format between paragraphs on the column relationship determination, thus improving the robustness of the post-processing process. Further optionally, when candidate text box C and multiple candidate text boxes B belong to the same paragraph area, candidate text box C and the candidate text box B* closest to it are merged. Set the left boundary difference weight coefficient λ, and construct the cost function Cost(C, B) = V + λ × |Cx1 − Bx1| where ⁡Cx1 is the left boundary x-coordinate of candidate text box C, and Bx1 is the left boundary x-coordinate of candidate text box B. The nearest candidate text box B* satisfies B* = argminCost(C, B). The merged new text box is the rectangle enclosing candidate text box C and the nearest candidate text box B*. Thus, multiple separate candidate text boxes in the same column are merged into a new text box containing the complete paragraph content, ensuring the logical and structural integrity and continuity of the selected content, facilitating subsequent overall cropping and text recognition. Furthermore, this left-boundary alignment method aligns with the left-to-right arrangement of text content in a paragraph. In steps 2-4, the nesting relationship between multiple candidate text boxes is determined, and candidate text boxes nested within other candidate text boxes are deleted. In an optional embodiment, a position tolerance coefficient ε is set. When candidate text box D and candidate text box E satisfy Dx1 − ε ≤ Ex1 ≤ Ex2 ≤ Dx2 + ε and Dy1 − ε ≤ Ey1 ≤ Ey2 ≤ Dy2 + ε, it is determined that the bounding box of candidate text box E is inside the bounding box of candidate text box D, thereby deleting candidate text box E. Here, Dx1, Dx2, Dy1, and Dy2 are the left, right, top, and bottom boundary coordinates of candidate text box D, respectively; and Ex1, Ex2, Ey1, and Ey2 are the left, right, top, and bottom boundary coordinates of candidate text box E, respectively. Optionally, a tolerance ratio coefficient η can be set such that the position tolerance coefficient ε satisfies ε = η × max⁡(W,H). By deleting nested redundant candidate text boxes, repeated text recognition can be avoided, improving the efficiency of text recognition. Simultaneously, it also avoids situations where strokes within characters are misdetected as independent text, enhancing the robustness of post-processing steps.In steps 2-5, the candidate text boxes are expanded horizontally and vertically. Optionally, a horizontal expansion coefficient αx, an upward expansion coefficient αytop, and a downward expansion coefficient αybot are set such that the horizontal expansion amount px, the vertical upward expansion amount pytop, and the vertical downward expansion amount pybot of the candidate text box satisfy px = αx × W, pytop = αytop × H, and pybot = αybot × H. Therefore, the new text box after the border expansion satisfies: x1′ = max⁡(0, x1 − px), x2′ = min⁡(W, x2 + px), y1′ = max⁡(0, y1 − pytop), and y2′ = min⁡(H, y2 + pybot). Where x1, x2, y1, and y2 are the left, right, top, and bottom horizontal coordinates of the original text box, respectively, and x1′, x2′, y1′, and y2′ are the vertical coordinates of the original text box. Let ' be the left, right, top, and bottom horizontal coordinates of the new text box. Proportionally expanding the bounding box of the candidate text box can prevent damage to the text integrity during subsequent cropping, thus improving the accuracy and stability of text recognition. In a further optional embodiment, the downward expansion range of the candidate text box is set to be greater than the upward expansion range, i.e., αytop < αybot. Since commonly used characters such as Chinese and English have asymmetry in their vertical structure, most have a descending portion extending downwards, and cropping these descending portions (HK 30134896 A Specification 14) significantly affects the recognizability of the characters. Therefore, a larger downward expansion is performed during border expansion to ensure the accuracy of subsequent text recognition. Based on these post-processing methods described above, multiple stable and effective text regions containing complete text information can be obtained from the candidate text box. It should be noted that the post-processing flow shown in Figure 1 is only an illustrative diagram of one embodiment of the present invention. The present invention is not limited to the steps shown, nor is it intended to require or imply that the post-processing steps of each embodiment must be performed in the order shown. Those skilled in the art will understand that other steps or sequences of steps can be used to implement the post-processing flow of this invention. As shown in Figure 1, in step 3, the original image is cropped based on the effective text region obtained through post-processing in step 2, resulting in a low-resolution text sub-image corresponding to the effective text region. It should be noted that the "low" resolution of the text sub-image here is a relative concept compared to the original image; that is, after cropping, the pixel density of the text sub-image remains unchanged to meet the basic requirements of text recognition.After cropping, the resolution of these text sub-images is significantly reduced, while the relative proportion of text is significantly increased. On the one hand, this magnifies the feature scale of each character to a range suitable for text recognition, avoiding the problem of small-sized text being unrecognizable in the original image. Furthermore, the layout differences between text regions in the original image no longer affect the accuracy of text recognition. On the other hand, it reduces background noise interference in the text sub-images, improving the efficiency and accuracy of text recognition. Simultaneously, the cropped text sub-images have relatively lower memory footprints, both individually and as a set, reducing the computational requirements for subsequent text recognition. In an optional embodiment, the coordinates of the effective text region can be represented in pixel coordinates or proportional coordinates, and a preview image containing region annotation information is generated for the user to fine-tune and confirm the partitioning results. In step 4, text recognition, such as OCR technology, is performed on the obtained multiple text sub-images. Optionally, before text recognition, these text sub-images are scale-normalized to keep the text height of the input text recognition model within a stable range, thereby improving the accuracy of text recognition. Text height refers to the vertical dimension of the entire text region, i.e., the input text subimage. Due to significant size differences between the original text subimages (e.g., larger title text regions and smaller footnote text regions), directly inputting cropped text subimages into the text recognition model may lead to recognition errors in smaller text regions due to insufficient character pixel height. Character pixel height refers to the pixel representation of the height of a single character in the image, i.e., the number of pixel rows occupied by the character after digitization. A scaling factor `scale` is calculated based on the text subimage height `Hblock`, text subimage width `Wblock`, the target input height `Htarget` of the text recognition model, and the maximum restricted width `Wmax` of the text recognition model, satisfying `scale = ⁡min⁡( Htarget Hblock , Wmax Wblock )`. This provides the scaling factor that adjusts the text height to the optimal recognition height of the model. Simultaneously, constraining it with the maximum restricted width avoids overstretching and generating excessively large images. Based on this scaling factor `scale`, the original text subimage `I` is scaled proportionally to obtain the scaled text subimage `I′`, and text recognition is then performed on this scaled text subimage `I′`. Where I′ = Resize(I, scale) means that the height of the scaled text sub-image I′ is equal to or slightly smaller than the target input height of the text recognition model. Through the scale normalization process described above, the text sub-image is adaptively scaled to a suitable recognition scale, which not only improves the character pixel height and text recognition accuracy in small-sized text regions, but also reduces the input scale difference between different text sub-images and improves the stability of text recognition.Furthermore, the obtained text recognition results can be optionally used in various application scenarios such as information extraction, content matching, and document review, and this invention does not limit this. Figures 2 and 3 schematically illustrate an embodiment of the original image to which this invention applies, and a schematic diagram of the effective text region obtained by the post-processing method of this invention. As shown in Figure 2, this is a product packaging image with a low text area ratio, diverse text font sizes and layout formats, containing various graphic elements other than text, multiple superscript annotations, and densely packed small-font text areas. Under traditional text recognition technology, not only are false detections prone to occur, but the small-font text cannot be "clearly seen." If the existing uniform window segmentation method is used to preprocess the image by cropping, the resulting sub-image to be recognized will have problems such as incomplete text, missing paragraph structure relationships, and obvious background interference, resulting in the recognition results being unable to extract complete and accurate paragraph semantic information, thus losing the practical application significance of text recognition for this image. Similarly, in other similar high-resolution images, relatively small characters and superscript symbols may be unrecognizable or misrecognized due to their low absolute size. Figure 3 shows the bounding boxes and serial numbers of the effective text regions obtained after the region-level text detection and post-processing of this invention. These effective text regions divide the text content in the original image into sections according to the page layout, ensuring that information within the same paragraph is preserved completely and without deviation, thus providing semantic continuity for subsequent text recognition results. Furthermore, for example, superscript symbols such as *, ^, and # in bounding box 1 are completely preserved and recognized along with the main text, allowing association with text content also marked with these superscript symbols, such as *, ^, and #, based on the text recognition results. This invention also discloses a text detection and recognition system for complex page layout images, used to implement the text detection and recognition method described above. The text detection and recognition system may include a text detection module, a post-processing module, an image cropping module, and a text recognition module. The text detection module performs region-level text detection on the original image and generates multiple candidate text boxes containing text. The post-processing module, based on the text's layout characteristics (HK 30134896 A Specification 17), performs post-processing on the multiple candidate text boxes to obtain multiple valid text regions containing complete text information. The image cropping module crops the original image according to the valid text regions, obtaining low-resolution text sub-images corresponding to the valid text regions. These text sub-images have a character feature scale suitable for text recognition. The text recognition module performs text recognition on each text sub-image and obtains the text recognition results.Optionally, the text detection and recognition system may also include a preview module for representing the effective text region in pixel coordinates or proportional coordinates, and generating a preview image containing region annotation information. Further, the text detection and recognition system of the present invention can also be deployed as a whole in, for example, an image information matching system or an advertising copy review system, and applied to various work scenarios. The technical scope of the present invention is not limited to the content of the above specification. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of the present invention, and all such modifications and variations should fall within the scope of the present invention. HK 30134896 A Claim 1 1. A method for text detection and recognition of complex layout images, characterized in that the method comprises: Step 1: performing text detection on a high-resolution original image and generating multiple candidate text boxes containing text; Step 2: post-processing the multiple candidate text boxes based on the layout structure characteristics of the text to obtain multiple effective text regions; Step 3: cropping the original image according to the effective text regions to obtain text sub-images corresponding to the effective text regions; Step 4: performing text recognition on the multiple text sub-images respectively to obtain text recognition results; wherein, Step 2 includes, but is not limited to: Step 2-1: performing non-maximum suppression processing on the candidate text boxes to eliminate overlapping duplicate text boxes; Step 2-2: distinguishing the candidate text boxes into subscript text boxes and text text boxes, and merging the subscript text boxes and the text text boxes closest to them; Step 2-3: determining the paragraph relationship between the multiple candidate text boxes, and merging the candidate text boxes belonging to the same paragraph region; Step 2-4: Determine the nesting relationship between multiple candidate text boxes and delete candidate text boxes nested inside other candidate text boxes; Step 2-5: Expand the borders of the candidate text boxes. 2. The text detection and recognition method as described in claim 1, characterized in that, in step 2-2: Set an area threshold ratio coefficient βA, a height threshold βh, and a width threshold βw. When the area A of a candidate text box satisfies A ≤ βA × W × H, and the height h and width w of the candidate text box satisfy h ≤ βh, w ≤ βw, the candidate text box is determined to be a subscript text box s, and other candidate text boxes that do not meet the conditions are determined to be text text boxes m, where W is the total width of the original image and H is the total height of the original image.3. The text detection and recognition method as described in claim 1, characterized in that, in step 2-2: the positional relationship between the superscript text box s and the text box m is determined, and when the following conditions are met: Sx1 ≥ mx2 − γ × (mx2 − mx1) Sx1 ≤ mx2 + βx ×W Sy2 ≥ my1 − βy × H Sy1 ≤ my2 + δ × (my2 − my1), the text box m is determined to be the related text box of the superscript text box s. Where Sx1, Sy1, and Sy2 are the left boundary x-coordinate, upper boundary y-coordinate, and lower boundary y-coordinate of the subscript text box s, respectively; mx1, mx2, my1, and my2 are the left boundary x-coordinate, right boundary x-coordinate, upper boundary y-coordinate, and lower boundary y-coordinate of the text box m, respectively; γ is the horizontal position tolerance coefficient; βx is the horizontal expansion tolerance coefficient; βy is the upper vertical tolerance coefficient; δ is the lower vertical tolerance coefficient; W is the total width of the original image; and H is the total height of the original image. In the relevant text boxes, the text box m* closest to the subscript text box s is selected, satisfying m* = argminCost(s,m) Cost(s,m) = max(0, Sx1 −mx2) + |Sy2 −my1|. The subscript text box s and the closest text box m* are merged, and the merged new text box is the bounding rectangle of the subscript text box s and the closest text box m*. 4. The text detection and recognition method as described in claim 1, characterized in that, in steps 2-3: a horizontal overlap threshold θx and a line spacing threshold coefficient θy are set; when the horizontal overlap ratio Rx and line spacing V between two candidate text boxes satisfy Rx ≥ θx and V ≤ θy × H, it is determined that the two candidate text boxes belong to the same paragraph region, wherein Rx = xoverlap xmin, xoverlap is the horizontal overlap width of the two candidate text boxes, xmin is the total width of the narrower text box in the two candidate text boxes, and H is the total height of the original image.5. The text detection and recognition method as described in claim 1, characterized in that, in steps 2-3: when the candidate text box C and multiple candidate text boxes B are determined to belong to the same paragraph region, the candidate text box C and the candidate text box B* closest to it are merged, and the merged new text box is the bounding rectangle of the candidate text box C and the candidate text box B* closest to it, where, B* = argminCost(C, B) Cost(C, B) = V + λ × |Cx1 − Bx1| V is the line spacing between the candidate text box C and the candidate text box B, λ is the left boundary difference weight coefficient, and Cx1 and Bx1 are the left boundary horizontal coordinates of the candidate text boxes C and B, respectively. 6. The text detection and recognition method as described in claim 1, characterized in that, in steps 2-4: a position tolerance coefficient ε is set, and when the candidate text box D and the candidate text box E satisfy Dx1 − ε ≤ Ex1 ≤ Ex2 ≤ Dx2 + ε and Dy1 − ε ≤ Ey1 ≤ Ey2 ≤ Dy2 + ε, it is determined that the candidate text box E is nested inside the candidate text box D, and the candidate text box E is deleted, wherein ε = η × max⁡(W,H), Dx1, Dx2, Dy1, and Dy2 are respectively the left boundary x-coordinate, right boundary x-coordinate, upper boundary y-coordinate, and lower boundary y-coordinate of the candidate text box D, Ex1, Ex2, Ey1, and Ey2 are respectively the left boundary x-coordinate, right boundary x-coordinate, upper boundary y-coordinate, and lower boundary y-coordinate of the candidate text box E, η is the tolerance ratio coefficient, and W is the total width of the original image. H is the total height of the original image.7. The text detection and recognition method as described in claim 1, characterized in that, in steps 2-5: the horizontal expansion amount px, the vertical upward expansion amount pytop, and the vertical downward expansion amount pybot of the border expansion satisfy px = αx × W, pytop = αytop × H, pybot = αybot × H, where αx is the horizontal expansion coefficient, αytop is the upward expansion coefficient, αybot is the downward expansion coefficient, satisfying αytop < αybot, W is the total width of the original image, H is the total height of the original image, and the new text box after border expansion satisfies x1′ = max⁡(0, x1 − px), x2′ = min⁡(W, x2 + px), y1′ = max⁡(0, y1 − pytop), y2′ = min⁡(H, y2 + pybot). Where x1, x2, y1, and y2 are the left, right, top, and bottom horizontal coordinates of the original text box, and x1', x2', y1', and y2' are the left, right, top, and bottom horizontal coordinates of the new text box. 8. The text detection and recognition method as described in claim 1, characterized in that, in step 4, before performing text recognition on the text sub-image: Calculate the scaling ratio scale, scale = ⁡min⁡( Htarget Hblock , Wmax Wblock ) HK 30134896 A Claim 5 Wherein, ⁡Hblock is the height of the text sub-image, ⁡Wblock is the width of the text sub-image, ⁡Htarget is the target input height of the text recognition model, and Wmax is the maximum limited width of the text recognition model; Scale the text sub-image proportionally, I′ = Resize(I, scale) Wherein, I is the text sub-image, and I′ is the scaled text sub-image; Perform text recognition on the scaled text sub-image I′.9. A text detection and recognition system for complex layout images, capable of implementing the text detection and recognition method as described in any one of claims 1-8, characterized in that the text detection and recognition system comprises: a text detection module, used to perform text detection on the original image and generate multiple candidate text boxes containing text; a post-processing module, used to perform post-processing on the candidate text boxes based on the layout structure characteristics of the text, and obtain an effective text region, wherein the post-processing includes: performing non-maximum suppression processing on the candidate text boxes to eliminate overlapping duplicate text boxes; distinguishing the candidate text boxes into subscript text boxes and text text boxes, and merging the subscript text boxes and the text text boxes closest to them; determining the paragraph relationship between the multiple candidate text boxes, and merging the candidate text boxes belonging to the same paragraph region; determining the nesting relationship between the multiple candidate text boxes, and deleting the candidate text boxes nested inside other candidate text boxes; expanding the borders of the candidate text boxes; and an image cropping module, used to crop the original image according to the effective text region, and obtain a text sub-image corresponding to the effective text region; The text recognition module is used to perform text recognition on multiple text sub-images respectively and obtain the text recognition results. HK 30134896 A Specification Figure 1 Figure 2 HK 30134896 A Specification Figure 2 Figure 3 HK 30134896 A.

Images