Multi-level text correction method and system based on document layout analysis

Abstract

The application relates to the technical field of application of artificial intelligence in document image processing, and discloses a multi-level text correction method and system based on document layout analysis, which comprises the following steps: combining a multi-scale self-similarity feature algorithm and a directional frequency domain peak value feature algorithm to distinguish the type of an image to be corrected, and performing adaptive pretreatment to obtain a standardized image; extracting a text connected domain in the standardized image; using an unsupervised clustering technology to cluster each symbol in the text connected domain to obtain a plurality of word clusters; merging the word clusters to form text blocks; obtaining the minimum circumscribed quadrilateral of each text block to obtain a corresponding text box; respectively obtaining the center point coordinates of each text box; determining whether two text boxes are the same line of text; performing horizontal alignment and tilt correction on the text boxes; performing morphological regularization processing on the characters in the rotated text boxes; and outputting the corrected image and structured JSON data. The application has strong adaptability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of application of artificial intelligence in document image processing, and particularly relates to a multi-level text correction method and system based on document layout analysis. BACKGROUND

[0002] In recent years, with the growing demand for electronic office and the widespread popularity of mobile devices, the demand for digital collection of paper documents has become increasingly prominent. In particular, thanks to the improvement of mobile phone camera performance and the progress of image processing algorithms, individual users can conveniently obtain document images and then apply them to intelligent tasks such as text recognition and information structured extraction. However, in actual application, the original document image often has serious distortion problems, such as page bending, shadow blocking, blurred imaging, and unclear text, which directly affect the subsequent OCR accuracy and layout restoration effect, thereby restricting the efficiency and accuracy of document automation processing.

[0003] For the problem of bending distortion correction of documents, there are currently a variety of solutions. Early solutions mainly rely on text line detection and use mathematical transformation models (such as coordinate mapping) to reconstruct the text direction to restore it to a horizontal or vertical arrangement. However, these traditional methods are highly sensitive to the quality of the document image and the page layout structure, and often perform poorly when dealing with documents containing complex charts and non-standard layouts. In addition, false detection of text lines can lead to distorted correction results.

[0004] In order to improve processing accuracy, researchers have proposed methods based on optimization algorithms in recent years, which train neural network models through iterative loss functions to approximate the displacement mapping under ideal conditions. Although this method performs better when the image quality is high, it often takes too long to calculate in actual applications, making it difficult to meet real-time requirements.

[0005] With the progress of deep learning technology and the emergence of large-scale labeled data sets, methods based on learning displacement field generation have gradually become a research hotspot. This type of method extracts features from document images by constructing a neural network model and directly predicts the pixel-level displacement corresponding to the bending distortion area, thereby achieving high-precision image unwarping. In particular, after introducing the self-attention mechanism architecture, this technology has achieved remarkable results in complex document recognition and layout analysis. Although deep learning-based methods have strong generalization ability and high performance, they still have certain dependencies on data quality, labeling standards, and hardware resources, and face certain adaptation bottlenecks in actual business applications.

[0006] Currently, a relatively mature alternative strategy is to directly use large multi-modal pre-training models for end-to-end document structure restoration. For example, by designing specific prompt words or instruction templates, the system can automatically complete the text recognition and information extraction tasks, and to some extent, realize the layout analysis function. However, this method relies on the processing capacity of the large model itself and the supporting resources, and there is a problem of "overcapacity" when processing simple document images, which is difficult to reflect cost effectiveness. SUMMARY

[0007] The present application is dedicated to overcoming many challenges faced in the current field of document image processing, such as page bending, shadow, blur and other factors often lead to unsatisfactory recognition rate of OCR (Optical Character Recognition), and the restoration of document structure is not accurate enough. In order to solve these problems, a multi-level text correction method and system based on document layout analysis are provided, which can still accurately detect, align and rotate correct text lines for complex document images without a large amount of training data and high-performance hardware support, and ultimately assist OCR technology to output structured information.

[0008] The present application is implemented by the following technical solutions:

[0009] A multi-level text correction method based on document layout analysis, comprising:

[0010] For the image to be corrected, combine the multi-scale self-similarity feature algorithm and the directional frequency domain peak feature algorithm to determine the type of the image to be corrected, and adaptively preprocess the image to be corrected according to the determined image type to obtain a standardized image;

[0011] Extract the text connected domain in the standardized image, use unsupervised clustering technology to cluster each symbol in the text connected domain to obtain a plurality of word clusters, merge the word clusters that meet the preset conditions to form at least one text block, and obtain the minimum bounding rectangle of each text block to obtain the corresponding text box;

[0012] Respectively obtain the center point coordinates of each text box, and determine whether two text boxes are the same line of text according to the difference between the center point vertical coordinates of adjacent text boxes;

[0013] Perform horizontal alignment and tilt correction on all text boxes belonging to the same line of text, so that all text boxes belonging to the same line of text are rotated to the same horizontal line;

[0014] Perform morphological regularization processing on the characters in the rotated text box, so that the area where the characters are located forms a standard rectangular text block with straight edges and no misalignment, to avoid the destruction of character morphology due to anti-aliasing or smoothing processing;

[0015] Output the corrected image and structured JSON data.

[0016] As an optimization, the multi-scale self-similarity feature algorithm calculates the structural similarity index (SSIM) of adjacent scales of the image to be corrected to determine image scaling stability. When the SSIM is ≥ 0.7, the image to be corrected is determined to be a high-quality digital document image; when the SSIM is < 0.7, the image to be corrected is determined to be a low-quality photograph. The directional frequency domain peak feature algorithm detects the peak values ​​in the horizontal and vertical directions of the image to be corrected to recognize screen-captured images. When the peak values ​​in the low and mid frequencies of the horizontal or vertical directions exceed a threshold, it is determined to be a screen-captured image.

[0017] As an optimization, when performing adaptive preprocessing on the image to be corrected, if it is determined to be a screen capture image or a low-quality photo, a noise removal operation is performed using a hybrid algorithm based on local extrema and frequency domain energy distribution combined with gamma curves. Then, the image to be corrected after the noise removal operation is standardized to obtain a standardized image.

[0018] As an optimization, the process of extracting the text connected components from the standardized image and using unsupervised clustering techniques to cluster each symbol in the text connected components to obtain several word clusters is as follows:

[0019] The unsupervised clustering technique is set as the DBSCAN clustering algorithm. When using the DBSCAN clustering algorithm to cluster each symbol in the connected component of the text, a 4-dimensional feature vector is constructed based on the horizontal distance, vertical distance, area ratio and centroid angle difference between any two adjacent symbols. The clustering neighborhood radius ε and the minimum number of samples MinPts are set. When the feature vector distance between two adjacent symbols is ≤ε and the number of samples in the neighborhood is ≥MinPts, the two symbols are assigned to the same cluster.

[0020] For each cluster, calculate the set of horizontal distances between all symbol pairs within the cluster. With vertical spacing set Take the set of horizontal spacings The median is the horizontal spacing of the symbol pairs. Take the set of vertical spacings The median is the vertical spacing of the symbol pairs. If the number of symbols in the cluster is ≥3, then the horizontal spacing set is removed. With vertical spacing set The deviation from the median is >1.5× The word clusters are obtained by using the symbols of outliers, the median is recalculated, the median is corrected, and the horizontal spacing set is recalculated based on the corrected median. Set of vertical distances IQR is the interquartile range.

[0021] As an optimization, when merging word clusters that meet preset conditions to form a text block, the preset conditions include a first merging condition and a second merging condition: the first merging condition is that the horizontal direction overlap ratio between word clusters > 50% and the vertical center difference < 1 / 2 of the median of the word cluster height, and the word clusters that meet the first merging condition are merged into a text line, and the centroid coordinates of each text line are calculated; the second merging condition is used to merge text lines into a text block, and the specific determination logic is:

[0022] Calculate the vertical distance of all adjacent text line centroids, and take the median of all vertical distances as the standard line spacing ;

[0023] For any two text lines to be determined, calculate the actual vertical distance d of the centroids of the two text lines, and calculate the overlap ratio of all pixels in the vertical direction of the two text lines to be determined;

[0024] When the actual vertical distance d of the two text lines to be determined meets , and the overlap ratio of all pixels in the vertical direction > 20%, determine that the two text lines are associated text lines, and merge the two text lines to be determined into the same text block;

[0025] Repeat the previous step until all text lines that meet the association condition are merged to form independent text blocks.

[0026] As an optimization, the specific way to obtain the center point coordinates of the text box is: the four vertices of the minimum circumscribed quadrilateral corresponding to the text box are clockwise from the lower left corner , and the area weight coefficient is constructed ;

[0027] Calculate the center point coordinates of the text box , the horizontal coordinate and vertical coordinate of the center point coordinates are , and the calculation formula of the center point coordinates is: , .

[0028] As an optimization, the specific process of determining whether it is the same row of text according to the vertical coordinate difference of adjacent text boxes is:

[0029] Calculate the vertical projection span T of each text box; ; represents the coordinate value of the i-th vertex in the vertical direction;

[0030] calculating a dynamic threshold based on two horizontally adjacent text boxes , , is an experience parameter, respectively represent the center point vertical coordinates of the qth text box and the (q-1)th text box.

[0031] when the center point vertical coordinates difference of two adjacent text boxes is less than a first threshold value, the two adjacent text boxes are determined as the same row of text, respectively represent the center point vertical coordinates of the qth text box and the (q-1)th text box.

[0032] As an optimization, horizontal alignment and skew correction are performed on all text boxes belonging to the same row of text, so that all text boxes belonging to the same row of text are rotated to the same horizontal line. The specific process is as follows:

[0033] calculating the average vertical coordinate of all text boxes belonging to the same row of text and adjusting the vertical position of all text boxes belonging to the same row of text based on the average vertical coordinate so that the top vertices of the lower left corner pixels of all text boxes belonging to the same row of text are horizontally aligned, where N is the number of text boxes, represents the area of the qth text box, represents the center point vertical coordinate of the qth text box.

[0034] calculating the single text box slope and calculating the single box skew angle based on the single text box slope , the calculation formula of the single text box slope is: , , wherein, represents the coordinates of the pixel vertex of the lower left corner of the single text box ; (1) represents the vertex with the largest horizontal distance from the vertex ; if the single box skew angle is greater than a set first threshold value, the single text box is rotated with the pixel vertex as the center and the angle .

[0035] As an optimization, the character in the rotated text box is subjected to a morphological regularization process, so that the area where the character is located forms a standard rectangular text block with a neat edge and no misalignment, to avoid the character form being destroyed due to anti-aliasing or smoothing processing. The specific process is as follows:

[0036] ​The rotated text box is enlarged by an integer multiple of 4, and then an integer grid composed of a plurality of cells is established from the lower left corner with a step of 4 pixels; and the lower left corner cell is set as the origin for numbering and marked as ;

[0037] From the origin cell , adjacent cells with consistent content attributes are merged in the order of right first and then up to obtain a maximum cell according to a set merging rule, and the merging rule includes:

[0038] Horizontal priority: if the cell has consistent attributes with the cell , the cell is horizontally merged with the cell to obtain a horizontal merged cell group, wherein the cell coordinates are represented by , and represents a right shift of one cell,

[0039] Vertical second: if the attribute of the previous cell of each cell in the horizontal merged cell group is consistent with the attribute of the horizontal merged cell group, each cell in the horizontal merged cell group is merged with the corresponding previous cell;

[0040] Attribute consistency determination: if the gray histogram of the pixels in the cell has only one main peak value, the cell is a pure color block , otherwise it is a colorless block ;

[0041] According to the merging rule, each maximum cell is recorded in the order of , and then the origin maximum cell connected to the lower left pixel after rotation correction is taken as a reference for reverse splicing in the order of vertical first and horizontal second to obtain a preliminarily combined text block;

[0042] Gap filling processing is performed on the preliminarily combined text block: for each horizontal gap, the most edge column of pixels of the left and right adjacent small blocks is taken for Gaussian blur filling; the same filling is performed on the upper and lower rows of pixels of the vertical gap; after gap filling, a standard rectangular text block with neat edges and no misalignment is finally obtained, and the layout reorganization of the original pixel information is completed.

[0043] The application also discloses a multi-level text correction system based on document layout analysis, which is used to execute the multi-level text correction method based on document layout analysis.

[0044] An image preprocessing module is configured to determine the type of a to-be-corrected image by combining a multi-scale self-similarity feature algorithm and a directional frequency domain peak feature algorithm, and to adaptively preprocess the to-be-corrected image according to the determined type of the image to obtain a standardized image;

[0045] A text layout analysis module is configured to extract text connected domains in the standardized image, to cluster each symbol in the text connected domains by using an unsupervised clustering technique to obtain a plurality of word clusters, to merge word clusters satisfying a preset condition to form at least one text block, and to obtain a minimum bounding rectangle of each text block to obtain a corresponding text box;

[0046] A text box level processing module is configured to obtain the center point coordinates of each text box, and to determine whether two text boxes are in the same line according to the difference between the center point vertical coordinates of adjacent text boxes.

[0047] An out-of-box line level processing module is configured to perform horizontal alignment and tilt correction on all text boxes belonging to the same line of text, so that all text boxes belonging to the same line of text are rotated to the same horizontal line.

[0048] An in-box character level processing module is configured to perform morphological regularization processing on the characters in the rotated text boxes, so that the regions where the characters are located form standard rectangular text blocks with neat edges and no misalignment, to avoid the characters from being damaged due to anti-aliasing or smoothing processing.

[0049] A processing image and position information structured integration output module is configured to output the corrected image and structured JSON data.

[0050] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0051] The present application does not require large-scale training data, and its core advantage lies in its unique design based on geometric principles and grouping rules, which makes the present application highly adaptable. Even in the face of diverse document formats and contents, it can be flexibly handled without relying on a large number of training samples for learning.

[0052] For complex layout and distorted documents, the present application exhibits high robustness, and can effectively process document images in various practical scenarios, whether they are scans, photos or printed documents, and can accurately identify and analyze them. Even in the presence of wrinkles, stains or uneven lighting, the recognition accuracy remains high.

[0053] In terms of computational efficiency, the present application performs outstandingly, not only in terms of processing speed, but also in terms of ease of deployment, being able to adapt to the computing power of edge devices and ordinary PCs, without relying on high-performance GPUs, so that it can run efficiently in various hardware environments.

[0054] Furthermore, the structured position information output provided by this invention is very accurate, which greatly facilitates character position correction for subsequent downstream tasks such as OCR (Optical Character Recognition) and information extraction. The recognized information can be rearranged according to the results of this method as needed.

[0055] Finally, the present invention is highly scalable, with multi-granularity correction at the character level, line level, and text block level. This means that the present invention can flexibly adjust the processing granularity according to different application scenarios and needs, thereby meeting the needs of various complex document processing. Attached Figure Description

[0056] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0057] Figure 1 This is a flowchart of a multi-level text correction method based on document layout analysis as described in this invention;

[0058] Figure 2 A flowchart for image preprocessing;

[0059] Figure 3 Illustration of the clustering effect of the DBSCAN algorithm;

[0060] Figure 4 This is a schematic diagram of the correction result for a standard rectangular frame;

[0061] Figure 5 This invention defines the module relationships and output content of the system described herein.

[0062] Figure 6 This is a schematic diagram illustrating how unsupervised clustering techniques are used to cluster adjacent characters in an image to obtain clusters in a specific scheme.

[0063] Figure 7 To Figure 6 A schematic diagram illustrating the results of text line relationship analysis using interline character group (word) clustering;

[0064] Figure 8 To Figure 7 A diagram showing the result after processing the text box;

[0065] Figure 9 To Figure 8 A diagram obtained after character-level / line-level processing;

[0066] Figure 10 To Figure 9 A schematic diagram obtained by dividing the text block into internal rectangular blocks;

[0067] Figure 11 A schematic diagram of a text block in the image after correction and Gaussian blur filling processing; Figure 10

[0068] A schematic diagram of the result after OCR extraction. Figure 12 Figure 11 DETAILED DESCRIPTION

[0069] In order to make the objects, technical solutions and advantages of the present application clearer, further detailed description will be given below in combination with embodiments and drawings, and the schematic embodiments of the present application and the description thereof are only used to explain the present application, but not to limit the present application.

[0070] The present embodiment 1 provides a multi-level text correction method based on document layout analysis. In the field of image processing, accurate detection and clustering of text lines are key steps in the task of text recognition. As shown in the following formula, it includes: Figure 1

[0071] S1, for the image to be corrected, combining a multi-scale self-similarity feature algorithm and a directional frequency domain peak feature algorithm to determine the type of the image to be corrected, and adaptively pre-processing the image to be corrected according to the determined image type to obtain a standardized image.

[0072] S2, extracting text connected domains in the standardized image, using unsupervised clustering technology to cluster each symbol in the text connected domain to obtain a plurality of word clusters, merging word clusters satisfying a predetermined condition to form at least one text block, and obtaining the minimum circumscribed quadrilateral of each text block to obtain the corresponding text box.

[0073] S3, obtaining the center point coordinates of each text box, and determining whether two text boxes are the same line of text according to the difference between the center point vertical coordinates of adjacent text boxes.

[0074] S4, performing horizontal alignment and tilt correction on all text boxes belonging to the same line of text, so that all text boxes belonging to the same line of text are rotated to the same horizontal line.

[0075] S5, performing morphological regularization processing on the characters in the rotated text box, so that the area where the characters are located forms a standard rectangular text block with straight edges and no misalignment, to avoid the destruction of character morphology due to anti-aliasing or smoothing processing;

[0076] S6, outputting the corrected image and structured JSON data.

[0077] Next, each step will be described in detail.

[0078] ​​​Step S1 is responsible for correctly extracting the required content from the image and normalizing it. The main goal of image preprocessing is to improve the quality and accuracy of subsequent feature extraction, as shown in the flowchart Figure 2

[0079] In some embodiments, the multi-scale self-similarity feature algorithm realizes image scaling stability judgment by calculating the structural similarity index SSIM of adjacent scales of the image to be corrected. When the structural similarity index is ≥ 0.7, it is determined that the image to be corrected is a high-quality digital document picture; when the structural similarity index is < 0.7, it is determined that the image to be corrected is a low-quality photo; the directional frequency domain peak feature algorithm realizes screen camera picture recognition by detecting the comprehensive peak value of the horizontal direction and the vertical direction in the frequency domain of the image to be corrected. When the comprehensive peak value of the horizontal direction or the vertical direction in the low frequency and the medium frequency exceeds the threshold value, it is judged as a screen camera picture.

[0080] It should be noted that moire is generated by the interference of the sensor grid and the screen pixel grid. The superposition of the two grids will form obvious directional peaks in the low and medium frequency domain. Whether it is horizontal direction or vertical direction is affected by the angle direction of the camera sensor and the screen pixel grid. In the horizontal direction and the vertical direction, when the peak value of one direction is obvious, the peak value of the other direction will be weak, so as long as there is a directional peak in one direction, it can be judged. If it does not exceed the threshold value, it means that the image is a clear photo, a screenshot or other picture without moire or noise.

[0081] This part is to distinguish the image type of the image to be corrected: by judging the content in the image to be corrected, the subsequent processing method is determined, including but not limited to document photo / scanned copy, screen camera picture, combining multi-scale self-similarity feature algorithm and directional frequency domain peak feature algorithm, and then performing targeted image processing after judging the image type.

[0082] The multi-scale self-similarity feature algorithm captures the stability of the image when scaling by calculating the structural similarity index between adjacent scales. Scanned copies are digitally collected and enhanced by scanning steps, so the self-similarity is high. Paper photos have low self-similarity due to blurring. The image is scaled by k scales, and then the structural similarity is calculated using the SSIM algorithm.

[0083] ;

[0084] wherein, ​S represents the average value of structural similarity, and is used to measure the degree of structural similarity between multiple images or image regions. The higher S is, the higher the stability of the image is, and the content is almost not affected by scaling. In the present application, when S≥0.7, the image content is judged to be a high-quality digitized document picture such as a scanned copy or a screenshot with a relatively high digital acquisition instruction, and when S<0.7, the image content is judged to be a photo or a screen capture picture.

[0085] The directional frequency domain peak feature algorithm is aimed at the moire of the screen capture picture. Since the camera position of the screen capture picture is usually in the opposite state, the peak values in the horizontal and vertical directions are mainly detected in the frequency domain. When the projection in the horizontal direction or the vertical direction exceeds the normal image in the low frequency and medium frequency, the screen capture picture is judged. The directional projection is P, the frequency domain is f (the frequency domain to be judged is the low frequency and the medium frequency), the comprehensive peak value is T, h represents the vertical direction, and v represents the horizontal direction. When it is judged that T exceeds the set threshold value, the screen capture picture is judged.

[0086] The calculation formula of the comprehensive peak value is: ; T(f) represents the comprehensive peak value in the frequency domain f, P h(f) and P v(f) represent the projection in the vertical direction and the projection in the horizontal direction in the frequency domain f, respectively.

[0087] It should be noted that the vertical direction can be understood as the longitudinal direction.

[0088] In some embodiments, when the adaptive preprocessing of the to-be-corrected image is performed, if it is judged to be a screen capture picture or a low-quality photo, a hybrid algorithm based on local extreme value and frequency energy distribution is combined with a gamma curve to perform a noise removal operation, and then the to-be-corrected image after the noise removal operation is standardized to obtain a standardized image.

[0089] More specifically:

[0090] Noise removal: for screen capture pictures or poor-quality photos, a hybrid algorithm based on local extreme value and frequency energy distribution is combined with a gamma curve to remove screen reflections, moire, overexposure / underexposure and other noises in the image, enhance the contrast of key information in the image, and improve the accuracy of subsequent text and image feature extraction.

[0091] Image standardization: for document photos / scanned copies, the document in the image needs to be twisted and corrected and angle-corrected to ensure that the geometric shape of the image is standardized and the interference caused by the shooting angle, deformation and other factors is eliminated.

[0092] This step provides high-quality and standardized input for the downstream steps, ensuring the reliability of the subsequent analysis.

[0093] Step S2 mainly involves performing text layout analysis on the image to be corrected in order to determine the original structure of the text.

[0094] In some embodiments, the specific process of extracting the text connected components in the standardized image and using unsupervised clustering technology to cluster each symbol in the text connected components to obtain several word clusters includes three steps: S2.1, S2.2, and S2.3.

[0095] S2.1. Set the unsupervised clustering technique as the DBSCAN clustering algorithm. When using the DBSCAN clustering algorithm to cluster each symbol in the connected components of the text, construct a 4-dimensional feature vector based on the horizontal distance, vertical distance, area ratio, and centroid angle difference between any two adjacent symbols. Set the clustering neighborhood radius ε and the minimum number of samples MinPts. When the feature vector distance between two adjacent symbols is ≤ε and the number of samples in the neighborhood is ≥MinPts, the two symbols are assigned to the same cluster.

[0096] The centroid can be understood as the geometric center, which is the average of the coordinates of all pixels within the set of text lines.

[0097] Geometric center pixel coordinates: ;

[0098] Where N is the number of pixels in the text line set, ( , ) represents the coordinates of the i-th pixel.

[0099] First, the standardized image is binarized to extract the text connected components (character-level symbols).

[0100] Cluster analysis: For each symbol, the unsupervised clustering technique (DBSCAN) is used to group the feature vectors of adjacent symbols into clusters, outputting character group (word) clusters. The main method is to perform clustering on any symbol... According to Manhattan distance in radius Find the nearest neighbor symbol j within the inner circle. Indicates the average character width The median of the dataset is calculated, and then a 4-dimensional eigenvector is calculated:

[0101] ;

[0102] ;

[0103] ;

[0104] in For horizontal spacing, For vertical spacing, For area ratio, This is due to the difference in the angle of the center of mass. These are the center coordinates in the k (horizontal or vertical) direction associated with the symbols j and i, respectively; These are the dimensions (such as width or height) of symbols i and j in the k direction, respectively. Let i and j represent the areas, respectively; It is the arctangent function; These represent the center x-coordinates of symbols i and j, respectively; These represent the center ordinates of symbols i and j, respectively.

[0105] The next step is to put all ( , , , Using as input, the DBSCAN clustering algorithm is employed. The conceptual diagram of this algorithm is shown below. Figure 3 As shown, it can adapt to the layout analysis requirements of this invention.

[0106] The basic concepts of this algorithm are as follows:

[0107] ε-neighborhood: The neighborhood of an object within a radius of ε. The area within.

[0108] Core object: if symbol of -The neighborhood contains at least One sample, i.e. ,but It is a core object.

[0109] Noise point: An object that does not belong to any cluster is called a noise point.

[0110] Cluster: A density-based cluster is the set of all objects that are connected by the highest density.

[0111] In the application of this invention, an optical recognition method is used to roughly determine the average object radius of the character. Parameters, core objects The parameters are set to 1-4, corresponding to character group (word) clusters, and noise points are marked as single characters. Figure 3 In the diagram, a dashed circle represents a cluster. , , , These are the core objects of the four clusters.

[0112] S2.2 For each cluster, calculate the set of horizontal distances between all symbol pairs within the cluster. With vertical spacing set Take the set of horizontal spacings The median is the horizontal spacing of the symbol pairs. Take the set of vertical spacings The median is the vertical spacing of the symbol pairs. If the number of symbols in the cluster is ≥3, then the horizontal spacing set is removed. With vertical spacing set The deviation from the median is >1.5× The word clusters are obtained by using the symbols of outliers, the median is recalculated, the median is corrected, and the horizontal spacing set is recalculated based on the corrected median. With vertical spacing set IQR stands for interquartile range.

[0113] For each cluster, count the set of horizontal spacings between all symbol pairs within the cluster. With vertical spacing set ;Pick The median is the horizontal spacing between characters / words. ,Pick The median is the vertical spacing between characters / words. If the number of symbols in a cluster is ≥3, then remove it. , The deviation from the median is >1.5× Outliers identified by the interquartile range (IQR) method are recalculated and the median is corrected. This method is more effective for pixel-level processing in character clustering tasks.

[0114] Specifically, for the set of horizontal spacing and vertical spacing set In this dataset, the median is first determined, and then the deviation of each data point from the median is calculated. If a data point deviates from the median by more than 1.5 times the interquartile range, it is considered an outlier and removed. This is to reduce the interference of outliers on subsequent statistical analyses such as median calculation, ensuring a more accurate horizontal spacing. and vertical spacing It better reflects the central tendency of the data, making subsequent processing based on these intervals (such as cluster-related operations) more accurate.

[0115] S2.3 When merging word clusters that meet preset conditions to form text blocks, the preset conditions include a first merging condition and a second merging condition: the first merging condition is that the horizontal overlap ratio between word clusters is >50% and the vertical center difference is <1 / 2 of the median height of the word clusters. Word clusters that meet the first merging condition are merged into text lines, and the centroid coordinates of each text line are calculated. The second merging condition is used to merge text lines into text blocks, and the specific determination logic is as follows:

[0116] A1, calculate the vertical distance of all adjacent text line centroids, take the median of all vertical distances as the standard line spacing ; the calculation logic is:

[0117] Subtract the vertical coordinate y value of the geometric center of the next line from the vertical coordinate y value of the geometric center of the previous line, and take the absolute value, which is the vertical distance of the two line centroids. The formula can be understood as .

[0118] A2, for any two text lines to be determined, calculate the actual vertical distance d of the centroids of the two text lines, and calculate the overlap ratio of all pixels in the vertical direction of the two text lines to be determined;

[0119] A3, when the actual vertical distance d of the two text lines to be determined satisfies , and the overlap ratio of all pixels in the vertical direction is > 20%, determine that the two text lines are associated text lines, and merge the two text lines to be determined into the same text block;

[0120] A4, repeat step A3 until all text lines that meet the association condition are merged to form independent text blocks.

[0121] The above process can be simply understood as: if the clustering output is a word-level cluster, traverse the word cluster, and merge adjacent word clusters with a horizontal direction overlap ratio > 50% and a vertical center difference < 2.5* (the vertical distance of the two text lines to be determined) into text lines; the formula for the overlap ratio of all pixels in the vertical direction is: ;

[0122] is the highest coordinate value in the vertical direction of the text line, is the lowest coordinate value in the vertical direction of the text line.

[0123] Calculate the median of the vertical distance of the centroids of adjacent line clusters as the line spacing . Calculate the vertical distance between all text lines , take a tolerance of 30%, if satisfies (i.e. 30% tolerance range) and the horizontal projection overlap ratio > 20%, merge the corresponding lines into the same text block, for each text block, calculate the minimum bounding rectangle vertex set of all symbols, output the four corner point coordinates to form a text box for subsequent processing.

[0124] In some embodiments, step S3 mainly includes S3.1. and S3.2, S3.1 is to obtain the center point coordinates of the text box, and S3.2 is to determine whether two text boxes are the same line of text according to the difference between the vertical coordinates of the center points of adjacent text boxes.

[0125] ​The specific manner of S3.1 is as follows:

[0126] S3.1.1, the four vertices of the minimum circumscribed quadrilateral corresponding to the text box are recorded as from the lower left corner in clockwise order, and an area weight coefficient is constructed , ;

[0127] S3.1.2, the center point coordinates of the text box are calculated , the horizontal coordinate and the vertical coordinate of the center point coordinates are respectively , and the calculation formula of the center point coordinates is: .

[0128] represents the coordinate value of the i-th vertex in the k direction. The meaning of is: the subscript k represents the coordinate axis direction, and the value is x or y. The subscript i represents the i-th vertex of the quadrilateral (numbered clockwise from the lower left corner, which is mentioned above i=1, 2, 3, 4). Therefore, is the horizontal coordinate (x coordinate) of the i-th vertex, and is the vertical coordinate (y coordinate) of the i-th vertex. Simplified as, is the x y coordinate value of the i-th vertex of the quadrilateral in the k direction.

[0129] First, each detected text box in the image is analyzed in detail, and the text box is most likely an irregular quadrilateral. The specific manner of calculating the coordinate position of the center point is to record the four vertices of the quadrilateral as from the lower left corner in clockwise order, and an area weight coefficient is constructed , which makes the long diagonal contribute more and suppresses the center drift caused by perspective distortion.

[0130] This step is to determine the relative position of the text box in the image, which provides basic data for subsequent clustering and sorting.

[0131] In some embodiments, the specific process of S3.2 is as follows:

[0132] S3.2.1, calculate the vertical projection span T of each text box;

[0133] S3.2.2, calculate the dynamic threshold , , is an empirical parameter, is the q-th text box and the q-1-th text box, respectively;

[0134] S3.2.3, When the difference in the ordinates of the center points of two adjacent text boxes When the two adjacent text boxes are determined to be on the same line of text, These represent the ordinates of the center points of the q-th and (q-1)-th text boxes, respectively.

[0135] Calculate the projected span of the quadrilateral in the vertical direction. Based on the center point sequence obtained in the previous step Define dynamic thresholds The decision to group all text boxes onto the same line.

[0136] Determined by the horizontally adjacent text boxes, the difference in the ordinates of the center points of two adjacent detection boxes is less than the vertical length of the text box. times, of which Further fine-tuning based on the statistical variance of the dataset is needed, but the empirical constant was confirmed to be [value missing] when this method was applied. At this time, it can usually meet the resolution robustness requirements of most document images:

[0137] The judgment criteria are: At this point, the two detection boxes will be determined to belong to the same line of text.

[0138] In some embodiments, the specific process of S4 is as follows:

[0139] S4.1 Calculate the average ordinate of all text boxes belonging to the same line of text. And based on the average ordinate Adjust the vertical position of all text boxes belonging to the same line of text so that the bottom left pixel of all text boxes belonging to the same line of text is horizontally aligned. Where N is the number of text boxes. This represents the area of ​​the q-th text box. This represents the y-coordinate of the center point of the q-th text box;

[0140] S4.2 Calculate the slope of a single text box, and calculate the tilt angle of the single text box based on the slope of the single text box. The formula for calculating the slope of a single text box is: , ,in, Represents the pixel vertex of the bottom left corner of a single text box. The coordinates; ) indicates the relationship with the vertex The vertex with the furthest horizontal distance If the tilt angle of the single frame greater than a set first threshold value, the single text box will be rotated with the pixel vertex at the center, an angle of the single text box is calculated.

[0141] After the clustering of the text lines is completed, the detection box of each text line is further processed. The average longitudinal coordinate of each text line is calculated, and the N boxes are used to calculate their respective areas When the weight:

[0142] ;

[0143] Horizontal alignment is performed on the basis of this, and this manner can make the horizontal line component of the text box with a large area greater, and the entire line more stable. The result of the horizontal alignment can ensure the consistency of the text line in the horizontal direction.

[0144] Then, the text content needs to be rotated and corrected, and the text box with an excessively large inclination angle needs to be corrected. The inclination angle is obtained by calculating the ratio of the height to the width of the text box, and then calculating the inverse tangent value, with the pixel vertex at the lower left corner as the reference point.

[0145] First, the vertex with the greatest horizontal distance is calculated. Then, the vertical drop of the vertex to is calculated, and then the slope of the single box is calculated and the inverse tangent value is obtained to obtain the inclination angle .

[0146] The calculation formula is as follows, wherein is the longitudinal coordinate of the point:

[0147] ;

[0148] If the inclination angle exceeds 3°, then the text content is rotated and corrected, with the pixel vertex at the center, an angle of the box is rotated to be parallel to the edge, and the line segment extending from the lower left corner of the text box is preferentially ensured to be a right angle parallel to the edge of the image, preliminary calibration is performed, and the number of subsequent rectangular divisions is minimized.

[0149] However, at this time, the edge of the text box is not smooth due to the pixels, and simple rotation cannot ensure that the edge of the rotated rectangular box is consistent in the horizontal / longitudinal direction, and 1-3 pixel errors may occur, and anti-aliasing and edge smoothing need to be performed.

[0150] Therefore, anti-aliasing and edge smoothing are implemented through step S5.

[0151] ​In some embodiments, the specific process of S5 is:

[0152] S5.1, enlarge the rotated text box as a whole to an integer multiple of 4, then establish an integer grid composed of several cells from the lower left corner with a step size of 4 pixels; and set the lower left corner cell as the origin for numbering, marked as ;

[0153] S5.2, starting from the origin cell , merge adjacent cells with consistent content properties in the order of right first and then up to obtain the largest cell, the merging rule includes:

[0154] S5.2.1, horizontal priority: if the cell has the same attribute as the cell , first horizontally merge the cell and the cell to obtain a horizontal merged cell group, , and represents a right shift of one cell, represents a shift up by one cell;

[0155] S5.2.2, vertical second: if the attribute of the previous cell of each cell in the horizontal merged cell group is consistent with the attribute of the horizontal merged cell group, merge each cell in the horizontal merged cell group with the corresponding previous cell;

[0156] S5.2.3, attribute consistency determination: if the gray histogram of the pixels in the cell has only one main peak value, the cell is a solid block , otherwise it is a colorless block ;

[0157] S5.3, record each largest cell as in the order of the merging rule, then take the origin largest cell connected by the lower left pixel after rotation correction as the reference, perform reverse splicing in the order of vertical first and horizontal second to obtain the preliminary combined text block;

[0158] S5.4, gap filling processing of the preliminary combined text block: for each horizontal gap, take the most edge column of pixels of the adjacent small blocks on the left and right for Gaussian blur filling; take one row of pixels above and below for the same filling for vertical gaps; after gap filling, the final standard rectangular text block with straight edges and no misalignment is obtained, and the layout reorganization of the original pixel information is completed.

[0159] Since the traditional anti-aliasing algorithm and edge smoothing algorithm will destructively repair the characters inside the text box, it is difficult to guarantee the original character shape information. Therefore, the image is cut by using the maximum rectangular block method for the inside of each rectangular box.

[0160] Firstly, the rotated text box is enlarged to an integer multiple of 4, and then an integer grid is established with a step of 4 pixels from the lower left corner. The lower left rectangle is set as the origin rectangle (0, 0) for numbering and marking as .

[0161] Then, the maximum rectangular growth is performed, that is, the maximum rectangle is cut out, and from , the adjacent and content attribute consistent are merged into larger rectangles in the order of "right first and then up". The merging rules are as follows

[0162] 1. Horizontal priority: if and attribute consistent, expand right first;

[0163] 2. Vertical second: if the whole row can be merged, expand up to ;

[0164] 3. Attribute consistency determination: The gray histogram of the inner pixel has only one main peak value for pure color block , otherwise for no color block .

[0165] According to the maximum rectangular generation order, each maximum rectangle is recorded as , that is, the maximum cell, wherein is the horizontal order, is the vertical order. Then, the large rectangle connected by the left lower corner pixel after rotation correction is taken as the reference, and the reverse splicing is performed in the vertical priority (aligning the left side) and horizontal second (aligning the bottom side), so that the split maximum rectangle is recombined into a new standard rectangular text box.

[0166] However, during the operation of rotation → splitting → reference splicing, due to the fact that the pixels do not completely fall in the integer grid, combined with the image quality factors, there will be 1-2 pixel gaps between the maximum rectangular blocks during splicing. Although it is almost invisible to the naked eye, in order to identify the downstream task, the gap filling needs to be performed. For each horizontal gap, the most edge column pixels of the left and right adjacent small blocks are taken for Gaussian blur filling, and for the vertical gap, the same filling is performed by taking one row of pixels above and below. After filling the gap row, the final standard rectangular text block with straight edges and no misalignment is obtained, and the layout reorganization of the original pixel information is completed.

[0167] The text box after S5 is shown as Figure 4

[0168] The maximum rectangle generation order is recorded as The core is to assign a unique coordinate to each merged maximum rectangle block (block), where the meaning of i and j needs to be understood in combination with the "right first and then up" merging order:

[0169] Generation order: Continue the "right first and then up" merging logic.

[0170] The generation of the maximum rectangle starts from the origin cell , and follows the rule of "first horizontal expansion, then vertical expansion", and the generation order is as follows:

[0171] First step: First process the lowest horizontal layer (j=0 row of mergable cell group), generate horizontal maximum rectangle from left to right (such as first merge to form the first block, and then process the right side to form the second block).

[0172] Second step: Process the upper horizontal layer (j=1 row of mergable cell group), also generate horizontal maximum rectangle from left to right (such as merge to form the third block);

[0173] Continue in this way until all cells are merged into blocks.

[0174] Coordinate definition of (i, j):

[0175] i (horizontal order): In the same vertical height (i.e. the same j value corresponding to the horizontal layer), the block is arranged from left to right, the first one from left is i=0, and then increases;

[0176] j (vertical order): The block is arranged from bottom to top in the horizontal layer, the lowest horizontal layer is j=0, and then increases;

[0177] Example: If j=0 layer (the lowest layer) generates 2 blocks from left to right, respectively (left), (right); j=1 layer (above j=0) generates 1 block from left to right, recorded as , then the coordinates of all blocks clearly reflect their horizontal position and vertical height.

[0178] "Reverse splicing" is not "splicing against the generation order", but to rotate the lower left corner of the text box ​To fix the reference, according to the rule of "longitudinal priority → transverse second", other blocks are spliced around the reference to finally form a standard rectangle, the specific steps are as follows:

[0179] First step: determine the reference, The special status:

[0180] is "a large rectangle connected by the left lower corner pixels after rotation correction", that is, it is the largest rectangle containing the origin cell It is also the block whose "left lower corner position coincides with the left lower corner of the original text box", so it is selected as the splicing reference, which plays the role of:

[0181] Fixing the "left lower corner anchor point" of the spliced standard rectangle, ensuring that the final text box position is aligned with the core area of the original rotated text box, avoiding overall deviation;

[0182] Providing "alignment reference" for other blocks, the splicing position of all blocks is calculated based on the edge of .

[0183] Second step: perform splicing, "longitudinal priority (align left side) → transverse second (align bottom side)".

[0184] The splicing logic needs to combine the coordinates (i, j) of the block and be performed in the order of "splicing the same column first, then splicing the same row", to ensure that there is no misalignment and no gap after splicing:

[0185] Longitudinal priority (align left side): splice blocks with the same i value, which represents "horizontal position same" (i.e. left and right alignment) blocks, and need to be spliced from bottom to top (j=0→j=1→j=2...) according to j value, and the "left side" of all blocks must be aligned with the left side of .

[0186] Calculate the vertical distance between the bottom edge of and the top edge of , and seamlessly connect the bottom edge of to the top edge of , while ensuring that the left side edges are completely aligned (no horizontal deviation);

[0187] If there is a block of i=0, j=2, continue to connect its bottom edge to the top edge of , and the left side edge is still aligned with the left side edge of , finally forming a longitudinal continuous rectangular strip of "i=0 column".

[0188] Next, horizontally (aligning the bottom edge): Blocks with the same j value are joined. The same j value represents blocks with the same vertical height (i.e., vertically aligned). They must be joined from left to right according to the i value (i=0→i=1→i=2...), and the bottom edge of all blocks must be aligned with the reference bottom edge of the same j value.

[0189] Taking layer j=0 as an example, the baseline is The bottom edge (coinciding with the bottom left corner of the original text box);

[0190] Find other blocks with the same j=0 (e.g.) (i=1), calculate its left side and The horizontal distance on the right side will The left side and The right side of each is seamlessly joined, while ensuring that the bottom edges of both are perfectly aligned (with no vertical offset).

[0191] If there are still blocks with j=0 and i=2, continue to connect their left edges to... The right side is aligned, while the bottom side remains aligned. The bottom edge eventually forms a continuous horizontal rectangular strip with "j=0 row".

[0192] Repeat the splicing process to form a standard rectangle. Following the order of "first splice column i=0 vertically → then splice row j=0 horizontally → then splice column i=1 vertically → then splice row j=1 horizontally", after all the blocks are spliced ​​together, the originally scattered blocks will form a rectangle with "neat left side, neat bottom side, and no protrusions or depressions". This is the final "new standard rectangular text box".

[0193] Assume there are 3 blocks generated: (12 pixels wide, 8 pixels high, bottom left corner coordinates (0,0)) (12 pixels wide, 8 pixels high) (12 pixels wide, 8 pixels high), the stitching steps are as follows:

[0194] Prioritize merging columns i=0: bottom edge and The top edge (y=8) is aligned with the left edge (x=0), and the column i=0 forms a rectangular bar with a width of 12 pixels and a height of 16 pixels (coordinates (0,0)-(12,16)).

[0195] The horizontal row with j=0 is next: The left side and The right side (x=12) is aligned with the bottom edge, and the bottom edge is aligned with y=0. At this time, the row j=0 forms a rectangular strip with a width of 24 pixels and a height of 8 pixels (coordinates (0,0)-(24,8)).

[0196] Final stitching result: three blocks form a standard rectangle of "24 pixels wide, 16 pixels high" (coordinates (0, 0)-(24, 16)), without any gaps or misalignment.

[0197] The essence of this process is to solve the problem of "irregular edges of the text box after rotation". By splitting the text box into blocks with uniform properties and then aligning and stitching them with fixed reference points, the "convex, concave, and inclined edges" of the original text box are forcibly eliminated, and a standard rectangular text box with straight edges suitable for OCR recognition is finally output.

[0198] S6 outputs the corrected image and structured JSON data, providing a basis for subsequent information position calibration. By using an OCR tool to recognize the text in the corrected image and utilizing the integrated structured JSON data, which stores the number, lower-left corner coordinates, and text box size data of each text box, the recognized text content can be calibrated to the correct position recorded in the JSON data. This way, the new text generated can simultaneously retain the accurate document position information recorded by this method and the content information recognized by the OCR technology, facilitating subsequent digital archiving, storage, and other business needs.

[0199] Embodiment 2 discloses a multi-level text correction system based on document layout analysis, which is used to execute the multi-level text correction method based on document layout analysis described in Embodiment 1, as shown in Figure 5 , which includes:

[0200] An image preprocessing module is used to determine the type of the image to be corrected by combining a multi-scale self-similarity feature algorithm and a directional frequency domain peak feature algorithm, and to obtain a standardized image by adaptively preprocessing the image to be corrected according to the determined image type.

[0201] A text layout analysis module is used to extract text connected domains from the standardized image, cluster each symbol in the text connected domains using unsupervised clustering technology to obtain a plurality of word clusters, merge word clusters that meet a predetermined condition to form at least one text block, and obtain the minimum circumscribed quadrilateral of each text block to obtain the corresponding text box.

[0202] A text box level processing module is used to obtain the center point coordinates of each text box, and determine whether two text boxes are in the same row according to the difference in the vertical coordinates of the center points of adjacent text boxes.

[0203] An out-of-box row level processing module is used to perform horizontal alignment and tilt correction on all text boxes belonging to the same row of text, so that all text boxes belonging to the same row of text are rotated to the same horizontal line.

[0204] an in-frame character-level processing module for performing morphological normalization on the characters in the rotated text box, so that the area where the characters are located forms a standard rectangular text block with a neat edge and no misalignment, to avoid the characters' morphology being destroyed due to anti-aliasing or smoothing processing;

[0205] a processing image and position information structured integration output module for outputting the corrected image and the structured JSON data.

[0206] The following is an example of using the present application to assist in text correction of medical information picture document layout analysis. The embodiment is intended to demonstrate the key processes and results of the application in real-world scenarios.

[0207] 1. Text layout analysis:

[0208] Text layout analysis is performed on the image to determine the original structure of the text. Binary conversion is used to convert the characters, and then unsupervised clustering techniques are used to cluster the adjacent characters. The adjacent character group (word) clustering relationship obtained is plotted in the original image as shown in Figure 6 . Subsequently, the results of the text line relationship analysis of the interline character group (word) clustering are plotted in the original image as shown in Figure 7 .

[0209] 2. Text box processing:

[0210] First, the text box is drawn according to the text layout analysis using the moire-removed picture, and the part of the original picture containing all the text information is extracted separately, as shown in Figure 8 .

[0211] 3. Character-level / line-level processing:

[0212] Using the method proposed in the present application, the character group (word) clustering relationship obtained by document layout analysis is used to perform line-level alignment and rotation correction on the text block, so that the points at the lower left corners of the same row boxes are in a right angle aligned on a horizontal line, as shown in Figure 9 .

[0213] However, due to the small angle and pixel of the picture, the actual observation slope is caused by the jaggedness of the rectangular text block, and it cannot be corrected by rotation. At this time, the method proposed in the present application is used to perform internal rectangular blocking on the text block, as shown in Figure 10 .

[0214] Subsequently, the method proposed in the present application is used to correct and fill the internal rectangular text block with Gaussian blur, and the text block is corrected to a neat rectangle from the character level, as shown in Figure 11as shown.

[0215] The overall image processing result after OCR extraction is as shown in the following figure Figure 12 As indicated, the structure is clear and the content is complete, greatly facilitating the subsequent storage of the persistent electronic document.

[0216] The present application creatively proposes a set of text line detection and structured reconstruction technology system for complex scenes, and the core innovation point is that through a multi-level adaptive processing mechanism, the systematic problems in image type discrimination, text line clustering, geometric distortion correction and pixel-level reconstruction in traditional methods are solved. The specific innovations include the following:

[0217] 1. Image type adaptive preprocessing mechanism:

[0218] Fusion of multi-scale self-similarity features (scaling structure stability) and directional frequency domain peak features (Moiré pattern detection), through the construction of joint criteria of quantitative indicators S (scale stability mean) and T (directional projection peak), the limitations of traditional single image processing are avoided, and the type-optimized standardized picture is provided for the subsequent modules.

[0219] 2. Unsupervised driven text layout reconstruction technology:

[0220] Based on connected domain DBSCAN clustering, the character-word-line three-level architecture is completed, and the interval median dynamic reduction algorithm is used: based on the area-weighted center point vertical coordinate and the projection span T, combined with the empirical threshold λ = 1 / 5 , unsupervised line aggregation is realized. According to the centroid coordinate sorting, the vertical coordinate sorting and horizontal alignment of the text box center point are realized to realize the geometric consistency reconstruction of the inclined text line. The problems of line segmentation and character distortion drift in non-uniform typesetting text in traditional rule methods are solved.

[0221] 3. Pixel-preserving text line geometric correction technology:

[0222] The rotated text box is divided into a sub-rectangular grid, and the left lower corner is used as the reference to translate and splice in order, avoiding pixel misplacement caused by rotation; for the 1-2 pixel missing caused by block splicing, non-destructive Gaussian blur is used for filling, which maintains the integrity of the character structure while eliminating the jaggedness. Traditional methods cannot handle pixel-level tilt due to global rotation, and the present application realizes the compatibility of rotation correction and original pixel preservation through block translation + local filling.

[0223] 4. Position-content double closed-loop structured output system:

[0224] JSON-OCR bimodal alignment technology: write the left lower corner reference coordinates of the text line correction and the size of the circumscribed rectangle into the JSON metadata, and bind the space position with the OCR recognition result; break through the limitations of the structured OCR caused by the original image, and construct an integrated framework of geometric layout and semantic content to provide a reconstruction basis for structured documents such as certificates and contracts.

[0225] The above specific embodiments further illustrate the purposes, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A multi-level text correction method based on document layout analysis, characterized by, The method comprises: determining the type of the image to be corrected in combination with a multi-scale self-similarity feature algorithm and a directional frequency domain peak feature algorithm, and adaptively pre-processing the image to be corrected according to the determined image type to obtain a standardized image; extracting text connected domains in the standardized image, clustering each symbol in the text connected domains by using an unsupervised clustering technique to obtain a plurality of word clusters, merging word clusters satisfying a preset condition to form at least one text block, and obtaining the minimum circumscribed quadrilateral of each text block to obtain a corresponding text box; wherein the specific process of clustering each symbol in the text connected domains by using the unsupervised clustering technique to obtain a plurality of word clusters is as follows: setting the unsupervised clustering technique as a DBSCAN clustering algorithm, constructing a 4-dimensional feature vector with the horizontal spacing, vertical spacing, area ratio and centroid angle difference between any two adjacent symbols when clustering each symbol in the text connected domains by using the DBSCAN clustering algorithm, setting the clustering neighborhood radius ε and the minimum sample number MinPts, and when the feature vector distance of two adjacent symbols is ≤ε and the number of samples in the neighborhood is ≥MinPts, the two symbols are classified into the same clustering cluster; For each of the cluster, count the horizontal distance set between all symbol pairs in the cluster and the vertical distance set ; take the median of the horizontal distance set as the horizontal distance of the symbol pair , take the median of the vertical distance set as the vertical distance of the symbol pair , if the number of symbols in the cluster is ≥ 3, eliminate the symbol pairs whose horizontal distance and vertical distance deviate from the median > 1.5 x IQR , take the symbol pairs deviating from the median as outliers to form a word cluster, recalculate the median, correct the median and recalculate the horizontal distance set based on the corrected median and the vertical distance set , IQR is interquartile range ; obtaining the center point coordinates of each text box, and determining whether two text boxes are in the same row according to the difference in the center point vertical coordinates of adjacent text boxes; performing horizontal alignment and tilt correction on all text boxes belonging to the same row of text, so that all text boxes belonging to the same row of text are rotated to the same horizontal line; performing morphological regularization processing on the characters in the rotated text boxes, so that the regions where the characters are located form standard rectangular text blocks with straight edges and no misalignment, to avoid the destruction of character morphology due to anti-aliasing or smoothing processing; outputting the corrected image and structured JSON data.

2. The multi-level text correction method based on document layout analysis according to claim 1, wherein, The multi-scale self-similarity feature algorithm realizes image scaling stability judgment by calculating the structural similarity index SSIM of adjacent scales of the image to be corrected. When the structural similarity index is ≥0.7, it is determined that the image to be corrected is a high-quality digital document picture. When the structural similarity index is <0.7, it is determined that the image to be corrected is a low-quality photo. The directional frequency domain peak feature algorithm realizes screen camera picture recognition by detecting the peak values of the horizontal direction and the vertical direction in the frequency domain of the image to be corrected. When the peak values of the horizontal direction or the vertical direction in the low frequency and the medium frequency exceed the threshold value, it is determined as a screen camera picture.

3. The multi-level text correction method based on document layout analysis according to claim 1, wherein, When adaptively pre-processing the image to be corrected, if it is determined as a screen camera picture or a low-quality photo, a hybrid algorithm based on local extreme value and frequency energy distribution is used in combination with a gamma curve to perform noise removal operation, and then the image to be corrected after noise removal operation is standardized to obtain a standardized image.

4. The multi-level text correction method based on document layout analysis according to claim 1, wherein, The text blocks are formed by merging the word clusters satisfying the preset conditions, and the preset conditions include a first merging condition and a second merging condition: the first merging condition is that the horizontal direction overlap ratio between the word clusters is greater than 50% and the vertical center difference is less than 1 / 2 of the median of the word cluster height, the word clusters satisfying the first merging condition are merged into text lines, and the centroid coordinates of each text line are calculated; the second merging condition is used to merge the text lines into text blocks, and the specific determination logic is as follows: Calculate the vertical distance of all adjacent text line centroids, take the median of all vertical distances as the standard line spacing ; For any two text lines to be determined, the actual vertical distance d of the centroids of the two text lines is calculated, and the overlap ratio of all pixels of the two text lines in the vertical direction is calculated; When the actual vertical distance d of two text lines to be determined satisfies , and the overlapping ratio of all pixels in the vertical direction is >20%, the two text lines are determined as associated text lines, and the two text lines to be determined are merged into the same text block. The last step is repeated until all text lines satisfying the association condition are merged into independent text blocks.

5. The multi-level text correction method based on document layout analysis according to claim 1, wherein, The specific way of obtaining the center point coordinate of the text box is: four vertices of a minimum circumscribed quadrilateral corresponding to the text box are recorded clockwise from the lower left corner as , an area weight coefficient is constructed , ; Calculate the coordinates of the center point of the text box. The coordinates of the center point The x-coordinate and y-coordinate are respectively The coordinates of the center point The calculation formula is: , This represents the coordinate value of the i-th vertex in the k-direction.

6. The multi-level text correction method based on document layout analysis according to claim 1, wherein, The specific process of determining whether the adjacent text boxes are in the same row of text according to the vertical coordinate difference of the center points of the adjacent text boxes is as follows: calculating a vertical projection span T of each text box; ; represents the coordinate value of the i-th vertex in the vertical direction; calculating a dynamic threshold based on two horizontally adjacent text boxes , , is an empirical parameter, is the qth text box and the q-1th text box, respectively; When the vertical coordinate difference of the center points of two adjacent text boxes is determined as the same row of text, respectively represent the vertical coordinate of the center point of the qth text box and the (q-1)th text box.

7. The multi-level text correction method based on document layout analysis according to claim 1, wherein, The specific process of performing horizontal alignment and tilt correction on all text boxes belonging to the same row of text so that all text boxes belonging to the same row of text are rotated to the same horizontal line is as follows: calculating an average ordinate of all the text boxes belonging to the same line of text and adjusting the vertical position of all the text boxes belonging to the same line of text so that the top of the left bottom pixel of all the text boxes belonging to the same line of text is aligned in the horizontal direction, and adjusting the vertical position of all the text boxes belonging to the same line of text so that the top of the left bottom pixel of all the text boxes belonging to the same line of text is aligned in the horizontal direction, where N is the number of text boxes, denotes the area of the qth text box, denotes the center point ordinate of the qth text box; Calculate the slope of a single text box, and calculate the tilt angle of the single text box based on the slope of the single text box. The formula for calculating the slope of a single text box is: , ,in, Represents the pixel vertex of the bottom left corner of a single text box. The coordinates; ) indicates the relationship with the vertex The vertex with the furthest horizontal distance The coordinates; if the tilt angle of the single frame If the value is greater than the set first threshold, it will be calculated based on the pixel vertex. Center, Angle Rotate the single text box.

8. The multi-level text correction method based on document layout analysis according to claim 1, wherein, The specific process of performing morphological regularization on the characters in the rotated text boxes so that the regions where the characters are located form standard rectangular text blocks with straight edges and no misalignment is as follows: The rotated text box is enlarged by an integer multiple of 4, and then an integer grid composed of a plurality of cells is established from the lower left corner with a step of 4 pixels; and the lower left corner of the cell is set as the origin for numbering and marked as ; From the origin cell Begin, in the order of right first and then up, combine the adjacent cells with the same content attribute to get the maximum cell according to the set merging rule, the merging rule includes: Horizontal priority: If the cell With cells With consistent attributes, first select the cells mentioned above. With cells Perform a horizontal merge to obtain a horizontally merged cell group. Let the coordinates be the cell coordinates, and This means shifting one cell to the right. This indicates shifting up one cell; Vertically second: if the properties of the previous cell of each cell in the horizontally merged cell group are consistent with the properties of the horizontally merged cell group, then each cell in the horizontally merged cell group is merged with the corresponding previous cell; Attribute consistency determination: if the histogram of the gray scale of the pixels in the cell has only one main peak, the cell is a solid color block , otherwise, it is a non-color block ​ According to the merging rule, each maximum cell is sequentially recorded as , and then the origin maximum cell connected with the left lower corner pixel after rotation correction is taken as a reference to perform reverse splicing in the vertical direction first and then in the horizontal direction, to obtain a preliminarily combined text block; Gap filling processing is performed on the preliminarily combined text blocks: for each horizontal gap, the most edge column of pixels of the left and right adjacent small blocks is taken for Gaussian blur filling; for a vertical gap, one row of pixels above and below is taken for the same filling; after gap filling, the final standard rectangular text blocks with straight edges and no misalignment are obtained, and the layout reorganization of the original pixel information is completed.

9. A multi-level text correction system based on document layout analysis for performing the multi-level text correction method based on document layout analysis of any one of claims 1-8, characterized in that, It comprises: An image preprocessing module, configured to determine the type of a to-be-corrected image by combining a multi-scale self-similarity feature algorithm and a directional frequency domain peak feature algorithm, and to perform adaptive preprocessing on the to-be-corrected image according to the determined image type to obtain a standardized image; A text layout analysis module, configured to extract text connected domains from the standardized image, to cluster each symbol in the text connected domains by using an unsupervised clustering technique to obtain a plurality of word clusters, to merge word clusters satisfying preset conditions to form at least one text block, and to obtain a minimum bounding rectangle of each text block to obtain a corresponding text box; A text box level processing module, configured to obtain the center point coordinates of each text box, and to determine whether two text boxes are in the same row of text according to the vertical coordinate difference of the center points of the adjacent text boxes; An out-of-box row level processing module, configured to perform horizontal alignment and tilt correction on all text boxes belonging to the same row of text so that all text boxes belonging to the same row of text are rotated to the same horizontal line. In-box character-level processing module, used for performing morphological normalization processing on the characters in the rotated text box, so that the area where the characters are located forms a standard rectangular text block with neat edges and no misalignment, to avoid the characters' morphology from being destroyed due to anti-aliasing or smoothing processing; Processing image and position information structured integration output module, used for outputting the corrected image and the structured JSON data.

Images