A scene curved text correction method based on trigonometric function mapping

Abstract

This invention provides a method for correcting curved text in a scene based on trigonometric function mapping, comprising: acquiring a text image to be detected and preprocessing the text image; dividing the preprocessed text image into blocks to obtain sub-image blocks; identifying curved text based on the obtained sub-image blocks; correcting the identified curved text based on trigonometric function mapping; and accurately correcting the curved text based on projection mapping, outputting the text recognition result. This invention can completely and accurately detect curved text regions in scene text images, especially for correcting detected curved Chinese text regions in complex scenes. Furthermore, this invention eliminates the need for training and labeling of the dataset, saving initial training costs, and allows for timely output of images, accelerating processing speed and improving the efficiency of curved text detection and recognition.

Description

Technical Field

[0001] This invention relates to the field of text detection and correction technology, and more particularly to a method for correcting curved text in a scene based on trigonometric function mapping. Background Technology

[0002] Methods for detecting and correcting curved text mainly include the following: detection methods based on convolutional neural networks, detection methods based on manual feature extraction and analysis, and correction methods based on TPS interpolation. The main ideas are as follows:

[0003] Text detection methods based on convolutional neural networks (CNNs) train multiple CNN models to obtain a trained network model. Then, the text image to be detected is input into the target CNN model to obtain text region information.

[0004] Traditional text detection methods primarily rely on manually designed features for feature extraction and analysis, such as using gradient information from character edges for text detection. The specific implementation involves: first, edge detection is performed; then, the thickness of character strokes is calculated based on the edge detection results to synthesize connected components; finally, information such as the color variance of the connected components is used to filter out text regions.

[0005] The TPS interpolation-based correction method deforms N points in a curved text image to their corresponding N points, using the interpolation method that minimizes image bending energy. This method results in a smoother overall image deformation, with less pronounced bumps and indentations, thus avoiding text distortion.

[0006] However, existing text detection and correction algorithms have some problems, as follows:

[0007] While convolutional neural network-based text detection methods can accurately detect text images with high contrast, low noise, and relatively simple scenes, they are insufficient for handling tilted, curved, or irregularly laid-out text, especially 3D perspective text and curved text which are common in natural scenes. This makes it difficult for such methods to meet the needs of practical engineering applications.

[0008] Traditional text detection methods rely on basic features such as edges, texture, and color to distinguish text regions. These methods are only suitable for simple scenarios where the text image background is uniform and the text edges are relatively clear. However, real-world text scenarios are often complex, with varied and complex shapes. Using only these basic features can lead to false positives and false negatives. The performance of these methods is now far behind that of modern convolutional neural network methods.

[0009] Text correction methods based on TPS interpolation can only correct curved text regions with small angles. For curved text regions with larger angles, the correction effect of this method is not ideal and cannot achieve the expected correction effect. Summary of the Invention

[0010] To address the aforementioned technical problems, this invention provides a method for correcting curved text in scenes based on trigonometric function mapping. This method can completely and accurately detect curved text regions in scene text images, and is particularly effective in correcting detected curved Chinese text regions in complex scenes.

[0011] The technical means employed in this invention are as follows:

[0012] A method for correcting curved text in a scene based on trigonometric function mapping includes:

[0013] Acquire the text image to be detected and preprocess the text image;

[0014] The preprocessed text image is divided into blocks to obtain sub-image blocks;

[0015] Based on the acquired sub-image patches, identify curved text;

[0016] Based on trigonometric function mapping, the identified curved text is corrected;

[0017] Based on projection mapping, the curved text is accurately corrected, and the text recognition result is output.

[0018] Furthermore, the acquisition of the text image to be detected and the preprocessing of the text image specifically include:

[0019] Input the text image to be detected, and use a differentiable binarization-based segmentation algorithm to distinguish between text regions and background;

[0020] ResNet is used as the backbone network to extract text features, and the adaptive threshold μ of each pixel is predicted by jointly optimizing the segmentation network with a differentiable binarization function, so that the network generates a binarized segmentation map P1.

[0021] Furthermore, the step of segmenting the preprocessed text image into sub-image blocks specifically includes:

[0022] Extract the connected regions from the binarized segmentation image P1 and add them to the text candidate image set C, labeled as C = c1, c2, ..., c v ,…,c V Where V represents the number of candidate images for the text;

[0023] Each text candidate image c in the text candidate image set C vDivide the image into two equal sub-image blocks, left and right, respectively denoted as... and

[0024] Furthermore, the step of identifying curved text based on the acquired sub-image patches specifically includes:

[0025] Calculate the text radians:

[0026] Sub-images via Radon and Transform the generated image R respectively l ρ, θ and R r ρ,θ, calculate sub-image and Textual perspective and The calculation formula is as follows:

[0027]

[0028]

[0029] From the perspective of the text and The formula for calculating the text in radians (rad) is as follows:

[0030]

[0031] Determine curved text:

[0032] Determining the text image c using radians (rad). v Whether it is curved is determined by setting the radius threshold to β, and the determination process is as follows:

[0033] If rad ≥ β, then determine the text image c. v To make the text curved, use c v The text is input into the curved text correction module for correction processing, and then into the text recognition network for recognition.

[0034] If rad < β, then determine the text image c. v For non-curved text, use c v It is directly input into a text recognition network for recognition.

[0035] Furthermore, the correction of the identified curved text based on trigonometric function mapping specifically includes:

[0036] For image c that is identified as curved text v Traverse the pixel coordinates i, j, i = 1, 2, ..., rows, j = 1, 2, ..., cols, where rows is the coordinates of the image c. v The height, cols is the image cv The width of the image c is calculated. v The offset P of each pixel coordinate i,j is calculated using the following formula:

[0037]

[0038] Where ω is a parameter that adjusts the size of the offset P;

[0039] Compare the pixel coordinate offset P with the image before correction c v The corrected image is obtained by adding the ordinates i of the pixels. v The pixel coordinate transformations of the image before and after correction are shown below:

[0040] r v i,j=c v i+P,j.

[0041] Furthermore, the precise correction of curved text based on projection mapping and the output of text recognition results specifically include:

[0042] Determine the image r before mapping v The coordinates of the four vertices x i ,y i Let i = 1, 2, 3, 4. The coordinates x′ of the four vertices of the mapped image are determined by parameters α and η. i ,y′ i If i = 1, 2, 3, 4, the mapped image is denoted as b. v ; Calculate the coordinates of the images before and after mapping using the following formula:

[0043] x′1=α×x2, y′1=y1

[0044] x′2=1-α×x2,y′2=y2α

[0045] x′3=1+α×x4, y′3=η×y3

[0046] x′4=-α×x4, y′4=η×y4

[0047] Calculate the original image r v Projection mapping to target image b v In the projection transformation formula, the parameter m i i = 1, 2Λ, 8:

[0048]

[0049] When n=4, use the images r before and after mapping. v and b v The coordinates of the four vertices are used to calculate the parameter m in the projection transformation formula. ii = 1, 2Λ, 8, Substitute the parameters into the projection transformation formula:

[0050]

[0051] Original image r v Mapped to the target image b using the projection transformation formula v Perform precise correction of curved text;

[0052] The system outputs the text recognition results after the curved text is corrected. Based on the recognition results, the parameters of the curved text correction module are optimized to achieve the best correction effect.

[0053] Compared with the prior art, the present invention has the following advantages:

[0054] 1. The trigonometric function mapping-based method for correcting curved text in scenes provided by this invention can completely and accurately detect curved text regions in scene text images, especially for correcting detected curved Chinese text regions in complex scenes.

[0055] 2. The trigonometric function mapping-based method for correcting curved text in a scene provided by this invention does not require training and labeling of the dataset, saving the initial training cost, and can obtain output images in a timely manner, thus speeding up the processing and improving the efficiency of curved text detection and recognition.

[0056] Based on the above reasons, this invention can be widely applied in fields such as text detection and correction. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a flowchart of the method of the present invention.

[0059] Figure 2 The scene image text detection input and output images are provided for embodiments of the present invention.

[0060] Figure 2 In the image: (a) is the input image; (b) is the output image.

[0061] Figure 3 This invention provides a method for calculating angles and determining the curvature of text images.

[0062] Figure 3In the image: (a) is the image for calculating the angle of the text image; (b) is the image for determining the curved text image.

[0063] Figure 4 The curved text recognition result and corrected image provided in the embodiments of the present invention.

[0064] Figure 4 In the image: (a) is the output of the curved text recognition result; (b) is the trigonometric function mapping correction result image.

[0065] Figure 5 The coordinate transformation accuracy correction and text recognition results provided in the embodiments of the present invention.

[0066] Figure 5 In the image: (a) is the image of the coordinate transformation and accurate correction result; (b) is the output text recognition result after correction. Detailed Implementation

[0067] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0068] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0069] like Figure 1 As shown, this invention provides a method for correcting curved text in a scene based on trigonometric function mapping, including:

[0070] S1. Obtain the text image to be detected and preprocess the text image;

[0071] S2. Divide the preprocessed text image into blocks to obtain sub-image blocks;

[0072] S3. Based on the acquired sub-image blocks, identify curved text;

[0073] S4. Correct the identified curved text based on trigonometric function mapping;

[0074] S5. Based on projection mapping, accurately correct curved text and output text recognition results.

[0075] In a preferred embodiment of the present invention, step S1, which involves acquiring the text image to be detected and preprocessing the text image, specifically includes:

[0076] S11. Input the text image to be detected, and use a differentiable binarization-based segmentation algorithm to distinguish the text region from the background.

[0077] S12. ResNet is used as the backbone network to extract text features, and the adaptive threshold μ of each pixel is predicted by jointly optimizing the segmentation network with the differentiable binarization function, so that the network generates a binarized segmentation map P1.

[0078] In a specific implementation, as a preferred embodiment of the present invention, step S2 involves segmenting the preprocessed text image into sub-image blocks, specifically including:

[0079] S21. Extract the connected regions in the binarized segmentation image P1 and add them to the text candidate image set C, labeled as C = c1, c2, ..., c v ,…,c V Where V represents the number of candidate images for the text;

[0080] S22. Extract each text candidate image c from the text candidate image set C. v Divide the image into two equal sub-image blocks, left and right, respectively denoted as... and

[0081] In a specific implementation, as a preferred embodiment of the present invention, step S3, which involves determining curved text based on the acquired sub-image blocks, specifically includes:

[0082] S31. Calculate the text radians:

[0083] S311, Sub-image via Radon and Transform the generated image R respectively l ρ, θ and R r ρ,θ, calculate sub-image and Textual perspective and The calculation formula is as follows:

[0084]

[0085]

[0086] S312, From the perspective of the text and The formula for calculating the text in radians (rad) is as follows:

[0087]

[0088] S32. Determine curved text:

[0089] S321. Determine the text image c by using radians (rad). v Whether it is curved is determined by setting an arc threshold of β, where β is preferably 1.04. The determination process is as follows:

[0090] If rad ≥ β, then determine the text image c. v To make the text curved, use c v The text is input into the curved text correction module for correction processing, and then into the text recognition network for recognition.

[0091] If rad < β, then determine the text image c. v For non-curved text, use c v It is directly input into a text recognition network for recognition.

[0092] In a specific implementation, as a preferred embodiment of the present invention, step S4, based on trigonometric function mapping, corrects the identified curved text, specifically including:

[0093] S41. For image c that is determined to be curved text. v Iterate through the pixel coordinates i, j, i = 1, 2, rows, j = 1, 2, ..., cols, where rows is the coordinate of the image c. v The height, cols is the image c v The width of the image c is calculated. v The offset P of each pixel coordinate i,j is calculated using the following formula:

[0094]

[0095] Wherein, ω is a parameter that adjusts the size of the offset P; where ω is preferably 90.

[0096] S42. Compare the pixel coordinate offset P with the original image c. v The corrected image is obtained by adding the ordinates i of the pixels. v The pixel coordinate transformations of the image before and after correction are shown below:

[0097] r v i,j=c v i+P,j.

[0098] In a specific implementation, as a preferred embodiment of the present invention, step S5, based on projection mapping, accurately corrects the curved text and outputs the text recognition result, specifically including:

[0099] S51. Determine the image r before mapping. v The coordinates of the four vertices x i ,y i Let i = 1, 2, 3, 4. The coordinates x′ of the four vertices of the mapped image are determined by the parameter α. i ,y′ i The mapped image is denoted as b, where i = 1, 2, 3, 4. v The parameters α and y are verified experimentally, and their optimal values ​​are 0.2 and 0.8, respectively. The coordinates of the images before and after mapping are calculated using the following formula:

[0100] x′1=α×x2, y′1=α×y1

[0101] x′2=1-α×x2,y′2=y2

[0102] x′3=1+α×x4, y′3=η×y3

[0103] x′4=-α×x4, y′4=η×y4

[0104] S52, Calculate the original image r v Projection mapping to target image b v In the projection transformation formula, the parameter m i i = 1, 2Λ, 8:

[0105]

[0106] S53. When n = 4, use the images r before and after mapping. v and b v The coordinates of the four vertices are used to calculate the parameter m in the projection transformation formula. i i = 1, 2Λ, 8, Substitute the parameters into the projection transformation formula:

[0107]

[0108] Original image r v Mapped to the target image b using the projection transformation formula v Perform precise correction of curved text;

[0109] S54. Output the text recognition result after the curved text is corrected. Optimize the parameters of the curved text correction module based on the recognition result to achieve the best correction effect.

[0110] Example

[0111] To verify the effectiveness of the method of the present invention, simulation experiments were conducted, and the results are as follows: Figure 2-5 As shown, specifically: Figure 2 , 3 As shown, the text region detected using a rectangular bounding box can completely and accurately detect the text region in the scene image, extract curved text from the scene image, and calculate the text angle. Figure 4 , 5 As shown, by using calculated angles to perform multiple correction processes on curved text, the curved text image text can be completely and accurately identified.

[0112] In summary, the method of the present invention does not require training and labeling of the dataset, saving the initial training cost, and can obtain output images instantly, thereby speeding up the processing and improving the efficiency of curved text detection and recognition.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A scene warping text rectification method based on trigonometric function mapping, characterized in that, include: Acquire the text image to be detected and preprocess the text image; The preprocessed text image is divided into blocks to obtain sub-image blocks; Based on the acquired sub-image patches, identify curved text; Based on trigonometric function mapping, the identified curved text is corrected, including: Images identified as curved text Traverse pixel coordinates , , ,in It is an image height, It is an image Calculate the width of the image. Coordinates of each pixel offset The calculation formula is as follows: in, It is to adjust the offset. Size parameters; pixel coordinate offset Compared with the image before correction Pixel ordinate Add them together to get the corrected image, denoted as . The pixel coordinate transformations of the image before and after correction are shown below: Based on projection mapping, the curved text is accurately corrected, and the text recognition results are output, including: Determine the image before mapping The coordinates of the four vertices Through parameters and Determine the coordinates of the four vertices of the mapped image. The mapped image is denoted as ; Calculate the coordinates of the images before and after mapping using the following formula: Calculate the original image Projection mapping to target image Parameters in the projection transformation formula : when At that time, using the images before and after mapping and The coordinates of the four vertices are used to calculate the parameters in the projection transformation formula. Substitute the parameters into the projection transformation formula: Original image Mapped to the target image using the projection transformation formula Perform precise correction of curved text; The system outputs the text recognition results after the curved text is corrected. Based on the recognition results, the parameters of the curved text correction module are optimized to achieve the best correction effect.

2. The method for scene curvature text correction based on trigonometric function mapping according to claim 1, characterized in that, The acquisition of the text image to be detected and the preprocessing of the text image specifically include: Input the text image to be detected, and use a differentiable binarization-based segmentation algorithm to distinguish between text regions and background; ResNet is used as the backbone network to extract text features, and the adaptive threshold for each pixel is predicted by jointly optimizing the prediction with a differentiable binarization function and a segmentation network. This enables the network to generate a binary segmentation map. .

3. The method for scene curvature text correction based on trigonometric function mapping according to claim 1, characterized in that, The step of segmenting the preprocessed text image into sub-image blocks specifically includes: Extracting the binarized segmentation map Connected regions in the image are then added to the text candidate image set. In the middle, marked as ,in, The number of candidate images representing the text; Text candidate image set Each text candidate image Divide the image into two equal sub-image blocks, left and right, respectively denoted as... and .

4. The method for scene curvature text correction based on trigonometric function mapping according to claim 1, characterized in that, The process of identifying curved text based on the acquired sub-image patches specifically includes: Calculate the text radians: Sub-images via Radon and Transform the generated images respectively and Calculate sub-images and Textual perspective and The calculation formula is as follows: From the perspective of the text and Calculate the text radians The calculation formula is as follows: 。 Determine curved text: By radians Determine text image Whether it is curved, set the radius threshold. The judgment process is as follows: if Then determine the text image. For curved text, The text is input into the curved text correction module for correction processing, and then into the text recognition network for recognition. if Then determine the text image. For non-curved text, It is directly input into a text recognition network for recognition.

Images