A text correction method based on a deep convolutional neural network

Abstract

The application discloses a text correction method based on a deep convolutional neural network, and comprises the following steps: step 1, collecting image data of the text; step 2, training a model by using an edge detection neural network after preprocessing; step 3, processing the original text image according to the trained model obtained in step 2 to obtain an edge image; step 4, performing Hough voting on the edge image; step 5, screening the voting result to obtain four reasonable straight lines to form a text contour; and step 6, performing homography transformation on the detected quadrilateral contour to obtain the final corrected text.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of visual recognition, and relates to a text correction method based on a deep convolutional neural network. BACKGROUND

[0002] With the popularity of network technology and video communication, more and more students begin to learn on the network, and some teachers require students to upload homework by taking photos. Teachers in schools generally adopt network teaching, and homework is also uniformly uploaded by taking photos. However, due to the fact that students take photos in various ways, such as mobile phone level, mobile phone inclination, book rotation inclination and the like, the text forms in the photos are various. The originally rectangular homework paper is taken as a trapezoid, a parallelogram and the like, and it is difficult for teachers to read when correcting homework. Under this background, it is hoped that the homework of students can be detected, corrected and enhanced to facilitate teachers to read.

[0003] There are many straight line detection methods in traditional computer vision algorithms. For example, Hough straight line detection, LSD straight line detection and Canny Lines straight line detection. These algorithms basically perform straight line detection on an edge image. Common edge detection algorithm operators are, for example, a Canny operator and a Laplace operator. The edge image obtained by using these traditional algorithms has poor robustness in the case that the background is not ideal. For example, in the case that the text background is complex, Hough straight line detection will have many false detection straight lines; LSD can only detect long straight lines, and the text (paper) sometimes has a curled edge and other non-long straight line conditions, and thus some straight lines are missed; and Canny Lines also has a curled edge and misses some straight lines. In summary, these traditional algorithms can only be used in specific scenarios. SUMMARY

[0004] In view of the deficiencies of the prior art, the application provides a text correction method based on a deep convolutional neural network, so that the text detection problem that cannot be solved by traditional computer vision can be solved.

[0005] Technical scheme: The application discloses a text correction method based on a deep convolutional neural network, and the core lies in that various text images are collected, some occlusions are added to the images, a neural network model is trained based on the preprocessed text, an edge image is obtained, and then the edge image is processed by using a Hough voting algorithm to obtain four reasonable text edges. Then, homography transformation is performed on the four edges, and the corrected text is obtained. The method specifically comprises the following steps:

[0006] Step 1: Collecting text image data and pre-processing the data;

[0007] Step 2: Constructing a neural network model and training the model by using the collected data;

[0008] Step 3: After the training is completed, the original image is input into the network model to obtain an edge image;

[0009] Step 4: Hough voting is performed on the edge image obtained in step 3;

[0010] Step 5: The Hough voting result in step 4 is processed to obtain a quadrilateral;

[0011] Step 6: According to the four corner points of the quadrilateral in step 5, a homographic transformation is performed on the original image to obtain the corrected text.

[0012] Further, step 1 includes data collection and processing steps, including the following steps:

[0013] Text pictures are crawled on the network, and the crawled pictures are filtered to remove pictures that do not correspond to text, and desktop items are prepared as noise of text pictures to combine with text pictures at random positions;

[0014] Step 1-1: From all text images, select some images and randomly add some obstructions such as mouse, data line, pen and other common desktop items to obscure the text in the original image at random positions;

[0015] Step 1-2: Scale the images in the data set after step 2-1 to 256*256;

[0016] Step 1-3: Normalize the 256*256 image.

[0017] Further, step 2 includes a neural network model training step, including the following steps:

[0018] Step 2-1: Construct a neural network model;

[0019] Step 2-2: Initialize the neural network model using pre-trained network parameters;

[0020] Step 2-3: Train the neural network model with the square image data preprocessed in step 2, and save the trained model.

[0021] The specific content of the neural network model in step 2-1 is as follows: the model adopts RCF and is based on the VGG 16 network architecture. The model includes a five-level feature extraction architecture, and the specific structure is as follows: the input is an image with a size of 256*256*3; the first level includes two convolutional layers, the size of the convolutional kernel is 3*3, and the number of convolutional kernels is 64; the second level includes two convolutional layers, the size of the convolutional kernel is 3*3, and the number of convolutional kernels is 128; the third level includes three convolutional layers, the size of the convolutional kernel is 3*3, and the number of convolutional kernels is 256; the fourth level includes three convolutional layers, the size of the convolutional kernel is 3*3, and the number of convolutional kernels is 512; and the fifth level includes three convolutional layers, the size of the convolutional kernel is 3*3, and the number of convolutional kernels is 512. For each level, a convolutional layer is connected, the size of the convolutional kernel is 1*1, and the number of convolutional kernels is 21. For each level, the results of all 1*1 convolutions are connected to a convolution with a size of 1*1 and a number of convolutional kernels of 1, and then an inverse convolution is performed to the original picture size as the intermediate output of the level. Finally, the intermediate outputs of the five levels are concatenated, and a final convolution operation is performed on the result with a size of 1*1 and a number of convolutional kernels of 1 to obtain the final result.

[0022] Further, the Hough voting step in step 4 specifically includes:

[0023] Step 4-1, establish a parameter space. The key to Hough voting is to establish a parameter space, i.e., a Hough space. In polar coordinates, a straight line is determined by parameters θ and parameters ρ. The range of parameter θ is (0, π) and is divided into m units, and the range of parameter ρ is (-l, l) and is divided into n units. The parameter space has m*n units in total, where l is the length of the diagonal of the picture;

[0024] Step 4-2, establish a voting table with the same size m*n. Extract the edge pixels in the edge image obtained before, and the number of extractable pixels is denoted as s. For each pixel, find its m discrete points in the Hough space.

[0025] Step 4-3, collect all the discrete points in the Hough space corresponding to all the edge pixels, a total of s*m. Vote for all these discrete points in the voting table, and the candidate straight line with more votes can be obtained.

[0026] Further, in step 5, the voting results obtained in step 4 are processed. Specifically, it includes:

[0027] Step 5-1, using clustering method, the above candidate straight lines with more votes are divided into four categories of straight line clusters according to θ and ρ.

[0028] Step 5-2, for each straight line cluster Lines i , the straight line with the most votes is selected as Top i ;

[0029] Step 5-3, for each straight line cluster Lines i , the search for possible candidate straight lines is continued, but the votes of these straight lines must reach a certain threshold. Here, the number of candidate straight lines for each class is set to be no more than 3.

[0030] Further, in step 6, according to the four corner points of the quadrilateral contour in step 5, the coordinates of the corner points are restored to obtain 4 coordinate points in the original image. According to the 4 pairs of coordinate points, a homography transformation is performed on the original image to obtain the corrected text.

[0031] The present application has the following advantages:

[0032] 1) The method of the present application solves the text detection problem that cannot be solved by traditional computer vision, and reduces the error rate of text detection. Especially in the detection of the edge, the "pseudo edge" outside the text edge can be effectively filtered out.

[0033] 2) By collecting various text images and adding some occlusions to the images for labeling and training, the edge recognition accuracy of the text can be enhanced, and the algorithm is more robust.

[0034] 3) The present algorithm adopts the Hough voting algorithm for straight line detection, solves the problem of low confidence in straight line detection caused by discontinuous long straight lines, and enhances the detection effect of intermittent straight lines. BRIEF DESCRIPTION OF DRAWINGS

[0035] The above and other aspects of the present application will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings.

[0036] Figure 1 Flowchart of the text correction method based on deep convolutional neural network;

[0037] Figure 2 Schematic diagram of the RCF neural network used in the present application;

[0038] Figure 3 From left to right, they are the original image, the gray image of edge detection, the detected boundary of the text, and the corrected text. DETAILED DESCRIPTION

[0039] The present application will be further described below in conjunction with the drawings and examples.

[0040] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0041] Embodiment 1

[0042] The application discloses a text correction method based on a deep convolutional neural network, which is characterized in that various text images are collected, some occlusions are added to the images, a neural network model is trained based on the preprocessed text, an edge image is obtained, and then the edge image is processed by a Hough voting algorithm to obtain four reasonable text edges. Then, homography transformation is performed on the four edges, and the corrected text is obtained.

[0043] Specifically, the following steps are included:

[0044] Step 1: Collect text image data and pre-process the data;

[0045] Step 2: Construct a neural network model and train the model using the collected data;

[0046] Step 3: After the training is completed, input the original image into the network model to obtain an edge image;

[0047] Step 4: Perform Hough voting on the edge image obtained in Step 3;

[0048] Step 5: Process the Hough voting result in Step 4 to obtain a quadrilateral;

[0049] Step 6: Perform homography transformation on the original image according to the four corner points of the quadrilateral in Step 5 to obtain the corrected text.

[0050] In Step 1, data collection and processing include the following steps:

[0051] Step 1-1: From all the text images, select some images and randomly add some occlusions such as a mouse, a data cable, a pen and other common desktop items to occlude the text in the original image at random positions;

[0052] Step 1-2: Scale the images in the data set after Step 2-1 to 256*256;

[0053] Step 1-3: Normalize the 256*256 image.

[0054] In Step 2, the training of the neural network model includes the following steps:

[0055] Step 2-1, construct a neural network model;

[0056] Step 2-2, initialize the neural network model, and use the pre-trained network parameters;

[0057] Step 2-3, train the neural network model with the square image data preprocessed in step 2, and save the trained model.

[0058] The specific content of the neural network model in step 2-1 is as follows: as shown in the figure, Figure 2 The model uses RCF and is based on VGG 16 network architecture. The model includes a five-level feature extraction architecture. The specific structure is: the input is an image with a size of 256*256*3; the first level includes two convolutional layers, the size of the convolutional kernel is 3*3, and the number of convolutional kernels is 64; the second level includes two convolutional layers, the size of the convolutional kernel is 3*3, and the number of convolutional kernels is 128; the third level includes three convolutional layers, the size of the convolutional kernel is 3*3, and the number of convolutional kernels is 256; the fourth level includes three convolutional layers, the size of the convolutional kernel is 3*3, and the number of convolutional kernels is 512; the fifth level includes three convolutional layers, the size of the convolutional kernel is 3*3, and the number of convolutional kernels is 512. Each level of convolutional layer is connected with a convolutional layer, the size of the convolutional kernel is 1*1, and the number of convolutional kernels is 21. For each level, the results of all 1*1 convolution are connected with a convolution, the size of the convolutional kernel is 1*1, and the number of convolutional kernels is 1, and then a deconvolution is performed to the original picture size as the intermediate output of the level. Finally, the intermediate outputs of the five levels are concatenated, and the result is subjected to a final convolution operation, the size of the convolutional kernel is 1*1, and the number of convolutional kernels is 1, to obtain the final output.

[0059] The Hough voting algorithm in step 4 specifically includes:

[0060] Step 4-1, establish a parameter space. The parameter theta ranges from (0, pi) and is evenly divided into m units. The parameter rho ranges from (-l, l) and is evenly divided into n units. Therefore, the parameter space has m*n units in total. l is the length of the diagonal of the picture.

[0061] Step 4-2, establish a voting table with the same size m*n. Extract the edge pixels from the edge image obtained before, and the number of extractable pixels is denoted as s. For each pixel, find its m discrete points in the Hough space.

[0062] Step 4-3, collect all the discrete points in the Hough space corresponding to all the edge pixels, a total of s*m. Vote for all these discrete points in the voting table, and the one with more votes can be considered as a candidate straight line.

[0063] In step 5, the voting results obtained in step 4 are processed. Specifically, the following steps are included:

[0064] In step 5-1, the above-mentioned candidate straight lines with more votes are divided into four clusters of straight lines according to θ and ρ using a clustering method.

[0065] In step 5-2, for each cluster of straight lines Lines i , the straight line with the most votes Top i is selected.

[0066] In step 5-3, for each cluster of straight lines Lines i , the possible candidate straight lines are further searched, but the votes of these straight lines must reach a certain threshold, such as 0.5 times the votes of Top i . Here, the number of candidate straight lines in each cluster is limited to three.

[0067] In step 6, according to the four corner points of the quadrilateral in step 5, the coordinates of the corner points are restored to obtain four coordinate points in the original image. According to the four pairs of coordinate points, the original image is subjected to a homographic transformation to obtain the corrected text.

[0068] As shown in the results achieved according to the present application in Figure 3 , the leftmost is the text image to be corrected; the second leftmost is the gray-scale image obtained by edge detection; the third leftmost is the detected text boundary; and the fourth leftmost is the correction result.

[0069] There are many methods and approaches to achieve the technical solution provided by the present application. The above description is only the preferred embodiment of the present application. It should be noted that for ordinary technical personnel in the technical field, without departing from the principles of the present application, several improvements and refinements can be made, which should also be considered within the protection scope of the present application. The components not explicitly described in the present embodiment can be realized by existing technology.

Claims

1. A text correction method based on deep convolutional neural network, It is characterized in that The following steps are involved: Step 1: Collect text and image data and preprocess the data; Step 2: Build a neural network model and train it using the collected data; Step 3: After training is completed, the image to be detected is input into the network model to obtain the edge image; Step 4: Perform Hough voting on the edge image obtained in step 3; Step 5: Process the Hough voting results in step 4 to obtain a quadrilateral; Step 6: Based on the four corner points of the quadrilateral in step 5, perform homography transformation on the original image to obtain the corrected text. Step 1 includes: crawling the corresponding text images on the Internet, filtering the crawled images, removing images that do not correspond to the text, and preparing desktop objects as noise for the text images, combining them with the text images at random positions; Step 1-1: Select some images from all text images, randomly add some occluders to the text images to cover random parts of the text in the original images; Step 1-2: scale the images in the dataset after step 1-1 to a pixel size of 256*256; Steps 1-3, normalize the 256*256 image. Step 2 includes the following steps: Step 2-1, construct a neural network model; Step 2-2, initialize the neural network model and use the pre-trained network parameters; Step 2-3: Use the square image data preprocessed in step 1 to train the neural network model and save the trained model. Step 2-1 includes the following steps: Construct a neural network model, which uses RCF and is based on the VGG 16 network architecture. The model contains a five-level feature extraction architecture with the following structure: the input is an image of size 256*256*3; the first level includes 2 convolutional layers, the size of the convolution kernel is 3*3, and the number of convolution kernels is 64; the second level includes 2 convolutional layers, the size of the convolution kernel is 3*3, and the number of convolution kernels is 128; the third level includes 3 convolutional layers, the size of the convolution kernel is 3*3, and the number of convolution kernels is 256; the fourth level includes 3 convolutional layers, the size of the convolution kernel is 3*3, and the number of convolution kernels is 512; the fifth level includes 3 convolutional layers, the size of the convolution kernel is 3*3, and the number of convolution kernels is 512. The kernel size is 3*3, the number of convolution kernels is 512, and each level of convolution layer is connected to a convolution layer. The size of the convolution kernel is 1*1, and the number of convolution kernels is 21. For each level, all the results of 1*1 convolution are connected to a convolution. The size of the convolution kernel is 1*1, and the number of convolution kernels is 1. Then, deconvolution is performed to the size of the original image as the intermediate output of the level. Finally, all the intermediate outputs of the 5 levels are concat-operated, and the final convolution operation is performed on the result. The size of the convolution kernel is 1*1, and the number of convolution kernels is 1 to obtain the final result. Step 4-1, establish parameter space; The key to Hough voting is to establish a parameter space, namely the Hough space. In the polar coordinate system, a straight line is determined by the parameters θ and ρ. The range of parameter θ is (0, π), which is evenly divided into m units. The range of parameter ρ is (-l, l), which is evenly divided into n units. The parameter space has a total of m*n units, where l is the length of the diagonal of the image. Step 4-2: Create a voting table of the same size m*n. Extract the edge pixels from the edge image obtained previously. The number of pixels that can be extracted is denoted as s. For each pixel, find its m discrete points in the Hough space. Step 4-3: Collect all the discrete points in the Hough space corresponding to all the edge pixels. There are s*m discrete points in total. All the discrete points are voted in the voting table. The one with the most votes can be used as a candidate line. In step 4-3, the Hough voting results are obtained. Based on this, the following steps are performed: Step 5-1: Use the clustering method to divide the candidate lines with more votes into four major clusters based on θ and ρ; Step 5-2: For each line cluster Linesi, select the line Topi with the most votes; In step 5-3, for each line cluster Linesi, continue searching for possible candidate lines, but the number of votes for these lines must reach a certain threshold. Here, it is set that the number of candidate lines in each class cannot exceed 3; Step 5-4: Select one straight line from each of the four candidate straight lines obtained in step 5-3 and combine them. The result of the combination in step 5-5 must be a quadrilateral, and at least 3 intersection points must all appear inside the image; Steps 5-6 calculate the area of ​​all the quadrilaterals obtained, and the quadrilateral with the largest area is the final goal. The four corner points of the quadrilateral in step 5 are restored to their coordinates to obtain the four coordinate points in the original image. Based on the four coordinate points, the original image is homographically transformed to obtain the corrected text.

Images