Text page image restoration method and system, electronic device and computer readable medium

Abstract

A text page image restoration method and system, electronic device and computer-readable medium. The text page image restoration method comprises: one branch extracts a distortion coefficient for each pixel in the text page image to be restored; another branch performs page high-end processing and / or removes handwriting traces; the two branches perform parallel processing; based on the obtained distortion coefficient, the text page image obtained by the parallel processing is subjected to distortion correction to obtain a restored text page image. The text page image restoration method of the present invention adopts a modular approach and can be adapted to different scenarios and solve different technical problems by flexibly combining various modules.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition and processing, and more specifically relates to a text page image restoration method and system, an electronic device using the method, and a computer-readable medium. Background Art

[0002] Office automation has greatly facilitated office workers. The combination of scanning, copying, printing, faxing, and emailing technologies is very convenient for the dissemination and creation of copywriting design. The development of this technology also affects students' learning. Now teachers sometimes assign homework by sending an electronic test paper, and then students print it out with their own printers at home and answer the test paper. The development of new technologies has also brought new problems. If a student sees a test paper that someone else has done, and the questions are particularly suitable for their own practice, or a test paper without an electronic test paper, and wants to do it again after doing it, how can they get a new, undo version of this test paper? Going online to search may find some electronic versions, but it cannot guarantee that all test papers can find corresponding blank electronic versions. Therefore, it is urgent to research and develop a method and system for test paper restoration to meet students' needs. Summary of the invention

[0003] In view of this, the main purpose of the present invention is to provide a method and system for restoring a text page image, an electronic device and a computer-readable medium using the method, in order to at least partially solve at least one of the above-mentioned technical problems.

[0004] In order to achieve the above object, as a first aspect of the present invention, a text page image restoration method is proposed, comprising the following steps:

[0005] Extracting distortion coefficients from each pixel in the text page image to be restored is used as one branch of text page image restoration processing; performing page high-end processing and / or removing handwriting traces on the text page image to be restored is used as another branch; the two branches are processed in parallel;

[0006] Based on the obtained distortion coefficient, the text page image obtained by parallel processing is distorted and corrected to obtain a restored text page image.

[0007] As a second aspect of the present invention, a text page image restoration system is also proposed, comprising:

[0008] Input module, used to input the text page image to be restored:

[0009] A preprocessing module, used for performing direction correction and / or page body correction on the input text page image to be restored as appropriate;

[0010] The parallel processing module includes a distortion coefficient extraction module, which is used to extract the distortion coefficient for each pixel in the text page image to be restored, as a branch of the parallel processing; and a page high-end module and / or a handwriting trace removal module, which is used to perform page high-end processing and / or handwriting trace removal processing on the text page image to be restored, as another branch of the parallel processing; and a local distortion correction module, which is used to perform distortion correction on the text page image obtained after the parallel processing based on the obtained distortion coefficient;

[0011] A post-processing module, used for post-processing the text page image processed by the parallel processing module according to the situation;

[0012] The output module is used to output the text page image after restoration.

[0013] As a third aspect of the present invention, an electronic device is also proposed, including a processor and a memory, wherein the memory is used to store a computer executable program, and when the computer executable program is executed by the processor, the processor executes the text page image restoration method as described above.

[0014] As a fourth aspect of the present invention, a computer-readable medium is further proposed, storing a computer-executable program, and when the computer-executable program is executed, the text page image restoration method as described above is implemented.

[0015] Based on the above technical solutions, it can be seen that the text page image restoration method and system of the present invention have at least one of the following beneficial effects compared with the prior art:

[0016] The text page image restoration method of the present invention greatly saves processing time through a parallel processing mode, and the processing is rapid and the effect is excellent;

[0017] The distortion correction scheme of the present invention is simple and fast, and can quickly and accurately identify the degree of distortion in the image through a simple model trained by machine learning, and can achieve pixel-by-pixel level;

[0018] The text page image restoration method of the present invention adopts a modular approach and can adapt to different scenarios and solve different technical problems by flexibly combining various modules.

[0019] The text page image restoration method of the present invention proposes corresponding improvements for different modules, and all of them are better in effect and more efficient than the corresponding modules in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a block flow chart of a text page image restoration method of the present invention;

[0021] Figure 2 It is a structural schematic diagram of the text page image restoration system of the present invention;

[0022] Figure 3 is a schematic structural diagram of an electronic device of the present invention;

[0023] Figure 4 is a schematic diagram of a computer readable medium of the present invention;

[0024] Figure 5 is a structural schematic diagram of a text page image restoration system according to Embodiment 3 of the present invention;

[0025] Fig. 6A , 6B This is an example of a photo of the actual processing effect of the prior art;

[0026] Figures 7A-7F This is another example of the actual processing effect photo of the embodiment of the present invention (including grayscale and color modes). DETAILED DESCRIPTION

[0027] In the process of introducing specific embodiments, the detailed description of the structure, performance, effect or other characteristics is to enable those skilled in the art to fully understand the embodiments. However, it does not exclude that those skilled in the art can implement the present invention with a technical solution that does not contain the above-mentioned structure, performance, effect or other characteristics under certain circumstances.

[0028] The flowchart in the accompanying drawings is only an exemplary process demonstration, and does not mean that the solution of the present invention must include all the contents, operations and steps in the flowchart, nor does it mean that it must be executed in the order shown in the figure. For example, some operations / steps in the flowchart can be decomposed, some operations / steps can be combined or partially combined, etc. The execution order shown in the flowchart can be changed according to actual conditions without departing from the main purpose of the present invention.

[0029] Frames in the attached figure Figure 1 Generally, it refers to functional entities, which do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different network and / or processing module devices and / or microcontroller devices.

[0030] The same reference numerals in the drawings represent the same or similar elements, components or parts, and thus the repeated description of the same or similar elements, components or parts may be omitted below. It should also be understood that although the first, second, third and other attributives representing the numbers may be used herein to describe various devices, elements, components or parts, these devices, elements, components or parts should not be limited by these attributives. In other words, these attributives are only used to distinguish one from another. For example, the first device may also be called the second device, but it does not deviate from the essential technical solution of the present invention. In addition, the terms "and / or" and "and / or" refer to all combinations including any one or more of the listed items.

[0031] One aspect of the present invention is to remove traces of answered test papers and regenerate new test papers. It can remove the handwriting of students and teachers, correct distortion, remove unsightly noise in the shooting and test papers, and restore them into a flat, unanswered, high-quality test paper, which is convenient for students to answer questions again. The technical solution of the present invention is also applicable to other text page images, such as electronic images of textbooks, books, promotional materials, etc. with traces of corrections and damage.

[0032] Another aspect of the present invention is that if a customer wants to collect and organize the questions they have answered for later review, there is no need to remove the handwritten answers from the test papers they have answered, but only to adjust and restore the photographed test papers to a state with regular pages, clear fonts, and a neat surface. This technical solution of the present invention is also applicable to other text page images, such as student notes, experimental records, interview notes, etc.

[0033] Based on this, Figure 1 As shown, the present invention proposes a method for restoring a text page image, comprising the following steps:

[0034] Extracting a distortion coefficient for each pixel in the text page image to be restored as a branch of the text page image restoration process;

[0035] Another branch is to perform page high-end processing and / or remove handwriting traces from the text page image to be restored;

[0036] The two branches are processed in parallel;

[0037] Based on the obtained distortion coefficient, the text page image obtained by parallel processing is distorted and corrected to obtain a restored text page image.

[0038] The text page images include, for example but are not limited to, electronic images of books, magazines, newspapers, leaflets, test papers, etc., which are text-based pages.

[0039] Wherein, the step of extracting the distortion coefficient is implemented by a pre-trained artificial intelligence model;

[0040] The artificial intelligence model is, for example, a U-net model with a dilated convolution structure, and the role of the dilated convolution is to expand the receptive field while ensuring the resolution of the features. The input of the artificial intelligence model is a distorted image, and the output is a distortion coefficient, i.e., an offset in the x direction and an offset in the y direction. For the distortion coefficient of the output network regression, for example, Gaussian blur can be used for local smoothing to eliminate local abnormal distortion.

[0041] When the artificial intelligence model is trained, the training network does not need to annotate samples, and is all based on generated samples. For example, a completely flat document image is distorted to simulate the distorted image that the model needs to process, and the offsets in the x and y directions required for distortion recovery are recorded. It can be seen that the scheme of the present invention is simple and fast, and can quickly and accurately identify the degree of distortion in the image through a simple model trained by machine learning, and can reach a pixel-by-pixel level.

[0042] Among them, the step of correcting the distortion of the text page image obtained by parallel processing based on the obtained distortion coefficient is to achieve image reconstruction based on a deep learning model of an encoder-decoder structure, wherein the encoder extracts features from the input image, and the decoder restores a flat image from the extracted features, and batch normalization (batch_normalization) and layer normalization (layer_normalization) are used for normalization in the whole process.

[0043] The loss function of the deep learning model is calculated by the following method:

[0044] Based on the obtained distortion coefficient, the offset in the x direction and the offset in the y direction of the image output by the deep learning model are calculated, and the mean square error of the grid data in these two directions and the real offset in the x direction and the real offset in the y direction saved when the distorted image is generated are calculated as the grid loss function (loss) of the network; the mean square error loss function (loss) between the image output by the deep learning model and the real flattened image is calculated; the gradient is calculated according to the above two loss functions (loss), and the entire deep learning model is updated.

[0045] The step of upgrading the page to a high-end page at least includes removing stains, watermarks and moiré patterns, for example, the following method may be used:

[0046] A text image enhancement model is adopted, including a generator and a discriminator. Through adversarial learning of the generator and the discriminator, the original color text image (text page image) to be processed is enhanced to generate a grayscale / color image without image noise and watermark. Among them, the scheme of the present invention can output two modes: grayscale image mode or color image mode. The two schemes only output grayscale images or color images during processing. The whole processing process is the same and can be selected according to user needs.

[0047] The generator includes an encoding module, and the encoding module includes r convolutional layers, each of which is composed of a convolutional layer, an IN normalization layer, and a PReLU activation layer; the convolutional layer of the encoding module is convolutionally operated to extract different features of the input image, and the first convolutional layer may only extract some low-level features such as edges, lines, and corners. More layers of the network can iteratively extract more complex features from low-level features. Preferably, the convolution kernels of the first layer of the r convolutional layers are more than the convolution kernels of the remaining r-1 layers.

[0048] The generator includes a residual network module, the residual network module includes s residual modules, each residual module includes at least three convolutional layers, and at least one intermediate convolutional layer has more convolutional kernels than other convolutional layers. The use of the residual network module can transform the features extracted by the encoding module to the style to be generated without losing much original information. Here, the intermediate convolutional layer of the residual module package can increase or reduce the number of channels, so that other convolutional layers perform convolution operations with relatively low-dimensional inputs, thereby improving computational efficiency.

[0049] Among them, the generator includes a decoding module with a symmetrical structure with the encoding module, and the decoding module includes r deconvolution levels, and the convolution kernel of each deconvolution level corresponds to the convolution kernel of each convolution level of the encoding module.

[0050] Among them, the discriminator adopts a convolutional neural network, including t convolutional layers and 1 fully connected layer. The fully connected layer outputs the degree of closeness between the generated image and the real image, which is used to optimize the effect of the generator.

[0051] The text image enhancement model is trained by the following method:

[0052] A color text image sample set A containing various types of image noise and / or watermarks and a grayscale / color image sample set B without image noise and watermarks are obtained, and they are respectively input into the text image enhancement model. The parameters of the text image enhancement model are regressed by the back propagation method. After the model training is completed, the generator G is extracted. AB .

[0053] The text image enhancement model of the present invention enhances the original color text image to be processed to generate a grayscale / color image without image noise and watermark through adversarial learning of the generator and the discriminator, thereby achieving simultaneous processing of removing image noise, watermark and other interferences from the original color text image and grayscale / colorization of the image, improving the reader's perception, and having a better image processing effect and high robustness. At the same time, the original color text image processed by the text image enhancement model of the present invention can highlight the text area more, which can greatly improve the detection and recognition accuracy of OCR.

[0054] Optionally, inputting the color text image sample set into the text image enhancement model includes:

[0055] The color text images in the color text image sample set are randomly cut into different sizes, and the images of different sizes are adjusted to the same set size and input into the text image enhancement model;

[0056] Preferably, the color text image is randomly cut into at least two sizes in equal proportions;

[0057] Further preferably, the color text image is cropped by adopting a "crop" method, and the cropped image is resized to a set size specification and input into the text image enhancement model.

[0058] The training method of the text image enhancement model of the present invention realizes multi-scale input of training images by randomly cutting the color text images of the color text image sample set into different sizes, adjusting the images of different sizes to the same set size and inputting them into the text image enhancement model. The generated text image enhancement model can adapt to images of various sizes, thereby improving the robustness and stability of the model.

[0059] Among them, the loss function of the text image enhancement model includes:

[0060] Fighting Loss:

[0061] From A→B adversarial loss:

[0062] L GAN (G,D B , A, B) = E[log D B (b)]+E[log(1-D B (G(a))];

[0063] From B → A adversarial loss:

[0064] L GAN (G,D A , A, B) = E[log D A(a)]+E[log(1-D A (G(b))];

[0065] Cycle consistency loss:

[0066] L cyc (G A , G B )=E[||G B (G A (a))-a]||]+E[||G A (G B (b))-b]||];

[0067] Overall loss:

[0068] L total =L GAN (G,D B , A, B)+L GAN (G,D A , A, B)+L cyc (G A , G B ).

[0069] Among them, the first discriminator D A , the second discriminator D B , the first generator G A , the second generator G B , color text image sample set (dataset A), grayscale / color image sample set (dataset B).

[0070] The training method of the text image enhancement model of the present invention inputs an image in a color text image sample set A into a pre-trained text image enhancement model, outputs an image after multiple convolution processes, and calculates a first loss value of the output image relative to an image in a grayscale / color image sample set B according to an adversarial loss function from A→B. The output image is input into a pre-trained text image enhancement model, outputs a discriminant value after multiple convolution processes, and calculates a second loss value of the discriminant value relative to a preset label value according to an adversarial loss function from B→A. The parameters of the generator and the discriminator are updated in combination with the first loss value and the second loss value until the combination of the first loss value and the second loss value meets the preset conditions, the training ends, and the trained text image enhancement model is obtained.

[0071] Among them, the method of removing handwriting traces includes, for example:

[0072] Based on the multi-task learning training framework, a generative adversarial network with multi-signal supervision was designed. The dice-loss for printing and handwriting segmentation was also integrated into the model to improve its local attention to the printed and handwritten areas. Handwriting removal also includes grayscale models and color models. The difficulty lies in removing handwriting while retaining the printed fonts covered by the handwriting.

[0073] As a preferred method, the method for removing handwriting traces adopts the following method:

[0074] Processing the text page image using a pre-trained artificial intelligence model to output a text page image with handwriting traces removed;

[0075] The pre-trained artificial intelligence model is, for example, a high-resolution network (HRnet) model, and the model uses the pre-trained artificial intelligence model to process the text page image and outputs a text page image with handwriting traces removed;

[0076] The pre-trained artificial intelligence model is, for example, a high-resolution network (HRnet) model, which inputs an image and outputs three parts of information: a printing track, a handwriting track, and an image after removing the handwriting track; the training sample of the model includes four parts: an original image with handwriting, a printing track, a handwriting track, and an image without handwriting; the model structure can, for example, adopt a high-resolution network (HRnet) to extract features of a fusion of multiple receptive fields; a feature map pyramid is constructed by deconvolution and stacking (concat), deconvolution can increase the scale of the feature map, and features of different convolutional layers are spliced ​​by stacking (concat) to achieve a fusion effect, and finally a multi-head structure output is formed, and the output includes the following three parts: (1) a printing track of w*h*2, (2) a handwriting track of w*h*2, and (3) an image after removing the track of w*h*3; wherein w is the width of the input image and h is the height of the input image. In this preferred embodiment, the output of the printed track and the handwritten track is a semantic segmentation task. In the scenario where printing and handwriting overlap, the three-classification problem of printed, handwritten, and background pixels is improved into two two-classification problems, which can achieve non-mutually exclusive segmentation of printed and handwritten pixels. The extreme loss function of the segmentation is shown below, which adopts the loss function of the fusion of dice-loss and cross entropy loss to form the complementary advantages of the two. The dice-loss and cross entropy losses are composed of their own handwriting and printing losses respectively.

[0077] L seg =αL dice +(1-α)L ce ;

[0078] L dice=βL dice_print +(1-β)L dice_hw ;

[0079] L ce =γL ce_print +(1-γ)L ce_hw ;

[0080] Among them, L seg , L dice , L ce They are segmentation loss function, dice loss function, and cross entropy loss function, respectively. α is the adjustment weight of dice loss and cross entropy loss, β is the adjustment weight of internal printing and handwriting of dice loss, and γ is the adjustment weight of internal printing and handwriting of cross entropy loss.

[0081] The loss function L of the final stage network is as follows:

[0082] L=δL seg +(1-δ)L1;

[0083] Among them, L is the total loss function of image segmentation and restoration, δ is the adjustment weight of the segmentation loss function and the image restoration loss function; L1 is the image restoration loss function.

[0084] The multi-task learning in this implementation is prone to non-convergence during the training process. Therefore, the network is updated through a step-by-step learning method. First, the label of the segmentation task is used to supervise the network training. At this time, network learning will be relatively easy. The training is for about 30 stages (epochs). The L1-loss of image restoration is added. At this time, the focus of the network is to optimize the restoration effect, so the weight of L1-loss will be increased.

[0085] The method of the present invention may further include an input step before executing the above parallel processing step, for example, including but not limited to one or more of the following methods:

[0086] An electronic image file is obtained by photographing with an electronic camera or other image acquisition device;

[0087] Scanning electronically scanned documents through scanners, electronic extensions, etc.;

[0088] Transferring electronic files that already exist on the network through the network.

[0089] Since the input step is optional, Figure 1 The boxes for the input steps are indicated by dashed lines.

[0090] The method of the present invention may further include, before executing the above-mentioned parallel processing step, a step of preprocessing the text page image to be restored, such as a combination of one or more of the following steps:

[0091] Adjust the main body of the text page image, with the goal of adjusting the four top corners of the paper page so that the main body of the paper page fills the entire text page image and no longer includes areas unrelated to the paper page; and / or

[0092] Adjust the direction of the text page image so that the angle between the text lines in the text page image and the horizontal direction is 0 degrees.

[0093] Since the preprocessing step is optional, Figure 1 The boxes for the preprocessing steps are indicated by dashed lines.

[0094] The step of adjusting the page body is achieved, for example, by extracting the page body area and correcting the page body as described below:

[0095] The four vertices of the image are used as reference points, and regression calculation is used to determine a number of positioning key points, which are used to determine the main area of ​​the image; based on the positioning key points, the main area is identified and corrected to remove irrelevant information and normalize the main area.

[0096] The main body area of ​​the image includes the page area, and the plurality of positioning key points include the four vertices of the page. The positioning key points of the main body area to be corrected are obtained by using the four vertices of the image as reference points for regression calculation, and the main body area is directly intercepted from the image to be processed according to the positioning key points. The main body area in the image to be processed can be accurately identified without image segmentation processing, which improves the speed of image processing, effectively avoids the problem of unstable edge of image segmentation, and can accurately identify whether there is a main body area in the image to be processed.

[0097] The step of using regression calculation to determine a number of positioning key points includes: establishing an image positioning regression model, and using the trained image positioning regression model to obtain the position information of the positioning key points of the image.

[0098] Among them, the step of establishing the image positioning regression model includes: labeling sample data and using the sample data to train the image positioning regression model, wherein the labeled sample data refers to labeling the sample image to mark the positioning key points of the main area in the sample image and whether the main area is included.

[0099] The image positioning regression model inputs the image to be processed, and outputs the position coordinates and confidence of the positioning key points of the main area in the image, the confidence indicating the reliable probability of the output coordinates of the positioning key points; optionally, when the confidence is less than a predetermined threshold, the correction of the main area is stopped and a warning message of no main area is output. The present invention can identify whether there is a main area in the image to be processed by calculating the confidence of the position coordinates of the positioning key points, so as to determine whether to continue the subsequent correction process.

[0100] Wherein, when the main area of ​​the picture is the page area, the step of marking the sample picture includes: when the page area of ​​the sample picture is all located in the sample picture, directly marking the four vertices of the page as the positioning key points; when the page area of ​​the sample picture is partially located outside the sample picture, according to the page vertices located in the sample picture, and the intersection of the edge of the page and the edge of the picture, the position of the page vertices located outside the sample picture is inferred, and the page vertices in the picture and the inferred page vertices located outside the sample picture are marked as the positioning key points. This is mainly for some application scenarios, such as in the photo search scenario, when the main area of ​​the picture is the page area, by marking the picture, the four vertices of the page area in many cases can be more accurately marked, so that the training data set can be established using the pictures with the four vertices of the page area in the subsequent use, and the image positioning regression model can be trained using the training data set. Furthermore, the position of the page vertex located outside the sample picture can be inferred, for example, by a computer, according to the page vertices located in the sample picture, and the intersection of the page edge and the picture edge.

[0101] The present invention uses the four vertices of the image as reference points to perform regression calculation to obtain the positioning key points of the main area of ​​the image to be processed, and directly intercepts the main area from the image to be processed for correction based on these positioning key points. The scheme of the present invention can accurately identify the main area in the image to be processed without image segmentation processing, improves the speed of image processing, and effectively avoids the problem of unstable edge of image segmentation. This scheme can also remove irrelevant background outside the main area.

[0102] Optionally, the step of calculating the position of the page vertex located outside the sample image includes: marking the vertices of the sample image located in the image, and the intersection of the edge of the page and the edge of the image; traversing all the marked points of the sample image, and judging in turn whether two adjacent marked points are located at the same image edge at the same time, the marked points including the vertices located in the image and the intersection; when it is judged that the two adjacent marked points are located at the same image edge at the same time, continue to judge whether the angle formed by the extension and intersection of the two page area edges where the two marked points are located satisfies a first threshold angle; when the angle satisfies a first threshold angle, mark the two adjacent marked points as non-vertices, and mark the intersection formed by the extension and intersection of the two page area edges corresponding to the two non-vertices as the predicted vertex of the page area located outside the image, and delete the two marked points marked as non-vertices, and finally obtain a sample image marked with four positioning key points; optionally, the first angle threshold is 60 degrees.

[0103] The step of correcting the main body area includes: performing a radial transformation and / or a directional correction on the main body area according to the positioning key points to obtain a corrected image.

[0104] The method of the present invention can also be implemented as follows:

[0105] Inputting the image to be processed into the page detection model generated by the method for building a page detection model according to the second aspect of the present invention, obtaining four vertices and a confidence level of the page area of ​​the image to be processed; determining whether the confidence level is greater than or equal to a first threshold, and when the confidence level is greater than the first threshold, performing page area recognition and correction on the image to be processed according to the four vertices of the page area of ​​the image to be processed;

[0106] Among them, the method for building the page detection model includes the following steps: building a key point detection model; using a picture with four vertices of the page area marked, to train the key point detection model, so that the trained key point detection model can output the four vertices and confidence of the page area; wherein, the picture with four vertices of the page area marked is completed by the aforementioned step of marking the sample picture.

[0107] It can be seen that both methods of the present invention do not require image segmentation processing, which improves the speed of image processing and effectively avoids the problem of unstable edge of image segmentation. At the same time, it can more accurately identify whether there is a page area in the image to be processed, and also improves the image classification accuracy, thereby solving the problem of low image recognition accuracy caused by factors such as the page area being tilted relative to the image, having irrelevant background, non-subject area, etc., and at the same time, it can more accurately identify images with or without page areas.

[0108] The step of adjusting the rotation direction of the text page image is achieved, for example, by identifying a text line, determining an angle between the text line and a horizontal line, and adjusting the angle.

[0109] The steps may also be implemented in a tentative manner, for example, by simultaneously tentatively testing several rotation directions, detecting whether they are appropriate rotation directions, and then determining the actual rotation direction.

[0110] Preferably, the step of adjusting the rotation direction of the text page image is implemented, for example, by the following method:

[0111] Acquire a first text image;

[0112] Rotating the first text image according to N predetermined rotation angles to obtain N rotated images, where N is a natural number greater than or equal to 2;

[0113] estimating a correction direction of the first text image according to the N rotated images, wherein the correction direction refers to a direction of the first text image after a rotation operation is performed to facilitate subsequent image recognition, and the direction is generally a direction that conforms to the reading habits of a user, that is, the font in the image is normal when viewed by the user, and is neither reversed nor skewed;

[0114] Direction correction is performed on the first text image according to the correction direction of the first text image.

[0115] The step of estimating the correction direction of the first text image according to the N rotated images includes:

[0116] The N rotated images are input into a trained direction estimation model to evaluate the probability that the directions of the respective rotated images are correct; and the direction of the rotated image with the highest probability is used as the correction direction of the first text image.

[0117] Optionally, the direction estimation model is generated after training an image classification model.

[0118] The direction estimation model includes: N neural networks, and judgment modules respectively connected to the N neural networks; the N neural networks respectively calculate the probability that the direction of the input rotated image is correct; the judgment module judges the correction direction of the first text image according to the probability level of the correct direction of each rotated image;

[0119] Optionally, the neural network includes a shallow convolutional network and a self-attention neural network connected in sequence, and the self-attention neural network is connected to the judgment module; the shallow convolutional network is used to extract feature data of the rotated image; and the self-attention neural network is used to obtain the probability that the direction of the rotated image is correct according to the feature data;

[0120] Optionally, the shallow convolutional network includes: a plurality of convolutional blocks and a fully connected layer;

[0121] Optionally, each of the convolution blocks includes a convolution layer, a pooling layer, a batch normalization layer and an excitation layer.

[0122] Wherein, the self-attention neural network is based on the Transformer model;

[0123] Optionally, the self-attention neural network includes: a self-attention module and a binary classification module, the self-attention module is used to convert the input image into a specification that meets the requirements of the binary classification module, and the binary classification module is used to obtain the probability that the direction of the rotated image is correct; exemplarily, the self-attention module includes a position-dependent feedforward network and a multi-head self-attention layer, and the position-dependent feedforward network encodes the input data and its position information at the same time during the encoding stage. The multi-head self-attention layer uses a self-attention mechanism in each sublayer to associate the input data and its position in the same input sequence. In addition, attention is called multi-head because several attention layers are stacked in parallel with different linear transformations of the same input. This helps the model capture various aspects of the input and improves its expressiveness.

[0124] Optionally, the binary classification module includes a fully connected layer and a softmax layer, which structure achieves parallel processing, shorter training time and higher conversion accuracy without any repeated components.

[0125] The first text image is rotated according to N predetermined rotation angles to obtain N rotated images, which includes:

[0126] Converting the first text image into a first text image matrix;

[0127] A matrix operation is performed on the first text image matrix to obtain N rotation matrices corresponding to the N rotated images.

[0128] There are four predetermined rotation angles, including: the first text image is not rotated, the first text image is rotated 90 degrees along the first direction, the first text image is rotated 180 degrees along the first direction, and the first text image is rotated 270 degrees along the first direction;

[0129] The step of performing a matrix operation on the first text image matrix to obtain N rotated image matrices corresponding to the N rotated images includes:

[0130] Using the first text image matrix directly as the first rotation image matrix;

[0131] After transposing the first text image matrix, the vertical bisector of the transposed matrix is ​​used as the symmetry axis to perform axisymmetric processing on the matrix elements to obtain the second rotated image matrix;

[0132] After performing axisymmetric processing on the matrix elements by taking the vertical bisector of the first text image matrix as the axis of symmetry, performing axisymmetric processing on the matrix elements by taking the horizontal bisector of the processed matrix as the axis of symmetry, the third rotated image matrix is ​​obtained;

[0133] After transposing the first text image matrix, the horizontal bisector of the transposed matrix is ​​used as the symmetry axis to perform axisymmetric processing on the matrix elements to obtain the fourth rotated image matrix.

[0134] The above scheme of the present invention can improve the accuracy of text recognition and enhance the user experience; compared with the prior art, the scheme of the present invention comprehensively identifies the correction direction of the first text image based on the rotation images in multiple directions obtained after the first text image is synchronously rotated, and has a higher accuracy rate. In addition, the present invention only needs to input the text image, without detecting and identifying the text line, to quickly identify the correction direction of the text image, and can also identify the correction direction of the text image that does not contain text, and has the advantages of simple operation, fast recognition, and a wide range of applications.

[0135] The method of the present invention may further include, after executing the above-mentioned parallel processing steps, a step of post-processing the text page image, for example, including one or a combination of the following steps:

[0136] Steps for adjusting brightness and contrast of the text page image; and / or

[0137] The step of erasing, modifying or adding part of the content of the text page image according to the needs of the user, specifically, the present invention provides an editing tool for the user to freely operate, and can modify part of the content in the text page image according to the needs of the user, for example, the handwriting marks that are not erased can be manually erased by an eraser, if there are errors in the handwriting content, they can be erased first and then the correct ones can be written, and if there are new content or ideas to be added, new text can be added, etc.; and / or

[0138] Identify the printed fonts in the text page image, replace them with the fonts set by the user and re-output; and / or

[0139] Darken the fonts and lines (e.g., horizontal lines representing spaces, dividing lines, etc.) in the text page image; and / or

[0140] Add a personalized label or watermark to the newly generated text page image; and / or

[0141] Change the background color of text page images to the color set by the customer; etc.

[0142] The specific implementation methods of the above steps are, for example, well-known means in the art. Since the post-processing step is optional, Figure 1 The boxes for post-processing steps are indicated by dashed lines.

[0143] The method of the present invention may further include an output step, such as outputting the restored text page image to a screen, printer, projector, or communication device. The printer includes a ribbon printer, an inkjet printer, a laser printer, and a thermal transfer printer. The thermal transfer printer includes, for example, the Miaomiao printer, a compact printer specially designed for students to print notes or test papers at any time.

[0144] Since the output step is also optional, Figure 1 The boxes for the output steps are also indicated by dashed lines.

[0145] like Figure 2 As shown, the present invention also discloses a text page image restoration system, comprising:

[0146] Input module, used to input the text page image to be restored:

[0147] A preprocessing module, used for performing direction correction and / or page body correction on the input text page image to be restored as appropriate;

[0148] The parallel processing module includes a distortion coefficient extraction module, which is used to extract the distortion coefficient for each pixel in the text page image to be restored, as a branch of the parallel processing; and a page high-end module and / or a handwriting trace removal module, which is used to perform page high-end and / or handwriting trace removal processing on the text page image to be restored, as another branch of the parallel processing; and a local distortion correction module, which is used to perform distortion correction on the text page image obtained after the parallel processing based on the obtained distortion coefficient; wherein the distortion coefficient extraction module and the local distortion correction module are collectively referred to as a distortion correction module;

[0149] A post-processing module, used for post-processing the text page image processed by the parallel processing module according to the situation;

[0150] The output module is used to output the text page image after restoration.

[0151] The text page images include, for example but are not limited to, pages of books, magazines, newspapers, leaflets, test papers, and the like.

[0152] The distortion coefficient extraction module is implemented by training a U-net model with a dilated convolution structure as described above.

[0153] Among them, the page high-end module is implemented, for example, by a trained artificial intelligence model, which is used to identify stains and text noise (including moiré) on the page. The artificial intelligence model, for example, adopts a generative adversarial network model and is divided into two output models: grayscale and color.

[0154] Preferably, the page high-end module is, for example, a page high-end module that adopts the preferred page high-end method as described above (referred to as "preferred page high-end module").

[0155] The handwriting trace removal module is implemented, for example, by using a generative adversarial network model, which is also divided into two output models: grayscale and color.

[0156] Preferably, the handwriting trace removal module is, for example, a handwriting trace removal module that adopts the preferred handwriting trace removal method as described above (referred to as "preferred handwriting trace removal module").

[0157] The text page image restoration system further includes an input module. The input module may include a camera, an electronic camera, a mobile phone with a camera function, a scanner, a communication module, etc., so that the input module may input image files in one or more of the following ways:

[0158] An electronic image file is obtained by photographing with an electronic camera or other image acquisition device;

[0159] Scanning by a scanner to obtain an electronic scanned document;

[0160] Transmitting electronic files that already exist on the network through the network; etc.

[0161] The text page image restoration system may further include a preprocessing module, which may include a direction correction module and / or a page body correction module.

[0162] Among them, the direction correction module is used to adjust the direction of the text page image so that the angle between the text line in the text page image and the horizontal direction is 0 degrees, for example, by identifying the text line, determining the angle between the text line and the horizontal line, and adjusting the angle; preferably, the direction correction module is, for example, a direction correction module that adopts the preferred direction correction method as described above (referred to as "preferred direction correction module").

[0163] Among them, the page body correction module is used to adjust the page body position; preferably, the page body correction module is, for example, a page body correction module that adopts the preferred page body correction method as described above (referred to as "preferred page body correction module").

[0164] The text page image restoration system may further include a post-processing module for post-processing the text page image after parallel processing. The post-processing methods include but are not limited to:

[0165] Steps for adjusting brightness and contrast of the text page image; and / or

[0166] The step of erasing, modifying or adding part of the content of the text page image according to the needs of the user, specifically, the present invention provides an editing tool for the user to freely operate, and can modify part of the content in the text page image according to the needs of the user, for example, the handwriting marks that are not erased can be manually erased by an eraser, if there are errors in the handwriting content, they can be erased first and then the correct ones can be written, and if there are new content or ideas to be added, new text can be added, etc.; and / or

[0167] Identify the printed fonts in the text page image, replace them with the fonts set by the user and re-output; and / or

[0168] Darken the fonts and lines (e.g., horizontal lines representing spaces, dividing lines, etc.) in the text page image; and / or

[0169] Add a personalized label or watermark to the newly generated text page image; and / or

[0170] Change the background color of text page images to the color set by the customer; etc.

[0171] The specific implementation of the above steps is, for example, a well-known method in the art.

[0172] The text page image restoration system may further include an output module for outputting the restored text page image. The output module may further perform the following output steps, such as outputting the restored text page image to a screen, a printer, a projector, or a communication device. The printers include ribbon printers, inkjet printers, laser printers, thermal transfer printers, and thermal printers. The thermal printers include the Miaomiao printer, which is the latest product launched by Zuoyebang. It is a compact printer specially designed for students, which only has a print head and does not need to be replaced with ink, and can print notes or test papers at any time.

[0173] like Figure 3 As shown, the present invention also discloses an electronic device, which includes a processor and a memory, wherein the memory is used to store a computer executable program, wherein when the computer executable program is executed by the processor, the processor executes the text page image restoration method as described above.

[0174] The electronic device may be, for example, in the form of a general-purpose computing device. The processor may be one or more than one processor working in coordination. The present invention does not exclude distributed processing, that is, the processor may be distributed in different physical devices. The electronic device of the present invention is not limited to a single entity, but may also be the sum of multiple physical devices.

[0175] The memory stores a computer executable program, usually a machine-readable code, which can be executed by the processor to enable the electronic device to execute the method of the present invention, or at least some of the steps in the method.

[0176] The memory includes a volatile memory, such as a random access memory module (RAM) and / or a cache memory module, and may also be a non-volatile memory, such as a read-only memory module (ROM).

[0177] Optionally, in this embodiment, the electronic device further includes an I / O interface, which is used for the electronic device to exchange data with an external device. The I / O interface can represent one or more of several types of bus structures, including a storage module bus or a storage module controller, a peripheral bus, a graphics acceleration port, a processing module, or a local bus using any of a variety of bus structures.

[0178] The electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display modules such as display screens, and some electronic devices also include human-computer interaction elements such as buttons and keyboards. As long as the electronic device can execute the computer-readable program in the memory to implement the method of the present invention or at least part of the steps of the method, it can be considered as an electronic device covered by the present invention.

[0179] Preferably, the electronic device is, for example, a thermal transfer printer, and more preferably a cat printer.

[0180] like Figure 4 As shown, the present invention further discloses a computer-readable medium on which a computer executable program is stored, wherein when the computer executable program is executed, the text page image restoration method as described above is implemented.

[0181] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0182] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Python, Java, C++, etc., as well as conventional procedural programming languages ​​such as C, assembly language, or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0183] The present invention also discloses a computer executable program, which, when executed, executes the text page image restoration method as described above.

[0184] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings. It should be noted that the following embodiments are only used to illustrate the present invention, and are not used to limit the present invention.

[0185] Example 1

[0186] The text page image restoration system of this embodiment is loaded on the network server side (cloud), and achieves its purpose by scanning and photographing the test paper with a scanner, etc., uploading it and calling the corresponding function. The text page image restoration system includes an input module, a preferred page high-end module, a preferred handwriting trace removal module, a page distortion extraction module and an output module, so that for the test paper image (text page image) input through the scanner, various traces and handwriting traces therein can be efficiently removed to obtain a complete and new test paper.

[0187] Since the test paper is closely attached to the scanner during scanning, the rotation direction and the deviation of the main area can be ignored, so the corresponding module can be omitted.

[0188] Example 2

[0189] The text page image restoration system of this embodiment is loaded on the network server side (cloud), and its purpose is achieved by photographing the test paper with a mobile phone, uploading it and calling the corresponding function. The text page image restoration system includes an input module, a preferred page high-end module, a preferred handwriting trace removal module, a distortion coefficient extraction module, a preferred page body correction module and an output module. Therefore, for the test paper image (text page image) input by photographing with a mobile phone, various traces and handwritings therein can also be efficiently removed, and the distortion can be corrected to obtain a complete and new test paper.

[0190] Among them, the solution of this embodiment is suitable for test paper images (text page images) input by mobile phone photography. No matter what the level of the photographer is, as long as the clarity requirements are met, various traces and handwritings can be efficiently removed, and the main image and distortion of the paper can be corrected to obtain a complete and new test paper.

[0191] Example 3

[0192] like Figure 5 As shown, the text page image restoration system of this embodiment is loaded on the network server side (cloud), and its purpose is achieved by taking pictures of the test paper and uploading it with a mobile phone and calling the corresponding function. Its design purpose is to restore the test paper and exercises uploaded by the user, after answering, to a flat and unanswered high-quality test paper by removing the handwriting of students and teachers, correcting distortion, and removing unsightly noise in the shooting and test paper. The text page image restoration system includes an input module, a preferred page high-end module, a preferred handwriting trace removal module, a distortion coefficient extraction module, a preferred direction correction module, a preferred page body correction module and an output module. The back-end services are all optimized using tensorRT, thereby saving the processing time of the modules. Among them, according to different usage scenarios, this embodiment provides two different technical solutions for restoring to black and white test papers and color test papers.

[0193] Figures 7A-7F is a series of photos processed by this embodiment, wherein: Fig. 7A It represents an image of a text page to be processed taken with a mobile phone, in which the paper page can be seen at the top of the image, and there are large blocks of content not related to the page on the right and below; Figure 7B Indicates that the image processed by the preferred direction correction module and the preferred page body correction module has the entire page filled with paper pages; Figure 7CThe image is processed by the grayscale image module in the high-end module of the preferred test paper. The entire page becomes a grayscale image, and traces such as moiré are removed. Fig.7D It indicates that the image is processed by the grayscale image module in the preferred handwriting removal module, the handwriting traces are removed, and what remains is a blank grayscale image; Fig. 7E It indicates the image processed by the color image module in the color image module of the high-end test paper optimization module. The whole page still maintains the color image, removes the traces such as moiré, and retains the handwriting traces; Figure 7F It means that the image processed by the color image module in the preferred handwriting removal module is continued, and the entire page still maintains a color image, and the handwriting traces are removed.

[0194] Example 4

[0195] The text page image restoration system of this embodiment is loaded on the network server side (cloud), and its purpose is achieved by photographing the test paper with a mobile phone, etc., uploading it and calling the corresponding function. The text page image restoration system includes an input module, a preferred page high-end module, a distortion coefficient extraction module, a preferred direction correction module, a preferred page main body correction module, a post-processing module and an output module. Therefore, for the note image (text page image) input by photographing with a mobile phone, no matter what the level of the photographer is, as long as the clarity requirement is met, various stains and traces therein can be efficiently removed while retaining handwriting traces, and the rotation direction of the paper, the main image and the distortion deformation can be corrected to obtain a complete and new note with handwriting retained.

[0196] Example 5

[0197] The text page image restoration system of this embodiment is loaded on the network server side (cloud), and its purpose is achieved by uploading the test paper through the mobile phone APP associated with the Miaomiao machine and calling the corresponding function. The text page image restoration system includes an input module, a preferred page high-end module, a preferred handwriting trace removal module, a distortion correction module, a preferred direction correction module, a preferred page main body correction module and an output module. Therefore, for the test paper image (text page image) that needs to be output by the Miaomiao machine, no matter its source and the level of the photographer, as long as the clarity requirements are met, various traces and handwritings therein can be efficiently removed, and the rotation direction of the paper, the main picture and the distortion deformation can be corrected, and the complete and new test paper can be printed by the Miaomiao machine.

[0198] Compared with the distortion correction of the prior art, such as Fig. 6A , 6B As shown, there is some overlap of text, but there is no such problem in the embodiment of the present invention, such as the above-mentioned embodiments 1-5 and Figures 7A-7FThe description is easy for those skilled in the art to understand that the present invention adopts a modular approach and can adapt to different scenarios and solve different technical problems by flexibly combining various modules; and compared with the prior art, the processing power and efficiency of each module of the present invention are better than the existing methods, with stronger adaptability and better restoration effect.

[0199] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the present invention is not inherently related to any specific computer, virtual device or electronic device, and various general devices can also implement the present invention. The above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for restoring a text page image, characterized in that: The steps include: The two branches of the text page image to be restored are processed in parallel, including: A distortion coefficient is extracted from each pixel in the text page image to be restored through a pre-trained artificial intelligence model as a branch of the text page image restoration processing; wherein the artificial intelligence model is a U-net model with a dilated convolution structure, whose input is a distorted image, and whose output is a distortion coefficient, i.e., an x-direction offset and a y-direction offset, and when the artificial intelligence model is trained, the training network is entirely based on the generated samples, including: performing a distortion transformation based on a completely flat text page image to simulate the distorted image that the artificial intelligence model needs to process, and recording the x-direction and y-direction offsets required for distortion recovery; Another branch is to perform page enhancement and / or remove handwriting traces on the text page image to be restored; wherein the page enhancement includes: using a text image enhancement model, using the adversarial learning of the generator and the discriminator in the model, enhancing the original color of the text page image to be restored to generate a grayscale / color image without image noise and watermark; wherein removing handwriting traces includes: using a pre-trained high-resolution network model to process the text page image to be restored, and outputting the text page image with the handwriting traces removed; After parallel processing through two branches, the text page image obtained by parallel processing is distorted and corrected based on the obtained distortion coefficient to obtain a restored text page image, including: reconstructing the text page image obtained by parallel processing based on a deep learning model of an encoder-decoder structure, wherein the encoder extracts features from the input image, and the decoder restores a flat image from the extracted features.

2. The method according to claim 1, characterized in that Also includes: For the distortion coefficient output by the artificial intelligence model, Gaussian blur is used for local smoothing to eliminate local abnormal distortion.

3. The method according to claim 1 or 2, characterized in that: The loss function of the deep learning model is calculated by the following method: based on the obtained distortion coefficient, the offset in the x direction and the offset in the y direction of the image output by the deep learning model are calculated, and the mean square error of the grid point data in these two directions and the offset in the real x direction and the real y direction saved when the distorted image is generated are calculated as the grid point loss function of the network; Calculate the mean square error loss function between the image output by the deep learning model and the real flattened image; The gradient is calculated based on the above two loss functions and the entire deep learning model is updated.

4. The method according to any one of claims 1 to 3, characterized in that: The page high-end enhancement at least includes: removing stains, watermarks and moiré patterns; wherein the generator includes an encoding module and a residual network module, the encoding module includes r convolutional layers, the convolution kernel of the first layer of the r convolutional layers is more than the convolution kernels of the remaining r-1 layers, the residual network module includes s residual modules, each residual module includes at least three convolutional layers, and the convolution kernel of at least one intermediate convolutional layer is more than the convolution kernels of other convolutional layers; wherein the discriminator adopts a convolutional neural network including t convolutional layers and 1 fully connected layer; and / or, The method of removing handwriting traces includes: the high-resolution network model inputs an image and outputs three parts of information: a printing track, a handwriting track, and an image after removing the handwriting track. The training samples of the high-resolution network model include four parts: an original image containing handwriting, a printing track, a handwriting track, and an image without handwriting.

5. The method according to claim 4, characterized in that The high-resolution network model uses deconvolution and stacking to construct a feature map pyramid, ultimately forming an output of a multi-head structure.

6. The method according to claim 5, characterized in that The output of the multi-head structure includes: (1) a printed track of w*h*2, (2) a handwritten track of w*h*2, and (3) an image after removing the track of w*h*3; wherein w is the width of the input image and h is the height of the input image.

7. The method according to claim 4, characterized in that The high-resolution network model adopts a loss function that is a fusion of dice loss and cross entropy loss. The dice loss and cross entropy loss are composed of their own handwriting and printing loss functions respectively: L seg =αL dics +(1-α)L ce ; L dice =βL dice_print +(1-β)L dice_hw ; L ce =γL ce_print +(1-γ)L ce_hw ; Among them, L seg , L dice , L ce They are segmentation loss function, dice loss function, and cross entropy loss function, respectively. α is the adjustment weight of dice loss and cross entropy loss, β is the adjustment weight of internal printing and handwriting of dice loss, and γ is the adjustment weight of internal printing and handwriting of cross entropy loss. The loss function L of the final stage network is as follows: L=δL seg +(1-δ)L1; Among them, L is the total loss function of image segmentation and restoration, δ is the adjustment weight of the segmentation loss function and the image restoration loss function; L1 is the image restoration loss function.

8. The method according to claim 5 or 6, characterized in that: The high-resolution network model adopts a loss function that is a fusion of dice loss and cross entropy loss. The dice loss and cross entropy loss are composed of their own handwriting and printing loss functions respectively: L seg =αL dice +(1-α)L ce ; L dice =βL dice_print +(1-β)L dice_hw ; L ce =γL ce_print +(1-γ)L ce_hw ; Among them, L seg , L dice , L ce They are segmentation loss function, dice loss function, and cross entropy loss function, respectively. α is the adjustment weight of dice loss and cross entropy loss, β is the adjustment weight of internal printing and handwriting of dice loss, and γ is the adjustment weight of internal printing and handwriting of cross entropy loss. The loss function L of the final stage network is as follows: L=δL seg +(1-δ)L1; Among them, L is the total loss function of image segmentation and restoration, δ is the adjustment weight of the segmentation loss function and the image restoration loss function; L1 is the image restoration loss function.

9. The method according to claim 1, characterized in that: Before executing the parallel processing step, the method further includes adjusting the main body of the text page image to be restored and / or adjusting the direction of the text page image.

10. The method according to claim 1, characterized in that After performing the above distortion correction step, the method further comprises the following steps: Steps for adjusting brightness and contrast of the text page image; and / or Erase, modify or add parts of the text page image content according to user needs; and / or Identify the printed fonts in the text page image, replace them with the fonts set by the user and re-output; and / or Darken the fonts and lines in the text page image; and / or Add a personalized label or watermark to the newly generated text page image; and / or Changes the background color of text page images to the color set by the customer.

11. A text page image restoration system, characterized in that: include: Input module, used to input the text page image to be restored: A preprocessing module, used for performing direction correction and / or page body correction on the input text page image to be restored as appropriate; The parallel processing module is used to process the two branches of the text page image to be restored in parallel, including: A distortion coefficient extraction module is used to extract the distortion coefficient of each pixel in the text page image to be restored through a pre-trained artificial intelligence model as a branch of parallel processing; wherein the artificial intelligence model is a U-net model with a dilated convolution structure, whose input is a distorted image, and whose output is a distortion coefficient, i.e., an offset in the x direction and an offset in the y direction, and when the artificial intelligence model is trained, the training network is entirely based on the generated samples, including: performing a distortion transformation based on a completely flat text page image to simulate the distorted image that the artificial intelligence model needs to process, and recording the offsets in the x direction and y direction required for distortion recovery; and, The page high-end module and / or the handwriting trace removal module are used to perform page high-end and / or handwriting trace removal processing on the text page image to be restored, as another branch of parallel processing; wherein the page high-end includes: by using the text image enhancement model, using the adversarial learning of the generator and the discriminator in the model, enhancing the original color of the text page image to be restored to generate a grayscale / color image without image noise and watermark; wherein the handwriting trace removal includes: using a pre-trained high-resolution network model to process the text page image to be restored, and outputting the text page image with the handwriting trace removed; and a local distortion correction module, which is used to perform distortion correction on the text page image obtained after the parallel processing based on the obtained distortion coefficient after the parallel processing through the two branches, including: reconstructing the text page image obtained after the parallel processing based on a deep learning model of an encoder-decoder structure, wherein the encoder extracts features from the input image, and the decoder restores a flat image from the extracted features; A post-processing module, used for post-processing the text page image processed by the parallel processing module according to the situation; The output module is used to output the text page image after restoration.

12. An electronic device comprising a processor and a memory, wherein the memory is used to store a computer executable program, wherein: When the computer executable program is executed by the processor, the processor executes the text page image restoration method according to any one of claims 1 to 10.

13. A computer-readable medium storing a computer-executable program, characterized in that: When the computer executable program is executed, the text page image restoration method according to any one of claims 1 to 10 is implemented.

Images