Scene handwritten text erasing method based on two-stage deep learning network

Abstract

The invention discloses a scene handwritten text erasing method based on a two-stage deep learning network, and the method comprises the following steps: S1, constructing a handwritten text erasing data set containing a plurality of scenes, and dividing the data set into a training set, a verification set and a test set; s2, constructing a two-stage generator-discriminator network for training the data set to obtain a trained network model; according to the method, a scene handwritten text erasure data set containing a large number of complex backgrounds is constructed for training, and the handwritten text in the image is erased through the constructed two-stage generative adversarial network. And the tasks of completely erasing the handwritten text in the image and repairing the background image are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and relates to a scene handwritten text erasing method based on a two-stage deep learning network. Background Art

[0002] Text and images are very common in daily life and are an important means of information transmission. Handwritten text images in various scenes mainly come from printed documents, street scenes, scanned test papers, and text content in various environments. These images contain not only public text data, but also important personal privacy details, such as phone numbers, financial account information, identity information, and various ID numbers. Due to the general lack of public awareness of protecting personal privacy, a large number of images containing sensitive text are circulated on social media platforms and may be collected for various purposes, whether intentional or not. This situation may lead to personal privacy leakage, which may cause serious personal economic losses and threaten personal safety. Therefore, it is necessary to delete handwritten text.

[0003] At the same time, the current research focus is mainly on the field of printed text erasure. Compared with printed text, handwritten text in scenes has difficulties such as writing style, scattered distribution, diverse styles, and different text coverage, making it more challenging than erasing printed text. In addition, due to the lack of a systematic scene handwritten text erasure dataset, and the existing methods have problems such as unclean removal, mistakenly erasing the background, and insufficient image restoration accuracy, it is difficult to effectively remove handwritten text in complex scenes. Therefore, in view of the defects of the existing technology, it is necessary to propose a technical solution to solve the technical problems existing in the existing technology. Summary of the invention

[0004] To solve the above problems, the technical solution of the present invention is a scene handwritten text erasing method based on a two-stage deep learning network, comprising the following steps:

[0005] S1, build a handwritten text erasure dataset containing several scenarios, and divide the dataset into training set, validation set and test set;

[0006] S2, build a two-stage generator-discriminator network for training the data set to obtain a trained network model.

[0007] Preferably, after S2, the method further includes: S3, conducting a verification experiment on the trained network model, using the constructed handwritten text erasure dataset and several other text erasure datasets for verification experiments, and conducting a comparative experiment with several other network models.

[0008] Preferably, the S1 comprises the following steps:

[0009] S11, the constructed dataset contains two types of images, the first one is the image fused with the original image by generating text, and the second one is the image where the original handwritten text on the image is manually marked and erased;

[0010] S12, using a handwriting font generator, randomly generating a number of images of different sizes containing random handwritten texts, the handwritten texts are distributed in random positions and random sizes throughout the image, the image only contains the handwritten texts, the font is black or colored, and the background is pure white;

[0011] S13, collecting a plurality of background images, including a single background and multiple backgrounds, then performing noise elimination on two images, comparing the sizes of the two images to match the handwritten text with the background image, and fusing the images to generate an image containing the handwritten text; at this time, each group of data images includes an image containing the handwritten text, an image containing only the handwritten text, and an image containing only the background;

[0012] S14, collecting a plurality of original images containing handwritten texts, erasing the handwritten texts and filling the backgrounds by using an image restoration method; at this time, each set of data includes an original image containing handwritten texts and a background image without the handwritten texts; and then subtracting the two images and performing noise elimination to obtain an image containing only the handwritten texts;

[0013] S15, the constructed dataset contains more than a thousand groups of pictures, each group of pictures includes three types: images with handwritten text, reference images after erasing handwritten text, and handwritten text mask images. Among the more than a thousand groups of pictures, 80% are used for training, 10% are used for verification, and 5% are used for testing.

[0014] Preferably, S2 comprises the following steps:

[0015] S21, in the generator stage of the network, the handwritten text in the image is located through the designed multi-scale feature fusion attention mechanism to help the network in the preliminary erasure stage to recognize and erase the handwritten text. The image after the preliminary erasure stage still contains some unerased handwritten text, which is regarded as image noise. The image is erased again through the iterative noise removal network, in which a multi-scale feature extraction module is designed to extract image features, and finally the image after the handwritten text is erased is obtained;

[0016] S22, the network discriminator stage, inputs the image generated by the generator and the image of the real erased handwritten text into the pre-trained VGG feature extraction network to extract features for comparative analysis, and further generates an erased image through the generator network through the difference analysis of the two.

[0017] Preferably, the multi-scale feature fusion attention mechanism in S2 1 includes using multiple parallel hole convolution layers with different sampling rates to capture contextual information of different scales in the input feature layer for weighted fusion to enhance the representation ability of the feature map, and then obtaining two features on the channel dimension through average pooling, maximum pooling and multi-layer perceptron operations, adding the two features, and then adjusting the feature dimension through the activation function to obtain the channel dimension feature, and finally obtaining the spatial dimension feature through pooling and convolution layer operations, multiplying the channel dimension feature and the spatial dimension feature and adjusting the dimension to obtain the processed output feature.

[0018] Preferably, the multi-scale feature fusion attention mechanism process in S21 can be expressed by the following formula:

[0019] Y1=Concat(A1(X),A6(X),A 12 (X), A 18 (X)

[0020] Y=(Y1*M C (Y1)+Y1)*M S ((Y1*M C (Y1)+Y1))

[0021]

[0022]

[0023] Among them, X represents the features input to the module, Y1 represents the features after the dilated convolution, and A i (X) represents the feature map with a dilated convolution rate of i; U represents the input of the Mc function, where σ represents the Sigmoid activation function; W1 and W0 represent two convolution operations respectively; Umax and Uavg represent the maximum pooling and average pooling of the features; f 7*7 It represents a convolution operation with a convolution kernel size of 7*7, and the output feature obtained after module integration is Y.

[0024] Preferably, it is characterized in that the generator network in S21 is trained by guiding the loss function Gloss:

[0025]

[0026] Among them, maskloss is the image mask loss, holeloss and validArealoss are the image reconstruction losses, To generate adversarial loss, α m , α h , α v , α Lare the weights of each loss function, and their values ​​are 1, 1, 1, and 0.05 respectively.

[0027] Preferably, the multi-scale feature extraction module in S21 includes weighting the features obtained by different hole convolutions, connecting them through a concat function and restoring the feature scale through convolution to obtain the final features.

[0028] Preferably, the overall network in S2 is optimized and trained by a loss function, the expression of which is:

[0029]

[0030] where a m 、a h 、a v and a L is the weight of each loss function, and its values ​​are 1, 1, 1, 0.05 respectively. Maskloss is the image mask loss. The combination of holeloss and validArealoss is the image reconstruction loss. To generate adversarial loss; the specific expression is:

[0031] mask loss =aDice(P, G)+βBCE(P, G)

[0032] holeloss=l1((1-mask)*output, (1-mask)*gt)

[0033] validArealoss=l1(mask*output,mask*gt)

[0034]

[0035]

[0036] Among them, Dice(), BCE() and l1() are loss functions, and the specific expressions are:

[0037]

[0038]

[0039]

[0040] Where P i，j is the pixel value predicted by the network, G i，j is the true pixel value, Log is the natural logarithm, ∈ is a small constant to prevent the denominator from being zero, N is the number of samples, G i , P iare the true label and predicted value of the i-th sample respectively; αα and β are the weights of the two loss functions, which we set to (0.6, 0.4) in the network; mask is the text mask of the model, output is the final output of the model, and gt is the real erased image.

[0041] Preferably, S3 specifically includes: the constructed handwritten text erasure dataset and two scene text erasure datasets SCUT-Syn, SCUT-EnsText and the test paper handwritten text erasure dataset SCUT-EnsExa m Conduct a test.

[0042] The present invention also has at least the following beneficial effects:

[0043] 1. Taking advantage of the lack of scene handwritten text erasure datasets, the present invention collects and constructs handwritten text erasure datasets containing various backgrounds to support scene handwritten text erasure tasks, which provides assistance for current research and related investigations on various forms of scene handwritten texts.

[0044] 2. The present invention uses a two-stage generative adversarial network. In the generator stage, the coarse-fine two-stage network is used to locate and erase handwritten text. Multiple modules are designed to help the network perform refined training. The discriminator network is used to further generate guidance for the generated image, which can well complete the erasure task and improve the accuracy of handwritten text erasure. At the same time, the model can quickly process the input image and obtain the output image, and can process a large amount of handwritten text in a short time, which improves the speed of the handwritten text erasure task. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A mask generation strategy diagram for a data set of a scene handwritten text erasure method based on a two-stage deep learning network according to an embodiment of the present invention;

[0046] Figure 2 It is a partial data image of a data set of a scene handwritten text erasing method based on a two-stage deep learning network according to an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of a generative adversarial network of a scene handwritten text erasing method based on a two-stage deep learning network according to an embodiment of the present invention;

[0048] Figure 4 A schematic diagram of a multi-scale feature fusion attention mechanism of a scene handwritten text erasure method based on a two-stage deep learning network in an embodiment of the present invention;

[0049] Figure 5 A schematic diagram of multi-scale feature extraction of a scene handwritten text erasure method based on a two-stage deep learning network according to an embodiment of the present invention;

[0050] Figure 6 A schematic diagram of the overall network structure of a scene handwritten text erasing method based on a two-stage deep learning network according to an embodiment of the present invention;

[0051] Figure 7 This is a comparison result diagram of the scene handwritten text erasure method based on the two-stage deep learning network of an embodiment of the present invention and other network structures on a self-built handwritten text erasure dataset;

[0052] Figure 8 It is a schematic diagram of the comparison results of the scene handwritten text erasure method based on the two-stage deep learning network of an embodiment of the present invention and other network structures on the test paper handwritten text erasure dataset SCUT-EnsExam;

[0053] Fig. 9 It is a schematic diagram of the comparison results of the scene handwritten text erasure method based on the two-stage deep learning network according to an embodiment of the present invention and other network structures on the scene text erasure dataset SCUT-Syn;

[0054] Fig.10 This is a schematic diagram of the comparison results of the scene handwritten text erasure method based on the two-stage deep learning network of an embodiment of the present invention and other network structures on the scene text erasure dataset SCUT-EnsText. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0056] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the following detailed description of the present invention. Those skilled in the art can fully understand the present invention without the description of these details.

[0057] This invention is mainly divided into two parts. The first part is to promote the research on scene handwritten text erasure, thereby establishing a handwritten text erasure dataset containing various scenes. The second part is to implement and verify the performance of the scene handwritten text erasure method by building a two-stage deep learning network. In general, this invention includes three major steps:

[0058] S1: Build a handwritten text erasure dataset containing various scenarios and divide the dataset into training set, validation set and test set;

[0059] S2: Build a two-stage generator-discriminator network for training the dataset and obtain a trained network model;

[0060] S3: The trained network model is subjected to verification experiments, using the constructed handwritten text erasure dataset and other public text erasure datasets for verification experiments. At the same time, comparative experiments are conducted with other network models to verify the superiority and practicality of the model.

[0061] Wherein S1 further comprises:

[0062] S11: The constructed dataset contains two types of images. The first type is an image that is fused with the original image by generating text, and the second type is an image in which the original handwritten text on the image is manually marked and erased.

[0063] S12: Using a handwriting font generator, randomly generate a large number of images of various sizes containing handwritten texts with random handwriting. The handwritten texts are distributed at random positions and random sizes throughout the image. The image only contains handwritten texts, the fonts are black or colored, and the background is pure white.

[0064] S13: various images including simple background and complex background are collected, and then the two images are noise-removed and the sizes of the two images are compared to match the handwritten text and the background image, and the images are fused by image fusion to generate an image containing the handwritten text. At this time, each group of data images includes an image containing the handwritten text, an image containing only the handwritten text, and an image containing only the background.

[0065] S14: Collect various original images containing handwritten texts, and use image restoration technology to erase the handwritten texts and fill them with appropriate backgrounds. At this time, each set of data contains the original image containing handwritten texts and the background image without the handwritten texts. Figure 1 As shown in the figure, Image, Gt and Mask correspond to the input original image, the reference image with handwritten text erased and the image containing only handwritten text. By calculating the RGB mean of Image and Gt, grayscale conversion is performed to obtain a preliminary image mask, which contains some small noise points. Erosion operation is required to remove irregular noise points, and finally an image containing only handwritten text is obtained.

[0066] S15: Figure 2 The images shown are sample images selected from the constructed dataset. The constructed dataset contains 6900 sets of images, each of which includes three types: images with handwritten text, reference images after erasing handwritten text, and text mask images. 6000 sets are used for training, 600 sets are used for verification, and 300 sets are used for testing.

[0067] See also Figure 3-6In order to achieve the task of scene handwritten text erasure, we constructed a generative adversarial network, whose structure diagram is as follows: Figure 3 , input the image k with handwritten text, i.e., random noise z, into the generator network G, and learn the features of the reference real image x through the network, remove the handwritten text and generate an image, and compare the erased image generated by the generator with the real image in the discriminator, and update the parameters of the generator network G and the discriminator network D at the same time, so as to help the generator generate the desired image better. This corresponds to step S2, which further includes:

[0068] S21: In the scene handwritten text erasure task, to erase the handwritten text, we first need to detect the position and scale of the handwritten text, and then erase the text through subsequent operations. Therefore, this first step is crucial. In the face of the difficulty of locating handwritten text in complex backgrounds, we proposed the following method: Figure 4 Multi-scale feature fusion attention mechanism shown.

[0069] The input feature layer C*H*W is weightedly fused by using multiple parallel hole convolution layers with different sampling rates to capture contextual information of different scales and enhance the representation ability of the feature map. e Represents the hole value, and then obtains the features Feature a and Feature b in the channel dimension through average pooling AvgPool, maximum pooling MaxPool and multi-layer perceptron operation SharedMLP, adds the two features, and then adjusts the feature dimension through the activation function Sigmoid function to obtain Out feature1, and finally obtains the feature Outfeature2 in the spatial dimension through pooling and convolution layer operation. Multiply the channel dimension feature and the spatial dimension feature and adjust the dimension to obtain the processed C*H*W feature. The process formula can be expressed as:

[0070] Y1=Concat(A1(X),A6(X),A 12 (X), A 18 (X)

[0071] Y=(Y1*M C (Y1)+Y1)*M S ((Y1*M C (Y1)+Y1))

[0072]

[0073]

[0074] Where X represents the features input to the module, Y1 represents the features after the dilated convolution, and A i(X) represents the feature map with a dilated convolution rate of i; U represents the input of the Mc function, where σ represents the Sigmoid activation function; W1 and W0 represent two convolution operations respectively; Umax and Uavg represent the maximum pooling and average pooling of the features; f 7*7 It represents a convolution operation with a convolution kernel size of 7*7, and the output feature obtained after module integration is Y.

[0075] S22: In order to completely erase the handwritten text in the image, we used an iterative data refinement network to further locate and erase the handwritten text in the fine erasure stage. In order for the network to extract more useful and rich features, we proposed a multi-scale feature extraction module, such as Figure 5 As shown in the figure, the input feature is C*H*W. The features obtained by different hole convolutions are weighted, and W1-W5 are the weights of the features obtained by different convolutions when they are fused. Finally, they are connected through the concat function and the feature scale is restored through convolution to obtain the final feature. The multi-scale feature extraction module can effectively capture the multi-scale information of the image in the network to more comprehensively understand and represent the input, and it also has the characteristics of object size adaptability. The process is as follows:

[0076]

[0077] Where X, Y represent the input and output of the model respectively, Convri represents the dilated convolution operation, and the dilated rate ri is used. ri is the convolution kernel parameter related to the dilated convolution rate ri, W ri is the weight of each dilated convolution in the final connection.

[0078] S23: Figure 6 The overall network structure proposed in the present invention includes a generator stage and a discriminator stage; in the generator stage, the various modules proposed by us are used. The upper left part is the text positioning branch. The proposed multi-scale feature fusion attention mechanism module helps the network to locate the handwritten text. The generated positioning image Mout is compared with the real text positioning image GT, and the result is optimized by maskloss loss. The lower part is the handwritten text erasure network. The network is guided to erase and repair the image by locating the handwritten text result, and the final generated image is Irout, which is sent to the pre-trained VGG16 network with the real reference image to extract features for discrimination. The VGG network (same as VGG16 network) structure is a classic convolutional neural network structure proposed by the Visual Geometry Group (VGG) of Oxford University. The network structure consists of multiple convolutional layers, fully connected layers and softmax output layers, and has a deep network depth. The main feature of the VGG network structure is the use of continuous small convolution kernels, which increases the depth of the network and improves the feature expression ability of the network.

[0079] The overall network is optimized and trained through the loss function, and its expression is:

[0080]

[0081] where a m 、a h 、a v and a L is the weight of each loss function, and its values ​​are 1, 1, 1, 0.05 respectively. Maskloss is the image mask loss. The combination of holeloss and validArealoss is the image reconstruction loss. To generate adversarial loss. The specific expression is:

[0082] mask loss =aDice(P, G)+βBCE(P, G)

[0083] holeloss=l1((1-mask)*output, (1-mask)*gt)

[0084] validArealoss=l1(mask*output,mask*gt)

[0085]

[0086]

[0087] Among them, Dice(), BCE() and l1() are loss functions, and the specific expressions are:

[0088]

[0089]

[0090]

[0091] Where P i,j is the pixel value predicted by the network, G i,j is the true pixel value, Log is the natural logarithm, ∈ is a small constant to prevent the denominator from being zero, N is the number of samples, G i , P i are the true label and predicted value of the i-th sample respectively. α and β are the weights of the two loss functions respectively, which are set to (0.6, 0.4) in the network. mask is the text mask of the model, output is the final output of the model, and gt is the real erased image.

[0092] In order to verify the technical effect of the present invention, the effectiveness of the present invention will be verified by algorithm comparison:

[0093] The present invention selects various algorithm structures commonly used in the field of scene text erasure for comparative experiments with the present invention, and uses common indicators of image reconstruction to evaluate the performance of each model, including:

[0094] Mean square error (MSE), peak signal-to-noise ratio (PSNR), MSSIM for calculating the multi-scale structural similarity between two images, AGE for calculating the average absolute difference in grayscale between images, pEPs for the percentage of error pixels, and pCEPs for the percentage of clustering error pixels. The higher the MSSIM and PSNR values, the lower the AGE, pEPs, pCEPs and MSEZ values, indicating better results.

[0095] We conducted comparative experiments with various algorithm structures on our self-built dataset, the test paper handwritten text erasure dataset SCUT-EnsExam, and two scene text erasure datasets SCUT-Syn and SCUT-EnsText. SCUT-Syn is a synthetic dataset that contains a rich variety of text patterns, covering a variety of different scenarios and contexts. These text samples come from different fields, including but not limited to social media, news reports, medical records, etc., to ensure the diversity and coverage of the dataset. Each sample has undergone a carefully designed erasure process to simulate the text erasure needs in practical applications. The SCUT-EnsText dataset was proposed by Liu et al. in 2020. It is a large-scale real-world scene text erasure dataset, containing 2749 samples for training and 813 samples for testing. The final results are shown in Tables 1, 2, 3, 4 and Figure 7 , 8 , 9, and 10. Figure 7 The experimental results of the algorithm of the present invention and other algorithms on the self-built data set are shown in Figure 2. Figure 8 The experimental results of the algorithm of the present invention and other algorithms on the SCUT-EnsExam dataset are shown in Figure 2. Fig. 9 The experimental results of the algorithm of the present invention and other algorithms on the SCUT-Syn dataset are shown in Figure 2. Fig.10 The experimental results of the algorithm of the present invention and other algorithms on the SCUT-EnsText dataset. The first column of the image shows various images with handwritten text, the second column shows the reference image of the image with handwritten text erased, the last column shows the experimental results obtained by the model proposed by the present invention, and the other columns show the experimental results obtained by various comparison algorithms.

[0096] Table 1 Experimental data of the algorithm of the present invention and other algorithms on the self-built data set

[0097]

[0098] Table 2 Experimental data of the proposed algorithm and other algorithms on the SCUT-EnsExam dataset

[0099]

[0100] Table 3 Experimental data of the proposed algorithm and other algorithms on the SCUT-Syn dataset

[0101] Model PSNR↑ MSSIM↑ MSE↓ AGE↓ pEs↓ pCEPs↓ STE 25.40 90.12 0.65 9.49 0.0553 0.0347 EnsNet 37.36 96.44 0.21 1.73 0.0069 0.0020 FETNet 39.14 97.97 0.02 1.26 0.0046 0.0008 PERT 39.40 97.87 0.0002 1.41 0.0045 0.0006 EraseNet 37.03 97.86 0.0003 1.7834 0.0069 0.0024 Methods 38.32 98.45 0.002 1.60 0.0048 0.0004

[0102] Table 4 Experimental data of the proposed algorithm and other algorithms on SCUT-EnsText

[0103]

[0104] In the qualitative results, through the comparison of the above networks, it can be seen that the EraseNet network has the worst erasing effect on various images. Regardless of whether it is a complex background or a pure white background, a large amount of handwritten text is retained. Although the MTRNet network can remove most of the handwritten text, the erasing effect is unstable, and it will also erase non-handwritten text areas, and it fails to recognize and erase handwritten text in areas with complex backgrounds. Although the ViTEraser network erases most of the handwritten text, there are still subtle details that cannot be erased cleanly, and the image inpainting effect after the text is erased is not good. The model proposed in the present invention can not only accurately erase handwritten text, regardless of complex backgrounds or text overlap problems, but also the repair effect in the image repair stage can be well adapted to the real image. This shows that our model can accurately segment texts and remove them, and at the same time generate more semantically reasonable results.

[0105] In terms of quantitative results, our model outperforms other models in various indicators, especially in various indicators of the baseline model EraseNet network, which have been significantly improved, such as PSNR increased by 8.6, MSSIM increased by 1.54 percentage points, while MSE, AGE, pEPs and pCEPs values ​​have decreased to varying degrees. We attribute the performance improvement to our proposed multi-scale feature fusion attention mechanism, which can well locate handwritten text in various backgrounds and help the integrity of text erasure.

[0106] At the same time, in the fine erasing stage, the image noise is removed and the non-local module features are extracted to make the integrity and rationality of the restored image more perfect. Compared with various network models in recent years, the indicators obtained by our proposed model are higher than those of other models. This shows that our model can better adapt to the scene handwritten text erasure task.

[0107] The qualitative and quantitative results obtained from generalization experiments on two scene text erasure datasets show that our model is also very competitive on the scene text erasure dataset, which also indirectly proves the effectiveness of the algorithm proposed in this invention.

[0108] The above embodiments are only used to help understand the method and core idea of ​​the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0109] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A scene handwritten text erasure method based on a two-stage deep learning network, characterized in that: The following steps are involved: S1, build a handwritten text erasure dataset containing several scenarios, and divide the dataset into training set, validation set and test set; S2, build a two-stage generator-discriminator network for training the data set to obtain a trained network model.

2. The method according to claim 1, characterized in that After S2, the method further includes: S3, conducting a verification experiment on the trained network model, using the constructed handwritten text erasure dataset and several other text erasure datasets for verification experiments, and conducting a comparative experiment with several other network models.

3. The method according to claim 1, characterized in that The S1 comprises the following steps: S11, the constructed dataset contains two types of images, the first one is the image fused with the original image by generating text, and the second one is the image where the original handwritten text on the image is manually marked and erased; S12, using a handwriting font generator, randomly generating a number of images of different sizes containing random handwritten texts, the handwritten texts are distributed in random positions and random sizes throughout the image, the image only contains the handwritten texts, the font is black or colored, and the background is pure white; S13, collecting a plurality of background images, including a single background and multiple backgrounds, then performing noise elimination on two images, comparing the sizes of the two images to match the handwritten text with the background image, and fusing the images to generate an image containing the handwritten text; at this time, each group of data images includes an image containing the handwritten text, an image containing only the handwritten text, and an image containing only the background; S14, collecting a plurality of original images containing handwritten texts, erasing the handwritten texts and filling the backgrounds by using an image restoration method; at this time, each set of data includes an original image containing handwritten texts and a background image without the handwritten texts; and then subtracting the two images and performing noise elimination to obtain an image containing only the handwritten texts; S15, the constructed dataset contains more than a thousand groups of pictures, each group of pictures includes three types: images with handwritten text, reference images after erasing handwritten text, and handwritten text mask images. Among the more than a thousand groups of pictures, 80% are used for training, 10% are used for verification, and 5% are used for testing.

4. The method according to claim 1, characterized in that: The S2 comprises the following steps: S21, in the generator stage of the network, the handwritten text in the image is located through the designed multi-scale feature fusion attention mechanism to help the network in the preliminary erasure stage to recognize and erase the handwritten text. The image after the preliminary erasure stage still contains some unerased handwritten text, which is regarded as image noise. The image is erased again through the iterative noise removal network, in which a multi-scale feature extraction module is designed to extract image features, and finally the image after the handwritten text is erased is obtained; S22, the network discriminator stage, inputs the image generated by the generator and the image of the real erased handwritten text into the pre-trained VGG feature extraction network to extract features for comparative analysis, and further generates an erased image through the generator network through the difference analysis of the two.

5. The method according to claim 4, characterized in that The multi-scale feature fusion attention mechanism in S21 includes using multiple parallel hole convolution layers with different sampling rates to capture contextual information of different scales in the input feature layer for weighted fusion to enhance the representation ability of the feature map, and then obtaining two features on the channel dimension through average pooling, maximum pooling and multi-layer perceptron operations, adding the two features, and then adjusting the feature dimension through the activation function to obtain the channel dimension feature, and finally obtaining the spatial dimension feature through pooling and convolution layer operations, multiplying the channel dimension feature and the spatial dimension feature and adjusting the dimension to obtain the processed output feature.

6. The method according to claim 5, characterized in that The multi-scale feature fusion attention mechanism process in S21 can be expressed by the following formula: Y1=Concat(A1(X),A6(X),A 12 (X),A 18 (X)) Y=(Y1*M C (Y1)+Y1)*M S ((Y1*M C (Y1)+Y1)) Among them, X represents the features input to the module, Y1 represents the features after the dilated convolution, and A i (X) represents the feature map with a dilated convolution rate of i; U represents the input of the Mc function, where σ represents the Sigmoid activation function; W1 and W0 represent two convolution operations respectively; Umax and Uavg represent the maximum pooling and average pooling of the features; f 7*7 It represents a convolution operation with a convolution kernel size of 7*7, and the output feature obtained after module integration is Y.

7. The method according to claim 4, characterized in that The generator network in S21 is trained by the loss function Gloss: Among them, maskloss is the image mask loss, holeloss and validArealoss are the image reconstruction losses, To generate adversarial loss, α m , α h , α v , α L are the weights of each loss function, and their values ​​are 1, 1, 1, and 0.05 respectively.

8. The method according to claim 4, characterized in that The multi-scale feature extraction module in S21 includes weighting the features obtained by different hole convolutions, connecting them through a concat function, and restoring the feature scale through convolution to obtain the final features.

9. The method according to claim 4, characterized in that The overall network in S2 is optimized and trained by the loss function, and its expression is: where a m 、a h 、a v and a L is the weight of each loss function, and its values ​​are 1, 1, 1, and 0.05 respectively. Maskloss is the image mask loss, and the combination of holeloss and validArealoss is the image reconstruction loss. To generate adversarial loss; the specific expression is: mask loss =αDice(P,G)+αBCE(P,G) holeloss=l1((1-mask)*output,(1-mask)*gt) validArealoss=l1(mask*output,mask*gt) Among them, Dice(), BCE() and l1() are loss functions, and the specific expressions are: Where P i,j is the pixel value predicted by the network, G i,j is the true pixel value, Log is the natural logarithm, ∈ is a small constant to prevent the denominator from being zero, N is the number of samples, G i , P i are the true label and predicted value of the i-th sample respectively; α and β are the weights of the two loss functions, which we set to (0.6, 0.4) in the network; mask is the text mask of the model, output is the final output of the model, and gt is the real erased image.

10. The method according to claim 2, characterized in that The S3 specifically includes: testing on a constructed handwritten text erasure dataset and two scene text erasure datasets SCUT-Syn and SCUT-EnsText and an examination handwritten text erasure dataset SCUT-EnsExam.

Images