Scene handwritten text erasing method based on two-stage deep learning network

Abstract

This invention discloses a method for erasing handwritten text in various scenes based on a two-stage deep learning network, comprising the following steps: S1, constructing a handwritten text erasure dataset containing several scenes, and dividing the dataset into a training set, a validation set, and a test set; S2, constructing a two-stage generator-discriminator network to train the dataset, obtaining a trained network model. This invention constructs a scene-based handwritten text erasure dataset containing a large number of complex backgrounds for training, and uses a constructed two-stage generative adversarial network to erase handwritten text in images. This achieves the task of complete erasure of handwritten text in images and background image restoration.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and relates to a method for erasing handwritten text in a scene based on a two-stage deep learning network. Background Technology

[0002] Text and images are ubiquitous in daily life and serve as crucial means of information transmission. Handwritten text images in various scenarios primarily originate from printed documents, street scenes, scanned exam papers, and text content in various environments. These images not only contain public text data but also important personal privacy details, such as phone numbers, financial account information, identity information, and various ID numbers. Due to a general lack of public awareness regarding the protection of personal privacy, a large number of images containing sensitive text are disseminated on social media platforms, and whether intentional or not, they may be collected for various purposes. This situation can lead to the leakage of personal privacy, potentially causing serious personal financial losses and threatening personal safety. Therefore, the deletion of handwritten text is necessary.

[0003] Meanwhile, current research primarily focuses on erasing printed text. However, compared to printed text, handwritten text in various scenes presents challenges such as diverse writing styles, scattered distribution, varied patterns, and varying text coverage, making it more difficult to erase than printed text. Furthermore, the lack of a systematic dataset for erasing handwritten text in various scenes, coupled with existing methods' shortcomings such as incomplete removal, accidental background erasure, and insufficient image restoration accuracy, hinders the effective removal of handwritten text in complex scenes. Therefore, it is essential to propose a technical solution to address the deficiencies of existing technologies. Summary of the Invention

[0004] To address the above problems, the technical solution of this invention is a method for erasing handwritten text in a scene based on a two-stage deep learning network, comprising the following steps:

[0005] S1. Construct a handwritten text erasure dataset containing several scenarios, and divide the dataset into training set, validation set and test set;

[0006] S2, construct a two-stage generator-discriminator network to train the dataset and obtain the trained network model.

[0007] Preferably, after S2, the method further includes: S3, performing verification experiments on the trained network model, using the constructed handwritten text erasure dataset and several other text erasure datasets for verification experiments, and conducting comparative experiments with several other network models.

[0008] Preferably, step S1 includes the following steps:

[0009] S11, the constructed dataset contains two types of images: the first is an image fused with the original image by generating text, and the second is an image where the original handwritten text on the image is manually marked and erased.

[0010] S12, using a handwritten font generator, randomly generate several images of different sizes containing random handwritten text. The handwritten text is distributed at random positions and of random size throughout the image. The image contains only handwritten text, with the font being black or colored and the background being pure white.

[0011] S13, collect several background images, including single background and multiple background, then perform noise removal on the two images, compare the sizes of the two images to match the handwritten text and background images, and fuse the images to generate an image containing handwritten text; at this time, each set of data images includes an image containing handwritten text, an image containing only handwritten text, and an image containing only background.

[0012] S14: Collect several original images containing handwritten text, and erase the handwritten text and fill the background using image restoration methods; at this time, each set of data contains the original image containing handwritten text and the background image with the handwritten text removed; then, by subtracting these two images and removing noise, an image containing only handwritten text is obtained.

[0013] S15, the constructed dataset contains more than a thousand images, each image set includes three types: images with handwritten text, reference images after erasing the handwritten text, and handwritten text mask images. Of these more than a thousand images, 80% are used for training, 10% for validation, and 5% for testing.

[0014] Preferably, step S2 includes the following steps:

[0015] S21, in the generator stage of the network, the handwritten text in the image is located by a multi-scale feature fusion attention mechanism to help the network in the preliminary erasure stage recognize and erase the handwritten text. The image in the preliminary erasure stage still contains some unerased handwritten text, which is regarded as image noise. The image is then erased again by an iterative noise removal network. A multi-scale feature extraction module is designed to extract image features, and finally the image after erasing the handwritten text is obtained.

[0016] S22, the network discriminator stage, inputs the image generated by the generator and the real image of erased handwritten text into the pre-trained VGG feature extraction network to extract features for comparison and analysis. Based on the difference analysis between the two, the generator network further generates the erased image.

[0017] Preferably, the multi-scale feature fusion attention mechanism in S2 1 includes using multiple parallel dilated convolutional layers with different sampling rates to capture contextual information at different scales in the input feature layer, weighted fusion to enhance the representational capability of the feature map, then obtaining two features in the channel dimension through average pooling, max pooling and multilayer perceptron operations, adding the two features together, adjusting the feature dimension through an activation function to obtain the channel dimension feature, and finally obtaining the spatial dimension feature through pooling and convolutional layer operations, multiplying the channel dimension feature and the spatial dimension feature and adjusting the dimension to obtain the processed output feature.

[0018] Preferably, the multi-scale feature fusion attention mechanism process in S21 can be represented by the following formula:

[0019] Y1 = Concat(A1(X), A6(X), A 12 (X), A 18 (X))

[0020] Y = (Y1 * M) C (Y1)+Y1)*M S ((Y1*M C (Y1)+Y1))

[0021]

[0022]

[0023] Where X represents the features input to the module, Y1 represents the features after dilated convolution, and A i (X) represents the feature map with dilation rate i; U represents the input to the Mc function, where σ represents the Sigmoid activation function; W1 and W0 represent the two convolution operations respectively; Umax and Uavg represent max pooling and average pooling of the features; f 7*7 This indicates a convolution operation with a kernel size of 7*7, and the final output feature obtained after module integration is Y.

[0024] Preferably, the generator network in S21 is trained using a loss function Gloss:

[0025]

[0026] Where maskloss is the image masking loss, and holeloss and validArealoss are the image reconstruction losses. To generate adversarial loss, α m α h α v α LThese are the weights of each loss function, with values ​​of 1, 1, 1, and 0.05 respectively.

[0027] Preferably, the multi-scale feature extraction module in S21 includes weighting the features obtained by different dilated convolutions, connecting them by the concat function, and restoring the feature scale by convolution to obtain the final features.

[0028] Preferably, in step S2, the overall network is optimized and trained using a loss function, the expression of which is:

[0029]

[0030] Where a m a h a v and a L The weights for each loss function are 1, 1, 1, and 0.05, respectively. `maskloss` is the image masking loss, and `holeloss` and `validArealoss` are combined to form the image reconstruction loss. To generate adversarial loss; the specific expression is:

[0031] mask loss = aDice(P, G) + βBCE(P, G)

[0032] holeloss=l1((1-mask)*output, (1-mask)*gt)

[0033] validArealoss=l1(mask*output,mask*gt)

[0034]

[0035]

[0036] Where Dice(), BCE(), and l1() are loss functions, and their specific expressions are as follows:

[0037]

[0038]

[0039]

[0040] Where P i，j G represents the pixel value predicted by the network. i，j The values ​​are the actual pixel values, Log is the natural logarithm, ∈ is a small constant to prevent the denominator from being zero, N is the number of samples, and G... i P iα and β are the true label and predicted value of the i-th sample, respectively; α and β are the weights of the two loss functions, which we set to (0.6, 0.4) in the network; mask is the text mask of the model; output is the final output of the model; and gt is the image after real erasure.

[0041] Preferably, S3 specifically includes: constructing a handwritten text erasure dataset and two scenario text erasure datasets, SCUT-Syn and SCUT-EnsText, and a test paper handwritten text erasure dataset, SCUT-EnsExa. m Conduct the test.

[0042] The present invention also has at least the following beneficial effects:

[0043] 1. This invention utilizes the gap in scene-specific handwritten text erasure datasets. In order to support scene-specific handwritten text erasure tasks, it collects and constructs handwritten text erasure datasets containing various backgrounds, which provides assistance for current research and related investigations on various forms of scene-specific handwritten text.

[0044] 2. This invention utilizes a two-stage generative adversarial network. In the generator stage, a coarse-fine two-stage network is used to locate and erase handwritten text, with multiple modules designed to aid in refined training. A discriminator network further guides the generation of the image, enabling the model to effectively complete the erasure task and improve the accuracy of handwritten text erasure. Simultaneously, the model can quickly process input images and obtain output images, handling large amounts of handwritten text in a short time, thus increasing the speed of handwritten text erasure tasks. Attached Figure Description

[0045] Figure 1 This is a mask generation strategy diagram for the dataset of the handwritten text erasure method based on a two-stage deep learning network in an embodiment of the present invention.

[0046] Figure 2 These are partial data images of the dataset for the scene handwritten text erasure method based on a two-stage deep learning network according to an embodiment of the present invention.

[0047] Figure 3 This is a schematic diagram of the generative adversarial network for the scene handwritten text erasure method based on a two-stage deep learning network according to an embodiment of the present invention.

[0048] Figure 4 This is a schematic diagram of the multi-scale feature fusion attention mechanism of the scene handwritten text erasure method based on a two-stage deep learning network in an embodiment of the present invention.

[0049] Figure 5 This is a schematic diagram illustrating the multi-scale feature extraction of the scene handwritten text erasure method based on a two-stage deep learning network according to an embodiment of the present invention.

[0050] Figure 6 This is a schematic diagram of the overall network structure of the handwritten text erasure method based on a two-stage deep learning network according to an embodiment of the present invention.

[0051] Figure 7 The figure shows a comparison of the handwritten text erasure method based on a two-stage deep learning network in this invention with other network structures on a self-built handwritten text erasure dataset.

[0052] Figure 8 This is a schematic diagram showing the comparison results of the handwritten text erasure method based on a two-stage deep learning network in this invention with other network structures on the SCUT-EnsExam test paper handwritten text erasure dataset.

[0053] Figure 9 This is a schematic diagram showing the comparison results of the scene handwritten text erasure method based on a two-stage deep learning network in this embodiment of the invention with other network structures on the scene text erasure dataset SCUT-Syn;

[0054] Figure 10 This diagram illustrates the comparison between the scene handwritten text erasure method based on a two-stage deep learning network and other network structures on the scene text erasure dataset SCUT-EnsText, as described in this embodiment of the invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0056] Conversely, this invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of the invention as defined in the claims. Furthermore, to provide a better understanding of the invention, certain specific details are described in detail below. However, those skilled in the art will fully understand the invention even without these detailed descriptions.

[0057] This invention is mainly divided into two parts. The first part aims to promote research on handwritten text erasure in various scenarios by establishing a handwritten text erasure dataset covering various scenarios. The second part involves the implementation and performance verification of a scenario-based handwritten text erasure method through the construction of a two-stage deep learning network. Overall, this invention includes three main steps:

[0058] S1: Construct a handwritten text erasure dataset containing various scenarios, and divide the dataset into training set, validation set and test set;

[0059] S2: Construct a two-stage generator-discriminator network to train the dataset and obtain a trained network model;

[0060] S3: Conduct validation experiments on the trained network model. Use the constructed handwritten text erasure dataset and other public text erasure datasets for validation experiments. At the same time, conduct comparative experiments with other network models to verify the model's superiority and practicality.

[0061] S1 further includes:

[0062] S11: The constructed dataset contains two types of images. The first type is an image that is fused with the original image by generating text, and the second type is an image where the original handwritten text is manually marked and erased.

[0063] S12: Using a handwritten font generator, randomly generate a large number of images of various sizes containing random handwritten text. The handwritten text is distributed at random positions and of random size throughout the image. The image contains only handwritten text, with the font being either black or colored and the background being pure white.

[0064] S13: Collect various images, including those with simple and complex backgrounds. Then, perform noise removal on both images, compare their sizes to match the handwritten text with the background image, and fuse the images through image fusion to generate an image containing the handwritten text. At this point, each set of data images includes an image containing handwritten text, an image containing only handwritten text, and an image containing only the background.

[0065] S14: Collect various original images containing handwritten text. Using image inpainting techniques, erase the handwritten text and fill in a suitable background. At this point, each data set contains the original image containing handwritten text, and the background image after removing the handwritten text. For example... Figure 1 As shown, Image, Gt, and Mask correspond to the original input image, the reference image with the handwritten text erased, and the image containing only the handwritten text, respectively. By calculating the RGB mean of the two images, Image and Gt, a grayscale conversion is performed to obtain a preliminary image mask. This mask contains some small noise points, which need to be removed by erosion to remove irregular noise points. Finally, the image containing only the handwritten text is obtained.

[0066] S15: As Figure 2 The image shown is an example image selected from the constructed dataset. The constructed dataset contains 6900 images, each image including three types: images with handwritten text, reference images after erasing the handwritten text, and text mask images. Of these, 6000 images are used for training, 600 for validation, and 300 for testing.

[0067] See Figure 3-6To achieve the task of erasing handwritten text in a scene, we constructed a generative adversarial network, the structure of which is shown in the diagram below. Figure 3 The image k containing handwritten text (i.e., random noise z) is input into the generator network G. The network learns features from the reference real image x to remove the handwritten text and generate a new image. The erased image generated by the generator is compared with the real image in the discriminator, and the parameters of the generator network G and the discriminator network D are updated to help the generator better generate the desired image. This corresponds to step S2, which further includes:

[0068] S21: In the task of erasing handwritten text in a scene, to erase handwritten text, it is first necessary to detect the position and scale of the handwritten text before proceeding with subsequent operations to erase it. Therefore, this first step is crucial. However, given the difficulty of locating handwritten text in complex backgrounds, we propose the following... Figure 4 The multi-scale feature fusion attention mechanism is shown.

[0069] The input feature layer C*H*W is weighted and fused using multiple parallel dilated convolutional layers with different sampling rates to capture contextual information at different scales, thereby enhancing the representational power of the feature map. Rat e The hole value is represented by an average pooling (AvgPool) and max pooling (MaxPool) operation followed by a SharedMLP multilayer perceptron to obtain features a and b in the channel dimension. These two features are then summed using an Add function, and the feature dimensions are adjusted using the Sigmoid activation function to obtain Out feature1. Finally, pooling and convolutional layer operations are used to obtain Out feature2 in the spatial dimension. The channel dimension feature and the spatial dimension feature are multiplied and their dimensions adjusted to obtain the processed C*H*W feature. This process can be expressed as:

[0070] Y1 = Concat(A1(X), A6(X), A 12 (X), A 18 (X))

[0071] Y = (Y1 * M) C (Y1)+Y1)*M S ((Y1*M C (Y1)+Y1))

[0072]

[0073]

[0074] Where X represents the features input to the module, Y1 represents the features after dilated convolution, and A i(X) represents the feature map with dilation rate i; U represents the input to the Mc function, where σ represents the Sigmoid activation function; W1 and W0 represent the two convolution operations respectively; Umax and Uavg represent max pooling and average pooling of the features; f 7*7 This indicates a convolution operation with a kernel size of 7*7, and the final output feature obtained after module integration is Y.

[0075] S22: To completely erase handwritten text from the image, we employed an iterative data refinement network in the fine erasure stage to further locate and erase the handwritten text. To enable the network to extract more useful and rich features, we proposed a multi-scale feature extraction module, such as... Figure 5 As shown, the input features are C*H*W. The features obtained from different dilated convolutions are weighted, with W1-W5 representing the weights when fusing features from different convolutions. Finally, they are concatenated using the `concat` function and the feature scale is restored through convolution to obtain the final features. The multi-scale feature extraction module can effectively capture multi-scale information of the image in the network to more comprehensively understand and represent the input, while also exhibiting object size adaptability. The process is as follows:

[0076]

[0077] Where X and Y represent the model's input and output, respectively, Convri represents the dilated convolution operation, using the dilation rate ri, and Z... ri W is the kernel parameter related to the dilated convolution rate ri. ri It represents the weight of each dilated convolution at the final connection.

[0078] S23: Figure 6 The overall network structure proposed in this invention includes a generator stage and a discriminator stage. In the generator stage, various modules proposed in this paper are utilized. The upper left part is the text localization branch, which uses a proposed multi-scale feature fusion attention mechanism to help the network localize handwritten text. The generated localized image Mout is compared with the real text localization map GT, and the result is optimized using mask loss. The lower part is the handwritten text erasure network. The handwritten text localization result guides the network to erase and repair the image, ultimately generating an image called Irout. This Irout image is then fed into a pre-trained VGG16 network to extract features for discrimination, along with a real reference image. The VGG network (same as the VGG16 network) is a classic convolutional neural network structure proposed by the Visual Geometry Group (VGG) at Oxford University. This network structure consists of multiple convolutional layers, fully connected layers, and a softmax output layer, resulting in a relatively deep network. The main feature of the VGG network structure is the use of consecutive small convolutional kernels, which increases the network depth and improves its feature representation ability.

[0079] The overall network is optimized and trained using a loss function, the expression of which is:

[0080]

[0081] Where a m a h a v and a L The weights for each loss function are 1, 1, 1, and 0.05, respectively. `maskloss` is the image masking loss, and `holeloss` and `validArealoss` are combined to form the image reconstruction loss. To generate adversarial loss. The specific expression is:

[0082] mask loss = aDice(P, G) + βBCE(P, G)

[0083] holeloss=l1((1-mask)*output, (1-mask)*gt)

[0084] validArealoss=l1(mask*output,mask*gt)

[0085]

[0086]

[0087] Where Dice(), BCE(), and l1() are loss functions, and their specific expressions are as follows:

[0088]

[0089]

[0090]

[0091] Where P i,j G represents the pixel value predicted by the network. i,j The values ​​are the actual pixel values, Log is the natural logarithm, ∈ is a small constant to prevent the denominator from being zero, N is the number of samples, and G... i P i and are the true label and predicted value of the i-th sample, respectively. α and β are the weights of the two loss functions, which we set to (0.6, 0.4) in the network. mask is the text mask of the model, output is the final output of the model, and gt is the image after real erasure.

[0092] To verify the technical effects of this invention, its effectiveness will be verified through algorithmic comparison:

[0093] This invention compares its work with various commonly used algorithmic structures in the field of scene text erasure, using common image reconstruction metrics to evaluate the performance of each model, including:

[0094] Mean squared error (MSE), peak signal-to-noise ratio (PSNR), MSSIM (multi-scale structural similarity between two images), mean absolute difference (AGE) of gray levels between images, percentage of error pixels (pEPs), and percentage of clustering error pixels (pCEPs). Higher MSSIM and PSNR values, and lower AGE, pEPs, pCEPs, and MSEZ values, indicate better results.

[0095] We conducted comparative experiments with various algorithmic architectures on our self-built dataset, the SCUT-EnsExam dataset for erasing handwritten text from exam papers, and two scenario-based text erasure datasets, SCUT-Syn and SCUT-EnsText. SCUT-Syn is a synthetic dataset containing a rich variety of text patterns, covering various scenarios and contexts. These text samples come from different fields, including but not limited to social media, news reports, and medical records, to ensure the diversity and coverage of the dataset. Each sample has undergone carefully designed erasure processing to simulate the text erasure needs in real-world applications. The SCUT-EnsText dataset was proposed by Liu et al. in 2020. It is a large-scale real-world scenario-based text erasure dataset containing 2749 samples for training and 813 samples for testing. The final results are shown in Tables 1, 2, 3, and 4. Figure 7 , 8 As shown in Figures 9 and 10. Figure 7 The results are experimental findings of the algorithm of this invention and other algorithms on a self-built dataset. Figure 8 The experimental results of the algorithm of this invention and other algorithms on the SCUT-EnsExam dataset are shown below. Figure 9 The experimental results of the algorithm of this invention and other algorithms are shown on the SCUT-Syn dataset. Figure 10 This section presents the experimental results of the algorithm proposed in this invention and other algorithms on the SCUT-EnsText dataset. The first column of the image shows various images with handwritten text, the second column shows reference images with the handwritten text erased, the last column shows the experimental results obtained by the model proposed in this invention, and the other columns show the experimental results obtained by various comparison algorithms.

[0096] Table 1. Experimental data of the algorithm of this invention and other algorithms on a self-built dataset.

[0097]

[0098] Table 2. Experimental data of the algorithm of this invention and other algorithms on the SCUT-EnsExam dataset.

[0099]

[0100] Table 3 Experimental data of the algorithm of this invention and other algorithms on the SCUT-Syn dataset.

[0101] Model PSNR↑ MSSIM↑ MSE↓ AGE↓ pEPs↓ pCEPs↓ STE 25.40 90.12 0.65 9.49 0.0553 0.0347 EnsNet 37.36 96.44 0.21 1.73 0.0069 0.0020 FETNet 39.14 97.97 0.02 1.26 0.0046 0.0008 PERT 39.40 97.87 0.0002 1.41 0.0045 0.0006 EraseNet 37.03 97.86 0.0003 1.7834 0.0069 0.0024 This article's method 38.32 98.45 0.002 1.60 0.0048 0.0004

[0102] Table 4. Experimental data of the algorithm of this invention and other algorithms on SCUT-EnsText.

[0103]

[0104] In the qualitative results, the comparison of the aforementioned networks shows that EraseNet performs the worst in erasing various images, retaining a large amount of handwritten text regardless of whether the background is complex or pure white. While MTRNet can remove most of the handwritten text, its erasing effect is unstable, and it also erases non-handwritten text areas, failing to recognize and erase handwritten text in complex backgrounds. ViTEraser, although erasing the vast majority of handwritten text, still leaves some minor areas unremoved, and the image restoration after text erasure is poor. The model proposed in this invention, however, accurately erases handwritten text in both complex backgrounds and text overlap problems, and its restoration effect during the image restoration stage closely matches the real image. This demonstrates that our model can accurately segment and remove text, while generating semantically more reasonable results.

[0105] In terms of quantitative results, our model outperforms other models across various metrics, particularly showing significant improvements over the baseline EraseNet network. For example, PSNR is improved by 8.6 percentage points, MSSIM by 1.54 percentage points, while MSE, AGE, pEPs, and pCEPs are all reduced to varying degrees. We attribute this performance improvement to our proposed multi-scale feature fusion attention mechanism, which effectively locates handwritten text in various backgrounds, contributing to the integrity of text erasure.

[0106] Meanwhile, in the fine erasure stage, the removal of image noise and the extraction of non-local module features improve the integrity and rationality of the restored image. Compared with various network models in recent years, our proposed model achieves higher metrics. This indicates that our model is better suited for handwritten text erasure tasks.

[0107] The qualitative and quantitative results obtained from generalization experiments on two scene text erasure datasets demonstrate that our model is also highly competitive on scene text erasure datasets. This also indirectly proves the effectiveness of the algorithm proposed in this invention.

[0108] The above description of the embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0109] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for erasing handwritten text in a scene based on a two-stage deep learning network, characterized in that, Includes the following steps: S1. Construct a handwritten text erasure dataset containing several scenarios, and divide the dataset into training set, validation set and test set; S2, construct a two-stage generator-discriminator network to train the dataset and obtain the trained network model; Following S2, the system further includes S3, which involves conducting validation experiments on the trained network model using the constructed handwritten text erasure dataset and several other text erasure datasets, and comparing the model with several other network models. S1 includes the following steps: S11, the constructed dataset contains two types of images: the first is an image fused with the original image by generating text, and the second is an image where the original handwritten text is manually marked and erased. S12, using a handwritten font generator, randomly generate several images of different sizes containing random handwritten text. The handwritten text is distributed at random positions and of random size throughout the image. The image contains only handwritten text, with the font being black or colored and the background being pure white. S13, collect several background images, including single background and multiple background, then perform noise removal on the two images, compare the sizes of the two images to match the handwritten text and background images, and fuse the images to generate an image containing handwritten text; at this time, each set of data images includes an image containing handwritten text, an image containing only handwritten text, and an image containing only background. S14: Collect several original images containing handwritten text, erase the handwritten text and fill the background using image restoration methods; at this time, each set of data contains the original image containing handwritten text and the background image with the handwritten text removed; then, by subtracting these two images and removing noise, an image containing only handwritten text is obtained. S15, the constructed dataset contains over a thousand images, each image set including three types: images with handwritten text, reference images after erasing the handwritten text, and handwritten text mask images. Of these over a thousand images, 80% are used for training, 10% for validation, and 5% for testing; S2 includes the following steps: S21, in the generator stage of the network, the handwritten text in the image is located by a multi-scale feature fusion attention mechanism to help the network in the preliminary erasure stage recognize and erase the handwritten text. The image in the preliminary erasure stage still contains some unerased handwritten text, which is regarded as image noise. The image is then erased again by an iterative noise removal network. A multi-scale feature extraction module is designed to extract image features, and finally the image after erasing the handwritten text is obtained. S22, the network discriminator stage, inputs the image generated by the generator and the real image of erased handwritten text into the pre-trained VGG feature extraction network to extract features and compare them. Based on the difference analysis between the two, the generator network further generates the erased image. The multi-scale feature fusion attention mechanism in S21 includes using multiple parallel dilated convolutional layers with different sampling rates to capture contextual information at different scales in the input feature layer, weighted fusion to enhance the representation capability of the feature map, then obtaining two features in the channel dimension through average pooling, max pooling and multilayer perceptron operations, adding the two features, adjusting the feature dimension through an activation function to obtain the channel dimension feature, and finally obtaining the spatial dimension feature through pooling and convolutional layer operations, multiplying the channel dimension feature and the spatial dimension feature and adjusting the dimension to obtain the processed output feature; The multi-scale feature fusion attention mechanism process in S21 can be represented by the following formula: ； ； ； ； Where X represents the features input to the module, Y1 represents the features after dilated convolution, and A i (X) represents the feature map with dilation rate i; U represents the input to the Mc function, where σ represents the Sigmoid activation function; W1 and W0 represent the two convolution operations respectively; Umax and Uavg represent max pooling and average pooling of the features; f 7*7 This indicates a convolution operation with a kernel size of 7*7, and the final output feature obtained after module integration is Y.

2. The method according to claim 1, characterized in that... In S21, the generator network is trained using the loss function Gloss. ； in, For image mask loss, For image reconstruction loss, To generate adversarial losses, These are the weights of each loss function, with values ​​of 1, 1, 1, and 0.05 respectively.

3. The method according to claim 1, characterized in that, The multi-scale feature extraction module in S21 includes weighting the features obtained by different dilated convolutions, connecting them by the concat function, and restoring the feature scale by convolution to obtain the final features.

4. The method according to claim 1, characterized in that, The overall network in S2 is optimized and trained using a loss function, the expression of which is: ； Where a m a h a v and a L The weights for each loss function are 1, 1, 1, and 0.05, respectively. For image mask loss, Combined into image reconstruction loss, To generate adversarial loss; the specific expression is: ； ； ； ； ； Where Dice(), BCE(), and l1() are loss functions, and their specific expressions are as follows: ； ； ； Where P i,j G represents the pixel value predicted by the network. i,j The values ​​are the actual pixel values, Log is the natural logarithm, ∈ is a small constant to prevent the denominator from being zero, N is the number of samples, and G... i P i α and β are the true label and predicted value of the i-th sample, respectively; α and β are the weights of the two loss functions, which we set to (0.6, 0.4) in the network; mask is the text mask of the model, output is the final output of the model, and gt is the image after real erasure.

5. The method according to claim 1, characterized in that, Specifically, S3 includes testing on the constructed handwritten text erasure dataset and two scenario text erasure datasets, SCUT-Syn and SCUT-EnsText, and the exam paper handwritten text erasure dataset SCUT-EnsExam.

Images