A training method and device of a scene text erasing model
By using a weakly supervised training method and pseudo-labeling technology, a scene text erasure model was trained using a text detection dataset, which solved the problems of high-cost annotation and domain difference, and achieved efficient scene text erasure results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI HEHE INFORMATION TECH DEV
- Filing Date
- 2023-12-26
- Publication Date
- 2026-08-04
AI Technical Summary
In existing technologies, training scene text erasure models requires a large amount of pixel-level manual annotation, which leads to high annotation costs and the difference in data distribution between the synthetic domain and the real domain, resulting in a decrease in model performance in real-world scenarios.
We employ a weakly supervised training method, using a text detection dataset with bounding box level annotations. Through a parallel structure of background restoration and text erasure branches, we utilize text location labels to generate pseudo-labels for training, thus achieving an end-to-end scene text erasure model.
It significantly reduced the cost of training sample annotation, improved the model's performance in real-world scenarios, and achieved efficient scene text erasure.
Smart Images
Figure CN117709436B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for scene text removal (STR) model. Background Technology
[0002] Scene text erasure refers to the task of erasing text from an image and then restoring the background of the erased area to ensure a smooth transition with the surrounding areas. This task plays an important role in privacy protection and text image editing. With the development of deep learning technology, many studies in recent years have used deep neural networks (DNNs) to implement scene text erasure functionality through fully supervised learning training. However, optimizing deep neural networks requires a large number of labeled training samples, while the labels for scene text erasure tasks need to be manually annotated at the pixel level using image editing software. This greatly increases the annotation cost and limits the large-scale training of scene text erasure models.
[0003] To reduce the need for manual annotation, existing methods typically use synthetic data to train deep neural networks. However, there is a significant difference in data distribution between the synthetic and real domains, and scene text erasure models trained on synthetic datasets experience a significant performance drop when tested in real-world scenarios. Summary of the Invention
[0004] The technical problem to be solved by this invention is to propose a training method for a scene text erasure model with weakly supervised learning, which can train the scene text erasure model end-to-end using only a text detection dataset containing bounding box level annotations.
[0005] To address the aforementioned technical problems, this invention proposes a training method for a scene text erasure model, comprising the following steps: Step S1: Using a scene text detection dataset as the training set for the scene text erasure model; each training sample in the training set is an image, and each image has a text location label. Step S2: Selecting a baseline model. The order of steps S1 and S2 can be arbitrary or simultaneous. Step S3: Adjusting the network structure of the baseline model by changing the last classification layer of the baseline model to two parallel classification layers, thereby dividing the entire model into a background restoration branch and a text erasure branch, thus obtaining the scene text erasure model. Step S4: Train the background restoration branch; For any training sample in the training set, firstly, generate a text region mask using the text location labels in the input image, and then mask all text regions in the input image using the text region mask to obtain a background image; then, generate a random mask that randomly covers a portion of the background image, resulting in a partially occluded background image masked by the random mask; use the partially occluded background image as input to the background restoration branch, requiring the background restoration branch to predict the background filling content of the text regions and the randomly occluded regions; during training, use the background image as a label for this branch to supervise its learning process. Step S5: Train the text erasure branch; Replace the text regions in the input image with the background filling content of the text regions predicted by the background restoration branch, called the replaced image; use the input image as input to the text erasure branch, requiring the text erasure branch to predict the background filling content after the text regions are erased and restored; during training, use the replaced image as a pseudo-label for this branch to supervise its learning process. Step S6: The entire scene text erasure model is updated using gradient descent until the maximum number of training iterations specified by the user is reached; Steps S4 to S5 are repeated for each training sample in the training set.
[0006] Furthermore, in step S1, the text location label is a rectangle or polygon that surrounds each text region in the image.
[0007] Furthermore, in step S2, the baseline model is a deep learning model used for text erasure in an image, or for image restoration, or for image segmentation.
[0008] Furthermore, in step S3, the background restoration branch is used to predict the background content to be filled in the missing areas of the image; the text erasure branch is used to predict the background content to be filled in the text areas of the image after the text is removed; these two branches belong to the same neural network.
[0009] Furthermore, in step S4, supervision refers to requiring the background filling content of the randomly occluded area predicted by the background inpainting branch to be as consistent as possible with the original content of the randomly occluded area in the background image.
[0010] Furthermore, in step S5, supervision refers to requiring the background fill content of the text region predicted by the text erasure branch to be as consistent as possible with the background fill content of the text region in the replaced image.
[0011] Further, step S4 specifically includes the following sub-steps. Step S41: Randomly select a training image from the training set. And the corresponding text location labels, and establish a relationship between the text location labels and the input image. Text region masks of the same size Text region mask Used to label input images The text region in the text. Step S42: Use the text region mask. Input image Mask all text regions to obtain a background image that does not contain text regions. ; , where * denotes the Hadamard product. Step S43: In the background image Generate a random mask that randomly covers non-text areas. Random mask Used to mark background images Randomly occluded areas in the background image; Obtain a random mask The mask partially obscures the background image. ; Step S44: Remove the partially obscured background image. As input to the background restoration branch, the output of the background restoration branch is required. Includes predicted background content for text regions and randomly occluded regions; uses the background image when training the background inpainting branch. The output of the background repair branch is calculated as the label for this branch. Reconstruction losses , ; where loss() represents the loss function.
[0012] Further, step S5 specifically includes the following sub-steps. Step S51: Transfer the training images... The text region in the output is replaced with the background repair branch. The corresponding region is called the replaced image, serving as a pseudo-label for the text erasure task. ; Step S52: Transfer the training images As input to the text erasure branch, the output of the text erasure branch should be required. Includes background filling content after text regions have been erased and repaired; pseudo-labels are used when training the text erasure branch. Calculate the output of the text erasure branch Reconstruction losses , .
[0013] Further, step S6 specifically includes the following sub-steps. Step S61: Calculate the reconstruction loss of the background restoration branch. Reconstruction loss of text erasure branch Perform weighted summation and obtain the final loss. , ;in For hyperparameters. Step S62: Update the model parameters using the gradient descent algorithm, repeating steps S4 and S5 until the set maximum number of iterations is reached.
[0014] This invention also proposes a training device for a scene text erasure model, including a training data acquisition unit, a model initial construction unit, a model shaping unit, a background restoration branch training unit, a text erasure branch training unit, and an iterative training unit. The training data acquisition unit uses a scene text detection dataset as the training set for the scene text erasure model; each training sample in the training set is an image, and each image has a text location label. The model initial construction unit is used to select a baseline model. The model shaping unit is used to adjust the network structure of the baseline model, changing the last classification layer of the baseline model to two parallel classification layers, thereby dividing the entire model into a background restoration branch and a text erasure branch, resulting in the scene text erasure model. The background restoration branch training unit is used to train the background restoration branch. For any training sample in the training set, a text region mask is first generated using the text location labels in the input image. All text regions in the input image are then masked using the text region mask to obtain a background image. Next, a random mask is generated in this background image, randomly covering a portion of the region, resulting in a partially occluded background image. This partially occluded background image is used as input to the background restoration branch, which is required to predict the background filling content of the text regions and the randomly occluded regions. During training, the background image is used as a label to supervise the learning process of this branch. The text erasure branch training unit is used to train the text erasure branch. The text regions in the input image are replaced with the background filling content of the text regions predicted by the background restoration branch, resulting in a replaced image. The input image is used as input to the text erasure branch, which is required to predict the erased and restored background filling content of the text regions. During training, the replaced image is used as a pseudo-label to supervise the learning process of this branch. The iterative training unit is used to update the entire model using gradient descent until the maximum number of training iterations specified by the user is reached; and each training sample in the training set is sent to the background restoration branch training unit and the text erasure branch training unit for training.
[0015] The technical advantages achieved by this invention are as follows: By introducing image inpainting technology (i.e., background inpainting branch), a scene text erasure model can be trained in a weakly supervised manner using only text location labels from the text detection dataset. Compared with fully supervised training, weakly supervised training uses a larger amount of training data and has a simpler annotation method. This significantly reduces the cost of annotating training samples required to train the scene text erasure model, effectively alleviating the problem of insufficient training data and enabling the scene text erasure model to achieve higher accuracy. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating the training method for the scene text erasure model proposed in this invention.
[0017] Figure 2 This is a flowchart illustrating the specific process of step S4.
[0018] Figure 3 This is a flowchart illustrating the specific process of step S5.
[0019] Figure 4 This is a schematic diagram illustrating specific examples of steps S4 and S5.
[0020] Figure 5 This is a flowchart illustrating the specific process of step S6.
[0021] Figure 6 This is a schematic diagram of the training device for the scene text erasure model proposed in this invention.
[0022] The attached figures are labeled as follows: 1. Training data acquisition unit; 2. Initial model construction unit; 3. Model shaping unit; 4. Background restoration branch training unit; 5. Text erasure branch training unit; 6. Iterative training unit. Detailed Implementation
[0023] Please see Figure 1 The training method for the scene text erasure model proposed in this invention includes the following steps.
[0024] Step S1: Use a scene text detection (STD) dataset as the training set for the scene text erasure model proposed in this invention. The publicly available scene text detection dataset is, for example, a collection of publicly available datasets such as TotalText, ICDAR2015, Text OCR, COCO Text, RECTS, MLT19, ArT, and LSVT. Each training sample in the training set is an image, and each image has a text location label. The text location label is, for example, a rectangle or polygon surrounding each text region in the image.
[0025] Traditionally, scene text erasure models use the Scene Text Erasure (STR) dataset as their training set. Each training sample (image) in STR requires text erasure labeling, typically done manually at the pixel level using image editing software. The Scene Text Detection dataset, originally intended for use as a training set for scene text detection models, only requires text location labeling for each training sample, eliminating the need for pixel-level manual annotation. Experiments show that the average annotation time for each training sample in the Scene Text Erasure dataset is 450 seconds, while the average annotation time in the Scene Text Detection dataset is 172 seconds. Therefore, the annotation workload of the Scene Text Detection dataset is significantly less than that of the Scene Text Erasure dataset. Furthermore, the publicly available Scene Text Detection dataset contains a much larger amount of data than the publicly available Scene Text Erasure dataset; the former is tens of times larger. Therefore, using the Scene Text Detection dataset as a training set offers advantages such as high annotation efficiency and a large data volume.
[0026] Step S2: Select a deep learning model as a baseline model for text erasure, image restoration, or image segmentation in an image.
[0027] The order of steps S1 and S2 is not strictly limited; they can be performed either before or simultaneously.
[0028] Step S3: Adjust the network structure of the baseline model by changing the last classification layer of the baseline model to two parallel classification layers. This divides the entire model into a background restoration branch and a text erasure branch, resulting in the network structure of the scene text erasure model proposed in this invention. Existing deep learning models used for text erasure, image restoration, or image segmentation in images only have one classification layer at the end, which is used to predict the final result. This invention changes this single classification layer to two, thus obtaining two branches. The background restoration branch is used to predict the background content to be filled into the missing areas in the image. The text erasure branch is used to predict the background content to be filled into the text areas in the image after text removal. These two branches belong to the same neural network.
[0029] Step S4: Training the Background Inpainting Branch. For any training sample (input image) in the training set, firstly, a text region mask is generated using the text location labels in the input image. All text regions in the input image are then masked using the text region mask to obtain a background image. Next, a random mask is generated in this background image, randomly covering a portion of the image, resulting in a partially occluded background image. In other words, the input image displays the complete image content; the background image only displays all non-text regions of the input image, with the text regions obscured by the text region mask; the partially occluded background image only displays a portion of the background image content, with the remaining regions obscured by the random mask. This partially occluded background image is used as input to the background inpainting branch, which is required to learn to fill the text regions and randomly occluded regions with background content, i.e., the background inpainting branch predicts the background filling content for the text regions and randomly occluded regions. During the training of the background inpainting branch, the background image is used as a label to supervise its learning process, requiring that the background filling content predicted by the background inpainting branch for the randomly occluded regions be as consistent as possible with the original content of the randomly occluded regions in the background image.
[0030] Step S5: Train the text erasure branch. Replace the text region in the input image with the background fill content predicted by the background restoration branch, resulting in the replaced image. Use this input image directly as input to the text erasure branch. The text erasure branch is required to learn to erase the text in the input image and fill the erased text region with the background content; that is, the text erasure branch predicts the erased and restored background fill content of the text region. During the training of the text erasure branch, the replaced image is used as a pseudo-labelling tool to supervise its learning process. This requires that the background fill content of the text region predicted by the text erasure branch be as consistent as possible with the background fill content of the text region in the replaced image.
[0031] Pseudo-labeling technology involves using a model trained on labeled data to make predictions on unlabeled data, filtering samples based on the prediction results, and then re-inputting them into the model for training. Traditional pseudo-labeling techniques use labeled samples and pseudo-labeled samples to perform the same task. This invention cleverly borrows the idea of pseudo-labeling but uses labeled samples and pseudo-labeled samples to perform two different tasks, ultimately achieving the ability to train a scene text erasure model using only a scene text detection dataset.
[0032] Step S6: The entire scene text erasure model is updated using gradient descent until the manually specified maximum number of training iterations is reached. Steps S4 to S5 are repeated for each training sample in the training set.
[0033] This invention obtains a trained scene text erasure model through the above steps. In use, simply input an image containing text directly into the model; the output of the text erasure branch is the image with the text erased and the erased area repaired.
[0034] Please see Figure 2 Step S4 specifically includes the following sub-steps. Figure 4 This is a specific example of steps S4 and S5.
[0035] Step S41: Randomly select a training image from the training set. And the corresponding text location labels, and establish a relationship between the text location labels and the input image. Text region masks of the same size Text region mask Used to label input images The text region is defined in the image. Each pixel in the text region mask image is distinguished from the other by two different values of a certain image parameter. For example, a pixel with a value of 1 (white) indicates that it is in the text region, and a pixel with a value of 0 (black) indicates that it is in the non-text background region.
[0036] Step S42: Utilize text region mask Input image Set the pixel values of all text regions to 0 (i.e., mask all text regions, making them appear black) to obtain a background image that does not contain text regions. . , where * and * in subsequent formulas both represent the Hadamard product.
[0037] Step S43: In the background image Generate a random mask that randomly covers non-text areas. Random mask Used to mark background images Randomly occluded areas in the background image. Obtain a random mask The mask partially obscures the background image. . Or, a random mask. Instead of randomly covering a portion of the image, it can partially cover the input image. In the text region, this completely random method is simple to implement and does not affect the result.
[0038] Step S44: Remove the partially obscured background image As input to the background restoration branch, the branch is required to learn to fill text regions (i.e., text region masks) with background content. (white area in the image) and random occlusion areas (i.e., random mask) (the black area in the image), which is the output of the background repair branch. Includes predicted background content for text regions and randomly occluded areas. The background image is used when training the background inpainting branch. The output of the background repair branch is calculated as the label for this branch. Reconstruction losses , Here, loss() represents the loss function, which can be any conventional reconstruction loss function such as the L1 loss function.
[0039] Please see Figure 3 Step S5 specifically includes the following sub-steps. Figure 4 This is a specific example of steps S4 and S5.
[0040] Step S51: Transfer the training images The text region in the output is replaced with the background repair branch. The corresponding region is called the replaced image, serving as a pseudo-label for the text erasure task. . .
[0041] Step S52: Transfer the training images As input to the text erasure branch, the text erasure branch is required to learn how to erase the input image. The text in the text is filled with background content, i.e., the output of the text erasure branch is required. This includes the background fill content after the text region has been erased and repaired. Pseudo-labels are used when training the text erasure branch. Calculate the output of the text erasure branch Reconstruction losses , Here, loss() represents the loss function, which can be any conventional reconstruction loss function such as the L1 loss function.
[0042] Please see Figure 5 Step S6 specifically includes the following sub-steps.
[0043] Step S61: Reconstruct the loss of the background repair branch Reconstruction loss of text erasure branch Perform weighted summation and obtain the final loss. , .in This is a hyperparameter, for example, set to 2. Hyperparameters are external configuration variables used by data scientists to manage the training of machine learning models.
[0044] Step S62: Update the model parameters using the gradient descent algorithm, and repeat steps S4 and S5 until the set maximum number of iterations is reached.
[0045] Please see Figure 6 The training device for the scene text erasure model proposed in this invention includes a training data acquisition unit 1, a model initial construction unit 2, a model forming unit 3, a background repair branch training unit 4, a text erasure branch training unit 5, and an iterative training unit 6. Figure 6 The device shown corresponds to Figure 1 The method shown.
[0046] The training data acquisition unit 1 is used to employ the scene text detection dataset as the training set for the scene text erasure model proposed in this invention. Each training sample in the training set is an image, and each image has a text location label.
[0047] The initial model building unit 2 is used to select a deep learning model for text erasure, image restoration, or image segmentation in an image as a baseline model.
[0048] The model shaping unit 3 is used to adjust the network structure of the baseline model, changing the last classification layer of the baseline model into two parallel classification layers, thereby dividing the entire model into a background restoration branch and a text erasure branch, thus obtaining the network structure of the scene text erasure model proposed in this invention.
[0049] The background restoration branch training unit 4 is used to train the background restoration branch. The partially occluded background image is used as input to the background restoration branch, which is required to learn to fill text areas and randomly occluded areas with background content. During the training process, the background image is used as the label for the branch to supervise its learning process.
[0050] The text erasure branch training unit 5 is used to train the text erasure branch. The input image is used as the input to the text erasure branch, which is required to learn to erase the text in the input image and fill the erased text area with background content. During the training of the text erasure branch, the replaced image is used as a pseudo-label for the branch to supervise its learning process.
[0051] The iterative training unit 6 is used to update the entire model using gradient descent until the maximum number of training iterations specified by the user is reached; and each training sample in the training set is sent to the background restoration branch training unit 4 and the text erasure branch training unit 5 for training.
[0052] Compared with existing training methods for scene text erasure models, this invention has the following technical innovations and beneficial effects. First, this invention reduces the annotation of the training set from pixel-level text content erasure labels to text bounding box labels, significantly reducing the data annotation cost of scene text erasure tasks. Second, this invention can directly use existing large-scale scene text detection data as the training set without additional annotation, thereby training a high-performance scene text erasure model. Third, this invention innovatively designs a background restoration branch, using only text bounding box annotations to generate pseudo-labels for the text erasure branch based on image restoration principles. Furthermore, a text erasure branch is designed, utilizing pseudo-labels to train the scene text erasure model end-to-end, thus avoiding the need for manual annotation of the scene text erasure training set.
[0053] In summary, this invention first trains a background restoration branch to reconstruct background content generation knowledge in non-text regions. Then, this branch is used to restore all text regions in the image to background content, serving as pseudo-labels for training the text erasure branch. Finally, the text erasure branch is trained directly using these pseudo-labels, thus achieving end-to-end scene text erasure functionality.
[0054] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for training a scene text erasing model, characterized in that, Includes the following steps; Step S1: Use the scene text detection dataset as the training set for the scene text erasure model; each training sample in the training set is an image, and each image has a text location label; Step S2: Select a deep learning model as the baseline model for text erasure, image inpainting, or image segmentation in an image. The order of steps S1 and S2 can be arbitrary, or they can be performed simultaneously. Step S3: Adjust the network structure of the baseline model by changing the last classification layer of the baseline model to two parallel classification layers, thereby dividing the entire model into a background restoration branch and a text erasure branch, thus obtaining the scene text erasure model; Step S4: Training the background repair branch; For any training sample in the training set, firstly, a text region mask is generated using the text location labels in the input image. All text regions in the input image are then masked using the text region mask to obtain a background image. Then, a random mask is generated in the background image to randomly cover a portion of the region, resulting in a partially occluded background image masked by the random mask. The partially occluded background image is used as input to the background restoration branch, which is required to predict the background filling content of the text region and the randomly occluded region; during training, the background image is used as the label of the branch to supervise its learning process. Step S5: Train the text erasure branch; replace the text region in the input image with the background filling content of the text region predicted by the background restoration branch, which is called the replaced image; use the input image as the input of the text erasure branch, and require the text erasure branch to predict the background filling content of the erased and restored text region; use the replaced image as the pseudo-label of the branch to supervise its learning process during training; Step S6: The entire scene text erasure model is updated using gradient descent until the maximum number of training iterations specified by the user is reached; Steps S4 to S5 are repeated for each training sample in the training set. 2.The method of claim 1, wherein, In step S1, the text location label is a rectangle or polygon that surrounds each text region in the image.
3. The training method for the scene text erasure model according to claim 1, characterized in that, In step S3, the background restoration branch is used to predict the background content to be filled in the missing areas of the image; the text erasure branch is used to predict the background content to be filled in the text areas of the image after the text is removed; these two branches belong to the same neural network.
4. The training method for the scene text erasure model according to claim 1, characterized in that, In step S4, supervision means requiring the background filling content of the randomly occluded area predicted by the background inpainting branch to be as consistent as possible with the original content of the randomly occluded area in the background image.
5. The training method for the scene text erasure model according to claim 1, characterized in that, In step S5, supervision means requiring the background fill content of the text region predicted by the text erasure branch to be as consistent as possible with the background fill content of the text region in the replaced image.
6. The training method for the scene text erasure model according to claim 1, characterized in that, Step S4 specifically includes the following sub-steps; Step S41: Randomly select a training image from the training set. And the corresponding text location labels, and establish a relationship between the text location labels and the input image. Text region masks of the same size Text region mask Used to label input images Text area in; Step S42: Utilize text region mask Input image Mask all text regions to obtain a background image that does not contain text regions. ; , where * represents the Hadamard product; Step S43: In the background image Generate a random mask that randomly covers non-text areas. Random mask Used to mark background images Randomly occluded areas in the background image; Obtain a random mask The mask partially obscures the background image. ; ; Step S44: Remove the partially obscured background image As input to the background restoration branch, the output of the background restoration branch is required. Includes predicted background content for text regions and randomly occluded areas; When training the background restoration branch, use the background image. The output of the background repair branch is calculated as the label for this branch. Reconstruction losses , ; where loss() represents the loss function.
7. The training method for the scene text erasure model according to claim 6, characterized in that, Step S5 specifically includes the following sub-steps; Step S51: Transfer the training images The text region in the output is replaced with the background repair branch. The corresponding region is called the replaced image, serving as a pseudo-label for the text erasure task. ; ; Step S52: Transfer the training images As input to the text erasure branch, the output of the text erasure branch should be required. Includes background filling content after text regions have been erased and repaired; pseudo-labels are used when training the text erasure branch. Calculate the output of the text erasure branch Reconstruction losses , .
8. The training method for the scene text erasure model according to claim 7, characterized in that, Step S6 specifically includes the following sub-steps; Step S61: Reconstruct the loss of the background repair branch Reconstruction loss of text erasure branch Perform weighted summation and obtain the final loss. , ;in For hyperparameters; Step S62: Update the model parameters using the gradient descent algorithm, and repeat steps S4 and S5 until the set maximum number of iterations is reached.
9. A training device for a scene text erasure model, characterized in that, It includes training data acquisition unit, initial model building unit, model shaping unit, background restoration branch training unit, text erasure branch training unit, and iterative training unit; The training data acquisition unit is used to use the scene text detection dataset as the training set for the scene text erasure model; each training sample in the training set is an image, and each image has a text location label. The initial model building unit is used to select a deep learning model for text erasure, image restoration, or image segmentation as a baseline model. The model shaping unit is used to adjust the network structure of the baseline model, changing the last classification layer of the baseline model into two parallel classification layers, thereby dividing the entire model into a background restoration branch and a text erasure branch, resulting in a scene text erasure model. The background restoration branch training unit is used to train the background restoration branch. For any training sample in the training set, a text region mask is first generated using the text location labels in the input image. All text regions in the input image are masked using the text region mask to obtain a background image. Then, a random mask that randomly covers a portion of the background image is generated, and a partially occluded background image is obtained by masking the background image with the random mask. The partially occluded background image is used as input to the background inpainting branch, which is required to predict the background filling content of the text region and the randomly occluded region; during training, the background image is used as the label of the branch to supervise its learning process. The text erasure branch training unit is used to train the text erasure branch; the text region in the input image is replaced with the background filling content of the text region predicted by the background restoration branch, which is called the replaced image; the input image is used as the input of the text erasure branch, which is required to predict the background filling content of the text region after erasure and restoration; during training, the replaced image is used as the pseudo-label of the branch to supervise its learning process. The iterative training unit is used to update the entire model using gradient descent until the maximum number of training iterations specified by the user is reached. Each training sample in the training set is then fed into the background restoration branch training unit and the text erasure branch training unit for training.