Handwritten content removal method and device, equipment, storage medium and program product

Abstract

The invention discloses a handwritten content removal method and device, equipment, a storage medium and a program product, and relates to the technical field of image processing, and the method comprises the steps: carrying out the multi-scale feature extraction of a first picture containing handwritten content, and obtaining the features of multiple scales; performing non-local feature extraction on the features of any scale to obtain global features; performing multi-level decoding based on the global feature, the multi-scale features and the first picture to obtain a second picture; the second picture is a result obtained by erasing the handwritten content of the first picture. The handwritten content in the picture with the color background can be quickly and accurately removed on the basis of the method, and a scheme for effectively removing the handwritten content in a picture scene with the color background is provided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular, to a handwritten content removing method and device, equipment, storage medium and program product. BACKGROUND

[0002] Currently, some auxiliary learning electronic products provide functions of searching for questions by photographing and recognizing teaching materials by photographing. A user can obtain answers to questions searched by a terminal device or knowledge related to content in a teaching material by photographing the content in the question or the teaching material using the terminal device with the functions.

[0003] However, in some cases, the photographed picture may contain handwritten content. In order to prevent the handwritten content from interfering with the searching for questions by photographing and the recognizing teaching materials by photographing, the handwritten content in the picture needs to be removed. However, the current handwritten content removing method mainly targets the handwritten scene of white background and black characters and cannot remove the handwritten content in a scene with a color background (for example, a teaching material). SUMMARY

[0004] In view of the above problems, the present application provides a handwritten content removing method, device, equipment, storage medium and program product to remove handwritten content in a picture scene with a color background. The specific solutions are as follows:

[0005] The first aspect of the present application provides a handwritten content removing method, comprising:

[0006] performing multi-scale feature extraction on a first picture containing handwritten content to obtain features of multiple scales;

[0007] performing non-local feature extraction on features of any scale to obtain global features;

[0008] performing multi-level decoding based on the global features, the features of multiple scales and the first picture to obtain a second picture; the second picture is a result of erasing handwritten content from the first picture.

[0009] In a possible implementation, the multi-scale feature extraction on the first picture containing handwritten content comprises:

[0010] performing normalization processing on the first picture to obtain a normalized picture with a data distribution interval of [-1, 1];

[0011] performing multi-scale feature extraction on the normalized picture to obtain the features of multiple scales;

[0012] Correspondingly, the process of obtaining the second picture comprises: performing multi-level decoding based on the global features, the features of multiple scales and the normalized picture.

[0013] In a possible implementation, the handwriting content removing method is implemented based on a pre-trained network model, and the network model comprises:

[0014] a multi-feature extraction module, an input of the multi-feature extraction module being the normalized picture, and an output of the multi-feature extraction module being the features of the multiple scales;

[0015] a decoding module, an input of the decoding module being the features of the multiple scales and the normalized picture, and an output of the decoding module being the second picture.

[0016] In a possible implementation, the multi-feature extraction module comprises M encoding layers, an input of a first encoding layer being the normalized picture, and an input of each non-first encoding layer being an output of a previous encoding layer; different encoding layers output features of different scales, and a scale of features output by each non-first encoding layer is smaller than a scale of features output by a previous encoding layer.

[0017] The decoding module comprises a non-local network layer and M+1 decoding layers connected with the non-local network layer, an input of the non-local network layer being an output of an i-th encoding layer, an output of the non-local network layer being directly or indirectly connected with an input of at least one decoding layer, the at least one decoding layer comprising an M-i+1-th decoding layer, an input of a first decoding layer of the decoding module comprising an output of a last encoding layer, an input of a last decoding layer of the decoding module comprising an output of a previous decoding layer and the normalized picture, an input of another decoding layer comprising an output of a previous decoding layer and an output of an encoding layer if i=M, and an input of another decoding layer comprising an output of a previous decoding layer and an output of an encoding layer if i≠M; the M-i+1-th decoding layer further comprises an output of a previous decoding layer if i≠M, and features output by an encoding layer and a decoding layer connected with an input of the same decoding layer have the same scale.

[0018] In a possible implementation, the network model is trained in the following manner:

[0019] a normalized picture of a sample picture in a target training data set is taken as an input of the network model, and parameters of the network model are updated in a manner that a peak signal-to-noise ratio between a picture output by the network model and a label of the sample picture becomes larger, and a structural similarity between the picture output by the network model and the label of the sample picture becomes larger;

[0020] the sample picture in the target training data set at least comprises a picture with a color background and handwriting content, and the label of the sample picture is a picture obtained by removing handwriting content from the sample picture.

[0021] In one possible implementation, updating the parameters of the network model with the goal of increasing the peak signal-to-noise ratio of the image output by the network model and the label of the sample image, and increasing the structural similarity between the image output by the network model and the label of the sample image, includes:

[0022] When the network model is updated a predetermined number of times, the parameters of the network model are updated with the goal of increasing the peak signal-to-noise ratio of the label of the image output by the network model and the sample image;

[0023] After the network model is updated a preset number of times, the parameters of the network model are updated with the goal of increasing the structural similarity between the labels of the picture output by the network model and the sample picture; or, after the network model is updated a preset number of times, the parameters of the network model are updated with the goal of increasing the peak signal-to-noise ratio of the labels of the picture output by the network model and the sample picture, and increasing the structural similarity between the labels of the picture output by the network model and the sample picture.

[0024] In one possible implementation, updating the parameters of the network model with the goal of increasing the peak signal-to-noise ratio of the image output by the network model and the label of the sample image, and increasing the structural similarity between the image output by the network model and the label of the sample image, includes:

[0025] When the network model is updated a preset number of times, the parameters of the network model are updated with the goal of increasing the structural similarity between the labels of the image output by the network model and the sample image;

[0026] After the network model is updated a preset number of times, the parameters of the network model are updated with the goal of increasing the peak signal-to-noise ratio of the image output by the network model and the label of the sample image; or, after the network model is updated a preset number of times, the parameters of the network model are updated with the goal of increasing the peak signal-to-noise ratio of the image output by the network model and the label of the sample image, and increasing the structural similarity between the image output by the network model and the label of the sample image.

[0027] A second aspect of the present application provides a handwritten content removal device, comprising:

[0028] a feature extraction unit, configured to perform multi-scale feature extraction on the first image containing handwritten content to obtain features at multiple scales;

[0029] The decoding unit is configured to perform non-local feature extraction on the features of any scale to obtain global features, perform decoding at multiple levels based on the global features, the features of the multiple scales, and the first picture to obtain a second picture, and the second picture is a result of handwriting content erasing on the first picture.

[0030] The third aspect of the present application provides a learning machine, comprising:

[0031] The image acquisition module is configured to acquire a picture.

[0032] The processor is configured to process the picture acquired by the image acquisition module by using the handwriting content removing method of the first aspect or any implementation manner of the first aspect, so as to remove the handwriting content in the picture.

[0033] The fourth aspect of the present application provides a computer program product, comprising computer readable instructions, when the computer readable instructions run on an electronic device, the electronic device implements the handwriting content removing method of the first aspect or any implementation manner of the first aspect.

[0034] The fifth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected with the processor, wherein:

[0035] The memory is configured to store a computer program.

[0036] The processor is configured to execute the computer program, so that the electronic device can implement the handwriting content removing method of the first aspect or any implementation manner of the first aspect.

[0037] The sixth aspect of the present application provides a computer storage medium, the storage medium carries one or more computer programs, when the one or more computer programs are executed by an electronic device, the electronic device can implement the handwriting content removing method of the first aspect or any implementation manner of the first aspect.

[0038] By means of the above technical solution, the handwriting content removing method, device, equipment, storage medium and program product provided by the present application perform multi-scale feature extraction on a first picture containing handwriting content to obtain features of multiple scales, perform non-local feature extraction on the features of any scale to obtain global features, perform decoding at multiple levels based on the global features, the features of the multiple scales, and the first picture to obtain a second picture, and the second picture is a result of handwriting content erasing on the first picture. Based on the present application, handwriting content in a picture with a color background can be quickly and accurately removed, and a scheme for effectively removing handwriting content in a picture scene with a color background is provided. BRIEF DESCRIPTION OF DRAWINGS

[0039] The above and other features, advantages, and aspects of the present disclosure will become more apparent with reference to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, similar or same reference numerals are used to denote similar or same elements. It is to be understood that the drawings are schematic, and elements are not necessarily to scale.

[0040] Figure 1 An architecture diagram of an existing handwritten content removal scheme provided by the present application;

[0041] Figure 2 An implementation flowchart of a handwritten content removal method provided by the present application;

[0042] Figure 3 An implementation flowchart of multi-scale feature extraction on a first picture provided by the present application;

[0043] Figure 4 A structural schematic diagram of a network model provided by the present application;

[0044] Figure 5a Another structural schematic diagram of a network model provided by the present application;

[0045] Figure 5b Still another structural schematic diagram of a network model provided by the present application;

[0046] Figure 5c Still another structural schematic diagram of a network model provided by the present application;

[0047] Figure 5d Still another structural schematic diagram of a network model provided by the present application;

[0048] Figure 6 A structural schematic diagram of a handwritten content removal device provided by the present application;

[0049] Figure 7 A structural schematic diagram of a learning machine provided by the present application;

[0050] Figure 8 A structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0051] The embodiments of the present application are described below in conjunction with the accompanying drawings. The terms used in the embodiment part of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0052] The embodiments of the present application are described below in conjunction with the accompanying drawings. It is known to those skilled in the art that, as technology develops and new scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0053] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0054] In order to better understand the difference between this application and the prior art, the handwritten content removal solution of the prior art is first described. Figure 1 As shown in FIG, the architecture diagram of the existing handwriting content removal solution provided by this application includes an encoding module (Encoder), a mask prediction module (Mask Prediction), a decoding module (Decoder) and a refiner module (Refiner).

[0055] Among them, the encoding module is used to extract multi-scale features of the input image, the mask prediction module is used to process the minimum-scale features and predict the handwriting content mask, and the decoding module is used to process the minimum-scale features to generate a coarse repaired image; the refining module is used to process the multi-scale features (not shown in the figure), the handwriting content mask and the coarse repaired image to generate a refined repaired image. The handwriting content removal effect of the refined repaired image is better than that of the coarse repaired image.

[0056] This research found that Figure 1 The solution shown is mainly for handwritten scenes with black characters on a white background, such as test papers, and cannot effectively remove real handwritten content on textbooks with colored backgrounds. Figure 1 The solution shown was originally designed to run on a server. It has a large number of parameters (23.5M) and a large amount of computing power (1190 GFlops of floating-point computing power). If deployed on a device such as a learning machine, it would take more than 1 second to run, resulting in long user waiting times and a poor user experience.

[0057] In order to at least partially solve the technical problems existing in the prior art, this application is proposed. The solution of this application is explained below.

[0058] like Figure 2 As shown in FIG. 1 , a flowchart of an implementation of the handwritten content removal method provided in an embodiment of the present application may include:

[0059] Step S201: multi-scale feature extraction is performed on a picture containing handwritten content (for ease of narration and distinction, denoted as a first picture), to obtain features of multiple scales.

[0060] The first picture can be a picture obtained by photographing a text region (the text region has handwritten text, and the text region can or can not have a color background) by an image acquisition module, or can be a picture read from a storage unit.

[0061] In addition to containing handwritten content, the first picture can also contain printed content. The handwritten content in the first picture can include, but is not limited to, at least one of the following: handwritten characters, handwritten symbols, hand-drawn lines, hand-drawn tables, hand-drawn graffiti (such as various drawings drawn by hand, for example, animals drawn by hand, plants drawn by hand, devices drawn by hand, etc.), and the like; in addition to a color background, the printed content in the first picture can also include, but is not limited to, at least one of the following: printed characters, printed symbols, printed lines, printed tables, and the like.

[0062] Step S202: non-local feature extraction is performed on features of any scale, to obtain global features.

[0063] The features of any scale can be features of the smallest scale among the multiple scale features obtained in the previous step, can be features of the largest scale among the multiple scale features obtained in the previous step, or of course, can be features of other scales among the multiple scale features obtained in the previous step.

[0064] Corresponding to features of different scales, the amount of calculation for feature extraction is different, the larger the feature scale, the larger the amount of calculation for non-local feature extraction, but the better the handwritten content removal effect.

[0065] Through non-local feature extraction, the dependency between two feature points in a feature map can be captured, that is, a long-distance dependency at a pixel level can be captured.

[0066] Step S203: multiple levels of decoding are performed based on the global features, the multiple scale features, and the first picture, to obtain a second picture. The second picture is a result of handwritten content erasure performed on the first picture.

[0067] The decoding of the last level is implemented based on the decoding result of the previous level and the first picture, or the decoding of the last level is implemented based on the global features, the decoding result of the previous level, and the first picture. In the decoding of the multiple non-last levels, the decoding of at least one non-last level is implemented based on the global features and features of the same scale as the global features among the multiple scale features, the decoding of the other non-last levels is implemented based on the output of the previous decoding layer and the output of one encoding layer, or the decoding of the first level is implemented based on the output of the last encoding layer.

[0068] For example, in the case that the global feature is a non-local feature extraction result of the minimum scale feature, the decoding of the first level is implemented based on the global feature and the minimum scale feature, and the decoding of the non-first level is implemented based on the decoding result of the previous level and the feature of the same scale as the decoding result of the previous level or the first picture. That is, the sizes of the input features required by the decoding of different levels are different, and the lower the decoding level, the smaller the size of the input feature. The decoding of the last level is implemented based on the decoding result of the previous level and the first picture.

[0069] The handwriting content removal method provided by the embodiments of the present application performs multi-scale feature extraction on the first picture containing handwriting content to obtain features of multiple scales; performs non-local feature extraction on the features of any scale to obtain global features; performs multi-level decoding based on the global features, the features of multiple scales, and the first picture to obtain a second picture; and the second picture is the result of handwriting content erasing on the first picture. Based on the present application, the handwriting content in the picture with a color background can be quickly and accurately removed, and a scheme for effectively removing handwriting content in a picture scene with a color background is provided.

[0070] Of course, the present application is not only applicable to handwriting content removal of pictures with a color background, but also applicable to handwriting content removal of pictures without a color background.

[0071] Based on the present application, the accuracy of subsequent tasks can also be improved. For example, if there is handwriting text in the first picture, the second picture obtained based on the present application effectively removes the handwriting text in the picture, and text recognition on the second picture can improve the accuracy of text recognition and enhance the anti-interference ability of handwriting text. For another example, if there is hand-drawn graffiti in the first picture, the second picture obtained based on the present application effectively removes the hand-drawn graffiti in the picture, and textbook recognition based on the second picture can accurately identify the publisher of the textbook, and then obtain the corresponding knowledge of the textbook (such as analysis, etc.), thereby improving the accuracy of textbook recognition.

[0072] Further, after obtaining the second picture, the second picture can also be displayed for the user to view or perform other operations (such as editing, transmission, printing, etc.) on the second picture.

[0073] In an optional embodiment, one implementation manner of the above multi-scale feature extraction on the first picture can be as follows:

[0074] The first picture is normalized to obtain a normalized picture with a data distribution interval of [0, 1].

[0075] That is, in the normalized picture, the value of each pixel point is in the range of [0, 1]. The first picture can be normalized by using the following normalization formula:

[0076] P = Image / 255.0 (1)

[0077] Wherein, P is the normalized result corresponding to any pixel in the first picture; image is the value of any pixel in the first picture. That is, each pixel in the first picture is normalized by using the normalization method shown in formula (1).

[0078] The normalized picture is subjected to multi-scale feature extraction to obtain features of multiple scales.

[0079] The present inventor found that the data distribution of the first picture is normalized to the range of [0, 1], which has low discrimination. Although the removal effect of the handwritten content in the picture scene with a colored background can be improved compared with the prior art, the effect still needs to be further improved.

[0080] In an optional embodiment, an implementation flowchart of the above multi-scale feature extraction on the first picture can include Figure 3

[0081] Step S301: The first picture is normalized to obtain a normalized picture with a data distribution interval of [-1, 1].

[0082] That is, in the normalized picture, the value of each pixel point is in the range of [-1, 1]. The first picture can be normalized by using the following normalization formula:

[0083] P = Image / 127.5-1 (2)

[0084] Wherein, P is the normalized result corresponding to any pixel in the first picture; image is the value of any pixel in the first picture. That is, each pixel in the first picture is normalized by using the normalization method shown in formula (2).

[0085] Step S302: The normalized picture is subjected to multi-scale feature extraction to obtain features of multiple scales.

[0086] ​Compared with the former normalization of the data distribution of the first picture to the range of [0, 1], the normalization of the data distribution of the first picture to the range of [-1, 1] has better removal effect on the handwritten content in the picture scene with color background and higher accuracy. Experiments prove that, compared with the former normalization of the data distribution of the first picture to the range of [0, 1], the Peak Signal-to-Noise Ratio (PSNR) of the picture after removing the handwritten content and the original picture without handwritten content is increased by about 4 points, that is, the value of PSNR is increased by 4 dB.

[0087] In an optional embodiment, the handwritten content removal method provided by the present application can be implemented based on a pre-trained network model, as shown in Figure 4 The network model provided by the embodiment of the present application can include:

[0088] The multi-feature extraction module 401 and the decoding module 402;

[0089] The input of the multi-feature extraction module 401 is the normalized picture, and the output is the features of multiple scales. That is, the multi-feature extraction module 401 is used for multi-scale feature extraction on the normalized picture of the first picture to obtain features of multiple scales.

[0090] The input of the decoding module 402 is the features of multiple scales and the normalized picture, and the output is the second picture. That is, the decoding module 402 is used for non-local feature extraction on the features of any scale to obtain global features, and performs multiple levels of decoding based on the global features, the features of multiple scales and the first picture to obtain the second picture.

[0091] As an example, in the case where the decoding module is used for non-local feature extraction on the features of the smallest scale, the decoding of the first level is implemented based on the global features and the features of the smallest scale, and the decoding of the non-first level is implemented based on the decoding result of the previous level and the features of the same scale as the decoding result of the previous level or the first picture.

[0092] As shown in Figure 5a Another structure schematic diagram of the network model provided by the embodiment of the present application is shown in

[0093] The multi-feature extraction module 401 can include multiple (for ease of description, denoted as M, M is an integer greater than 1) encoding layers; wherein the input of the first encoding layer is the normalized picture, the input of each non-first encoding layer is the output of the previous encoding layer, different encoding layers output features of different scales, and the scale of the features output by each non-first encoding layer is smaller than the scale of the features output by the previous encoding layer.

[0094] Figure 5a In the example shown, the multi-feature extraction module 401 includes four coding layers, coding layer 1 is the first coding layer, coding layer 2 is the second coding layer, coding layer 3 is the third coding layer, and coding layer 4 is the fourth coding layer. Coding layers 2 to 4 are all non-first coding layers. The scale of the features output by coding layer 1 is the largest, the scale of the features output by coding layer 2 is smaller than the scale of the features output by coding layer 1, the scale of the features output by coding layer 3 is smaller than the scale of the features output by coding layer 2, and the scale of the features output by coding layer 4 is smaller than the scale of the features output by coding layer 3. Obviously, Figure 5a In the example shown, the multi-feature extraction module 401 outputs features at four scales.

[0095] Optionally, each encoding layer can include a downsampling layer and a convolutional layer. The downsampling layer can be a convolutional layer or a pooling layer (max pooling layer or mean pooling layer). The downsampling layer can use 2x downsampling. The convolutional layer is used to perform convolution processing on the output of the downsampling layer to obtain features at a certain scale.

[0096] The convolutional layer in each encoding layer can be a convolutional layer with an activation function. The output of the convolutional layer is the output of the encoding layer. The activation function can be a common activation function, such as Swish, GELU, ReLU, Leaky ReLU, Softmax, tanh, etc.

[0097] The decoding module 402 may include a non-local network layer and multiple (M+1) levels of decoding layers connected to the non-local network layer. The input of the non-local network layer is the output of the i-th (i is any value among {1, 2, 3, ..., M}) coding layer. The output of the non-local network layer is directly or indirectly connected to the input of at least one decoding layer, and the at least one decoding layer includes the M-i+1-th decoding layer. That is, the at least one decoding layer may include only the M-i+1-th decoding layer, or may include the M-i+1-th decoding layer and other decoding layers. In other words, the input of the M-i+1-th decoding layer in the decoding module is directly connected to the output of the non-local network layer, while the inputs of the other decoding layers in the decoding module are indirectly connected to the output of the non-local network layer, or are not connected to the output of the non-local network layer.

[0098] Among them, the first decoding layer of the decoding module (such as Figure 5a The input of the decoding layer 1 in includes the last encoding layer (such as Figure 5a The output of the encoding layer 4 in the decoding module, the last decoding layer of the decoding module (such as Figure 5a The input of the decoding layer 5 in includes the previous decoding layer (such as Figure 5athe output of the previous decoding layer and the output of one encoding layer; if i≠M, the input of the M-i+1 decoding layer further comprises the output of the previous decoding layer, and the input of the other decoding layers (except the first decoding layer, the last decoding layer and the M-i+1 decoding layer) comprises the output of the previous decoding layer and the output of one encoding layer; the scale of the features of the output of the encoding layer and the decoding layer connected with the input of the same decoding layer is the same.

[0099] That is, for the first decoding layer in the decoding module (e.g., the decoding layer 1 in the decoding module 5a), the input of the first decoding layer is connected with the output of the last non-encoding layer, regardless of whether the input of the first decoding layer is connected with the output of the non-local network layer. Figure 5a

[0100] That is, for the last decoding layer in the decoding module (e.g., the decoding layer 5 in the decoding module 5a), the input of the last decoding layer is connected with the output of the previous decoding layer and the normalized picture, regardless of whether the input of the last decoding layer is connected with the output of the non-local network layer. If the input of the last decoding layer is connected with the output of the non-local network layer, the output of the non-local network layer is indirectly connected with the input of the last decoding layer (e.g., the output of the non-local network layer is connected with the input of the last decoding layer after being up-sampled). Figure 5a

[0101] That is, for the other decoding layers in the decoding module (e.g., the decoding layers 2-4 in the decoding module 5a), if the input of the decoding layer is directly connected with the output of the non-local network layer, the input of the decoding layer further comprises the output of the previous decoding layer, and if the input of the decoding layer is indirectly connected with the output of the non-local network layer, or the input of the decoding layer is not connected with the output of the non-local network layer, the input of the decoding layer further comprises the output of the previous decoding layer and the output of one encoding layer; the scale of the features of the output of the encoding layer and the decoding layer connected with the input of the same decoding layer is the same.

[0102] In the case that the output of the non-local network layer is only connected with the input of the M-i+1 decoding layer, the input of the M-i+1 decoding layer further comprises the output of the last encoding layer or the output of the previous decoding layer; the input of the last decoding layer comprises the output of the previous decoding layer and the normalized picture; if i=M, the input of the other decoding layers comprises the output of the previous decoding layer and the output of one encoding layer; if i≠M, the input of the first decoding layer comprises the output of the last encoding layer, and the input of the other decoding layers comprises the output of the previous decoding layer and the output of one encoding layer; the scale of the features of the output of the encoding layer and the decoding layer connected with the input of the same decoding layer is the same.

[0103] ​​Taking i=M as an example, the input of the non-local network layer is the output of the last encoding layer, the input of the first decoding layer includes the output of the non-local network layer and the output of the last encoding layer, the input of the last decoding layer includes the output of the previous decoding layer and the normalized image, and the input of other decoding layers includes the input of the previous decoding layer and the output of one encoding layer. The feature scales of the encoding layer and decoding layer outputs connected to the input of the same decoding layer are the same.

[0104] Figure 5a In the example shown, the decoding module 402 includes a non-local network layer (NonLocalBlock) and 5 decoding layers; among them, decoding layer 1 is the first decoding layer, decoding layer 2 is the second decoding layer, decoding layer 3 is the third decoding layer, decoding layer 4 is the fourth decoding layer, and decoding layer 5 is the fifth decoding layer (that is, the last decoding layer).

[0105] The input of the non-local network layer is the output of the encoding layer 4. The scale of the features output by the non-local network layer is the same as the scale of the features output by the encoding layer 4. The scale of the features output by the decoding layer 1 is the same as the scale of the features output by the encoding layer 3. The scale of the features output by the decoding layer 2 is the same as the scale of the features output by the encoding layer 2. The scale of the features output by the decoding layer 3 is the same as the scale of the features output by the encoding layer 1. The scale of the features output by the decoding layer 4 is the same as the scale of the first image (normalized image).

[0106] The input of decoding layer 1 includes the features output by the non-local network layer and the features output by encoding layer 4. The input of decoding layer 2 includes the features output by decoding layer 1 and the features output by encoding layer 3. The input of decoding layer 3 includes the features output by decoding layer 2 and the features output by encoding layer 2. The input of decoding layer 4 includes the features output by decoding layer 3 and the features output by encoding layer 1. The input of decoding layer 5 includes the features output by decoding layer 4 and the normalized image.

[0107] Optionally, the last decoding layer (e.g. Figure 5a The decoding layer 5) in the example may include two convolutional layers with activation functions, which are used to perform two levels of convolution processing on the result of concatenating (or multiplying) the features output by the previous decoding layer and the normalized image. The output of the second convolutional layer is the output of the last decoding layer, that is, the output of decoding module 402. When the last decoding layer concatenates the features output by the previous decoding layer and the normalized image, it concatenates the features and the normalized image in the channel dimension. The scale of the concatenated features is the same as that of the features and the normalized image. That is, if the size of the features and the normalized image are both H1×W1, and the number of channels is C1 and C2, respectively, then the size of the features obtained by concatenating the features and the normalized image is still H1×W1, and the number of channels is C1+C2.

[0108] Non-last decoding layer (e.g. Figure 5a Decoding layers 1 through 4 in the

[15] network can include upsampling layers and convolutional layers. The upsampling layers can be deconvolutional layers or unpooling layers (maximum unpooling layers or mean unpooling layers). The upsampling layers upsample the result of concatenating (or multiplying) two input features; they can employ a factor of 2 upsampling. The convolutional layers convolute the output of the upsampling layers to produce features at a single scale.

[0109] When a non-last decoding layer concatenates two input features, it concatenates the two features in the channel dimension. The scale of the concatenated feature is the same as the scale of the two features. That is, if the size of the two features is H2×W2 and the number of channels is C3 and C4 respectively, the size of the concatenated feature is still H2×W2 and the number of channels is C3+C4.

[0110] The convolution in each decoding layer can be a convolution layer with an activation function, and the output of the convolution layer is the output of the decoding layer. Among them, the activation function can use common activation functions such as Swish, GELU, ReLU, Leaky ReLU, Softmax, tanh, etc.

[0111] As an example, the activation function in the last decoding layer is tanh, the activation function in other decoding layers is Leaky ReLU, and the activation function in the encoding layer is Leaky ReLU.

[0112] like Figure 5b As shown, it is another structural diagram of the network model provided in the embodiment of the present application. Figure 5a Compared with the example shown, the difference is that the connection relationship between the non-local network layer and the encoding layer and the decoding layer is changed. The specific implementation process inside the encoding layer and the decoding layer can be found in the above embodiment and will not be repeated here. Figure 5b In the example shown, both inputs of the decoding layer 1 are outputs of the encoding layer 4.

[0113] Figure 5a 、 5b In the illustrated example, the output of the non-local network layer is connected to only one decoding layer. In other optional embodiments, the output of the non-local network layer can be connected to multiple (i.e., at least two) decoding layers. Compared to connecting the output of the non-local network layer to only one decoding layer, connecting the output of the non-local network layer to multiple (i.e., at least two) decoding layers achieves better handwritten character removal results.

[0114] like Figure 5c As shown, it is another structural diagram of the network model provided in the embodiment of the present application. Figure 5aThe difference between the example shown and the example shown in FIG. 5c is that the number of decoding layers connected with the non-local network layer is increased. Figure 5c In the example shown, the non-local network layer is connected with two decoding layers (decoding layer 1 and decoding layer 3), the output of the non-local network layer is directly connected with the input of the decoding layer 1, and the output of the non-local network layer is connected with the input of the decoding layer 3 after being up-sampled (i.e., the output of the non-local network layer is indirectly connected with the input of the decoding layer 3). The up-sampling layer of the decoding layer 3 can be used to up-sample the result of splicing (in the channel dimension) the three features inputted; the up-sampling layer can use 2 times up-sampling; and the convolution layer is used to perform convolution processing on the output of the up-sampling layer to obtain a feature of one scale. Alternatively, the decoding layer 3 can first multiply the feature outputted by the encoding layer 2 with the feature outputted by the non-local network layer after being up-sampled (i.e., the output of the non-local network layer is indirectly connected with the input of the decoding layer 3), splice the multiplication result with the feature outputted by the encoding layer 2 in the channel dimension, up-sample the splicing result through the up-sampling layer, and perform convolution processing on the output of the up-sampling layer through the convolution layer to obtain a feature of one scale. Figure 5c In the example shown, the scale of the feature obtained by up-sampling the global feature outputted by the non-local network layer is the same as the scale of the feature outputted by the decoding layer 2 and the scale of the feature outputted by the encoding layer 2.

[0115] As shown in FIG. 5d, another structure of the network model provided by the embodiments of the present application is shown, which is similar to the structure shown in FIG. 5c. Figure 5b The difference between the example shown and the example shown in FIG. 5c is that the number of decoding layers connected with the non-local network layer is increased. Figure 5d In the example shown, the non-local network layer is connected with three decoding layers (decoding layer 1, decoding layer 2 and decoding layer 3), the output of the non-local network layer is directly connected with the input of the decoding layer 2, the output of the non-local network layer is connected with the input of the decoding layer 3 after being up-sampled (i.e., the output of the non-local network layer is indirectly connected with the input of the decoding layer 3), and the output of the non-local network layer is connected with the input of the decoding layer 1 after being down-sampled (i.e., the output of the non-local network layer is indirectly connected with the input of the decoding layer 1). The decoding process of the decoding layer 3 can refer to the decoding process of the decoding layer 3 shown in FIG. 5c, which will not be described herein again. The decoding process of the decoding layer 2 can refer to the decoding process of the decoding layer 2 shown in FIG. 5b, which will not be described herein again. Figure 5c In the example shown, the scale of the feature obtained by up-sampling the global feature outputted by the non-local network layer is the same as the scale of the feature outputted by the decoding layer 2 and the scale of the feature outputted by the encoding layer 2. Figure 5dIn the illustrated example, the feature scale of the global feature output by the non-local network layer after up-sampling (enlargement) is the same as the feature scale of the feature output by the decoding layer 2 and the feature scale of the feature output by the encoding layer 2; the feature scale of the global feature output by the non-local network layer after down-sampling (reduction) is the same as the feature scale of the feature output by the encoding layer 4.

[0116] In an optional embodiment, the network model can be trained by a target training data set, and a training sample in the target training data set is a picture (denoted as a sample picture), the sample picture at least includes a picture with a color background and handwritten content, and a label of the sample picture is a picture obtained by removing the handwritten content from the sample picture. When the sample picture has the color background, the label of the sample picture also has the color background.

[0117] Further, the sample picture can also include a picture without a color background but with handwritten content, and correspondingly, the label of the sample picture is a picture obtained by removing the handwritten content from the sample picture. Since the sample picture does not have the color background, the label of the sample picture also does not have the color background.

[0118] Compared with the existing handwritten content removal scheme, the scheme of the present application can be implemented by using a simpler network structure, has a smaller parameter quantity, and is more suitable for use on an embedded device (such as a learning machine, etc.). The parameter quantity can be reduced to 1.2M, the floating point operation quantity can be reduced to 74GFlops, and the time consumption on a small learning machine (similar to the size of a mobile phone) is expected to be only 74ms, the speed is faster, and the experience is better.

[0119] The process of training the network model can include:

[0120] The normalized picture of the sample picture in the target training data set is taken as the input of the network model, and the peak signal-to-noise ratio (PSNR) of the picture output by the network model and the label of the sample picture becomes larger, and the structural similarity index (SSIM) of the picture output by the network model and the label of the sample picture becomes larger, as the target to update the parameters of the network model.

[0121] That is, the peak signal-to-noise ratio of the picture output by the network model based on the parameter update and the label of the sample picture after the handwritten content removal of the sample image is greater than the peak signal-to-noise ratio of the picture output by the network model based on the parameter update before the handwritten content removal of the sample image; the structural similarity index of the picture output by the network model based on the parameter update and the label of the sample picture after the handwritten content removal of the sample image is greater than the structural similarity index of the picture output by the network model based on the parameter update before the handwritten content removal of the sample image.

[0122] Optionally, the loss value can be calculated based on the peak signal-to-noise ratio and the structural similarity of the picture output by the network model and the label of the sample picture, and the parameters of the network model are updated with the loss value being smaller and smaller as the goal. That is, the loss value is negatively correlated with the peak signal-to-noise ratio, and the loss value is negatively correlated with the structural similarity.

[0123] In an optional embodiment, the implementation of updating the parameters of the network model with the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture being larger and the structural similarity of the picture output by the network model and the label of the sample picture being larger as the goal can be:

[0124] The network model is updated for N (N is a positive integer) rounds of iterations, and the parameters of the network model are updated with the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture being larger and the structural similarity of the picture output by the network model and the label of the sample picture being larger as the goal in each round of iteration, that is, the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture and the structural similarity of the picture output by the network model and the label of the sample picture are considered in each round of parameter update.

[0125] In a possible implementation, the network model can be trained in stages, and different losses are used in different training stages, which can accelerate the convergence speed of the model.

[0126] In an optional embodiment, the implementation of updating the parameters of the network model with the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture being larger and the structural similarity of the picture output by the network model and the label of the sample picture being larger as the goal can be:

[0127] When the network model is updated for a preset number of times (for the sake of description, recorded as N1 times), the parameters of the network model are updated with the structural similarity of the picture output by the network model and the label of the sample picture being larger as the goal.

[0128] After the network model is updated for a preset number of times, the parameters of the network model are updated with the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture being larger as the goal.

[0129] That is, in the case of updating the network model for N rounds of iterations, the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture is not considered in each round of parameter update in the first N1 rounds of iteration; in the last N-N1 rounds of iteration, the structural similarity of the picture output by the network model and the label of the sample picture is not considered in each round of parameter update.

[0130] In an optional embodiment, the implementation manner of updating the parameters of the network model in the manner of increasing the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture and increasing the structural similarity between the picture output by the network model and the label of the sample picture can be as follows:

[0131] When the network model is updated for a preset number of times (for ease of description, denoted as N1 times), the parameters of the network model are updated in the manner of increasing the structural similarity between the picture output by the network model and the label of the sample picture; N1 is less than N.

[0132] After the network model is updated for the preset number of times, the parameters of the network model are updated in the manner of increasing the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture and increasing the structural similarity between the picture output by the network model and the label of the sample picture.

[0133] That is, in the case of updating the network model for N rounds of iterative updates, in the process of the first N1 rounds of iterative updates, the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture is not considered in each round of parameter update; in the process of the last N-N1 rounds of iterative updates, the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture and the structural similarity between the picture output by the network model and the label of the sample picture are considered in each round of parameter update.

[0134] In the above two training methods, the structural similarity between the picture output by the network model and the label of the sample picture is considered first, and then the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture is considered.

[0135] In an optional embodiment, the implementation manner of updating the parameters of the network model in the manner of increasing the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture and increasing the structural similarity between the picture output by the network model and the label of the sample picture can be as follows:

[0136] When the network model is updated for a preset number of times (for ease of description, denoted as N1 times), the parameters of the network model are updated in the manner of increasing the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture;

[0137] After the network model is updated for the preset number of times, the parameters of the network model are updated in the manner of increasing the structural similarity between the picture output by the network model and the label of the sample picture.

[0138] That is to say, when the network model is iterated and updated for N rounds, in the first N1 rounds of iterative updates, the parameter updates in each round do not consider the structural similarity between the labels of the images output by the network model and the sample images; in the subsequent N-N1 rounds of iterative updates, the parameter updates in each round do not consider the peak signal-to-noise ratio between the labels of the images output by the network model and the sample images.

[0139] Specifically, in the first N1 rounds, the network model parameters are updated with the goal of increasing the peak signal-to-noise ratio between the image output by the network model and the label of the sample image. In the next N-N1 rounds, the network model parameters are updated with the goal of increasing the structural similarity between the image output by the network model and the label of the sample image.

[0140] In an optional embodiment, another implementation method of updating the parameters of the network model with the goal of increasing the peak signal-to-noise ratio of the image output by the network model and the label of the sample image, and increasing the structural similarity between the image output by the network model and the label of the sample image can be:

[0141] When updating the network model for a preset number of times (N1 times), the parameters of the network model are updated with the goal of increasing the peak signal-to-noise ratio of the labels of the images output by the network model and the sample images;

[0142] After the network model is updated a preset number of times, the parameters of the network model are updated with the goal of increasing the peak signal-to-noise ratio of the image output by the network model and the label of the sample image, and increasing the structural similarity between the image output by the network model and the label of the sample image.

[0143] That is to say, when the network model is iteratively updated for N rounds, in the first N1 rounds of iterative updates, the parameter updates in each round do not consider the structural similarity between the labels of the image output by the network model and the sample images; in the subsequent N-N1 rounds of iterative updates, the parameter updates in each round consider the peak signal-to-noise ratio of the labels of the image output by the network model and the sample images, as well as the structural similarity between the labels of the image output by the network model and the sample images.

[0144] In the above two staged training methods, the peak signal-to-noise ratio of the image output by the network model and the label of the sample image is considered first, and then the structural similarity of the image output by the network model and the label of the sample image is considered.

[0145] Among the four staged training methods mentioned above, the model converges faster when the peak signal-to-noise ratio is considered first and then the structural similarity, compared with the method of first considering the structural similarity and then considering the peak signal-to-noise ratio.

[0146] The training round of the network model of the present application is also relatively small, and usually several hundred times are enough, that is, N is less than 1000. As an example, N1 can be 100.

[0147] Corresponding to the method embodiment, the present application also provides a handwritten content removing device. A structural schematic diagram of the handwritten content removing device provided by the present application embodiment is as shown in the figure, which can include: Figure 6

[0148] The feature extraction unit 601 and the decoding unit 602;

[0149] The feature extraction unit 601 is configured to perform multi-scale feature extraction on the first picture containing handwritten content to obtain features of multiple scales.

[0150] The decoding unit 602 is configured to perform non-local feature extraction on the features of any scale to obtain global features; and perform decoding of multiple levels based on the global features, the features of multiple scales and the first picture to obtain a second picture; the second picture is the result of handwritten content erasing on the first picture.

[0151] The handwritten content removing device provided by the present application embodiment performs multi-scale feature extraction on the first picture containing handwritten content to obtain features of multiple scales; performs non-local feature extraction on the features of any scale to obtain global features; and performs decoding of multiple levels based on the global features, the features of multiple scales and the first picture to obtain a second picture; the second picture is the result of handwritten content erasing on the first picture. Based on the present application, the handwritten content in the picture with a color background can be quickly and accurately removed, and a scheme for effectively removing handwritten content in a picture scene with a color background is provided.

[0152] The detailed functions and extended functions of the feature extraction unit 601 and the decoding unit 602 can refer to the foregoing method embodiments, which will not be described here.

[0153] The above-mentioned various units in the handwritten content removing device can be all or partially realized by software, hardware and combinations thereof. The above-mentioned various units can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above-mentioned various units.

[0154] Corresponding to the method embodiment, the present application also provides a learning machine, Figure 7 As shown in the figure, a structural schematic diagram of the learning machine provided by the present application embodiment can include:

[0155] The image acquisition module 701 and the processor 702;

[0156] The image acquisition module is configured to acquire pictures;​

[0157] The processor 702 is configured to process the picture collected by the image collection module 701 by using the handwriting content removal method as described above, so as to remove the handwriting content in the picture.

[0158] The learning machine provided in the present application can quickly and accurately remove the handwriting content in the picture with a color background, and improve the removal effect of the handwriting content in the picture with a color background.

[0159] Further, the learning machine can further include a display module configured to display the picture after the handwriting content is removed.

[0160] The present application also provides an electronic device. As shown in Figure 8 The electronic device in the present application can be a terminal device (such as a mobile phone, a tablet computer, a notebook computer, a learning machine, etc.), or a server (which can be a single server, a server cluster, or a cloud server, etc.). Figure 8 The electronic device shown is only an example, and should not impose any limitation on the functions and use range of the present application.

[0161] As shown in Figure 8 The electronic device can include a processing device (such as a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 802 or loaded from a storage device 808 to a random access memory (RAM) 803. In the state that the electronic device is powered on, the RAM 803 also stores various programs and data required for the operation of the electronic device. The processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0162] Generally, the following devices can be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a memory card, a hard disk, etc.; and communication devices 809. The communication devices 809 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device with various devices is shown, but it should be understood that all the shown devices are not required to be implemented or possessed. More or fewer devices can be alternatively implemented or possessed.

[0163] The embodiment of the present application further provides a computer program product comprising computer readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the handwriting content removing methods provided by the embodiments of the present application.

[0164] The embodiment of the present application further provides a computer readable storage medium, which carries one or more computer programs, which, when executed on an electronic device, can cause the electronic device to implement any of the handwriting content removing methods provided by the embodiments of the present application.

[0165] It should be noted that the apparatus embodiments described above are merely illustrative, and the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiments according to actual needs. In addition, in the apparatus embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0166] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure for implementing the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better embodiment. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including a plurality of instructions to make a computer device (which can be a personal computer, training device, or network device, etc.) execute the methods described in each embodiment of the present application.

[0167] In the above embodiments, all or part can be realized by software, hardware, firmware or any combination thereof. When realized by software, it can be realized in the form of a computer program product in whole or in part. Professional technicians can use different methods to implement the described functions for each specific solution, but such implementation should not be considered beyond the scope of the present application.

[0168] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, training device or data center to another website, computer, training device or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be stored by the computer or a data storage device such as a training device, a data center, etc. integrated with one or more available media sets. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0169] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0170] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A handwriting content removing method characterized by comprising: The method comprises the following steps: performing multi-scale feature extraction on a first picture containing handwritten content to obtain features of multiple scales; performing non-local feature extraction on features of any scale to obtain global features; performing multi-level decoding based on the global features, the features of multiple scales and the first picture to obtain a second picture; the second picture is the result of handwritten content erasure on the first picture.

2. The method of claim 1, wherein, The multi-scale feature extraction on the first picture containing handwritten content comprises: performing normalization processing on the first picture to obtain a normalized picture with a data distribution interval of [-1, 1]; performing multi-scale feature extraction on the normalized picture to obtain the features of multiple scales. Correspondingly, the process of obtaining the second picture comprises: performing multi-level decoding based on the global features, the features of multiple scales and the normalized picture.

3. The method of claim 2, wherein, The handwritten content removal method is implemented based on a pre-trained network model, and the network model comprises: a multi-feature extraction module, wherein the input of the multi-feature extraction module is the normalized picture, and the output of the multi-feature extraction module is the features of multiple scales; a decoding module, wherein the input of the decoding module is the features of multiple scales and the normalized picture, and the output of the decoding module is the second picture.

4. The method of claim 3, wherein, The multi-feature extraction module comprises: M encoding layers, wherein the input of the first encoding layer is the normalized picture, and the input of each non-first encoding layer is the output of the previous encoding layer; different encoding layers output features of different scales, and the scale of the features output by each non-first encoding layer is smaller than the scale of the features output by the previous encoding layer; the decoding module comprises: a non-local network layer and M+1 decoding layers connected with the non-local network layer, wherein the input of the non-local network layer is the output of the i-th encoding layer, the output of the non-local network layer is directly or indirectly connected with the input of at least one decoding layer, and the at least one decoding layer comprises the M-i+1 decoding layer; the input of the first decoding layer of the decoding module comprises the output of the last encoding layer, the input of the last decoding layer of the decoding module comprises the output of the previous decoding layer and the normalized picture; if i=M, the input of other decoding layers comprises the output of the previous decoding layer and the output of one encoding layer; if i≠M, the input of the M-i+1 decoding layer further comprises the output of the previous decoding layer, and the input of other decoding layers comprises the output of the previous decoding layer and the output of one encoding layer; the encoding layer and the decoding layer connected with the same input output features of the same scale.

5. The method according to claim 3 or 4, characterized in that, The network model is trained in the following manner: taking the normalized picture of a sample picture in a target training data set as the input of the network model, updating the parameters of the network model so that the peak signal-to-noise ratio of the picture output by the network model and the label of the sample picture becomes larger, and the structural similarity between the picture output by the network model and the label of the sample picture becomes larger; the sample pictures in the target training data set at least comprise pictures with color backgrounds and handwritten content. The label of the sample picture is a picture obtained by removing the handwritten content from the sample picture.

6. The method of claim 5, wherein, The updating of the parameters of the network model is performed according to the following two conditions: the peak signal-to-noise ratio (PSNR) of the picture output by the network model and the label of the sample picture is increased, and the structural similarity (SSIM) of the picture output by the network model and the label of the sample picture is increased. The updating of the parameters of the network model is performed according to the following two conditions: the peak signal-to-noise ratio (PSNR) of the picture output by the network model and the label of the sample picture is increased, and the structural similarity (SSIM) of the picture output by the network model and the label of the sample picture is increased. The updating of the parameters of the network model is performed according to the following two conditions: the peak signal-to-noise ratio (PSNR) of the picture output by the network model and the label of the sample picture is increased, and the structural similarity (SSIM) of the picture output by the network model and the label of the sample picture is increased.

7. The method of claim 5, wherein, The updating of the parameters of the network model is performed according to the following two conditions: the peak signal-to-noise ratio (PSNR) of the picture output by the network model and the label of the sample picture is increased, and the structural similarity (SSIM) of the picture output by the network model and the label of the sample picture is increased. The updating of the parameters of the network model is performed according to the following two conditions: the peak signal-to-noise ratio (PSNR) of the picture output by the network model and the label of the sample picture is increased, and the structural similarity (SSIM) of the picture output by the network model and the label of the sample picture is increased. The updating of the parameters of the network model is performed according to the following two conditions: the peak signal-to-noise ratio (PSNR) of the picture output by the network model and the label of the sample picture is increased, and the structural similarity (SSIM) of the picture output by the network model and the label of the sample picture is increased.

8. A learning machine characterized by The image acquisition module is configured to acquire the picture. The processor is configured to process the picture acquired by the image acquisition module by using the handwritten content removal method according to any one of claims 1 to 7, so as to remove the handwritten content in the picture. The computer readable instructions, when executed on the electronic device, cause the electronic device to implement the handwritten content removal method according to any one of claims 1 to 7.

9. A computer program product, characterised in that, The electronic device includes at least one processor and a memory connected to the processor, wherein:

10. An electronic device, comprising: The memory is configured to store a computer program. The processor is configured to execute the computer program, so that the electronic device can implement the handwritten content removal method according to any one of claims 1 to 7. The storage medium carries one or more computer programs, which, when executed by the electronic device, can enable the electronic device to implement the handwritten content removal method according to any one of claims 1 to 7.

11. A computer storage medium, characterized in that ​

Images