Model training method, anti-counterfeiting mark removing method, electronic equipment and storage medium
By adding anti-counterfeiting marks to document image samples and using U-shaped neural network to remove other things, the problem of anti-counterfeiting marks interfering with optical character recognition is solved, which improves recognition accuracy and reduces model training costs.
Patent Information
- Application Number
- CN202311126178.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-01
- Publication Date
- 2024-12-20
AI Technical Summary
When removing anti-counterfeiting marks in document images, the prior art can easily interfere with optical character recognition, resulting in a decrease in recognition accuracy and a high cost of model training.
Add anti-counterfeiting identifiers to document image samples that do not contain anti-counterfeiting identifiers to form a third document image sample, and train them through two U-shaped neural networks in series to remove anti-counterfeiting identifier networks, using the average absolute error loss function and structural similarity index Loss function to remove anti-counterfeiting marks.
It improves the recognition accuracy of anti-counterfeiting labels, reduces the cost of model training, and reduces the redundancy of calculation amount.
Smart Images
Figure CN119168916A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of image processing, and in particular to a model training, anti-counterfeiting mark removal method, electronic equipment and storage medium. Background Art
[0002] Seals and watermarks are common anti-counterfeiting measures used in document images. They add some graphics or text to the document as anti-counterfeiting marks to indicate the authenticity of the document. However, these anti-counterfeiting marks may interfere with optical character recognition. For example, seals and watermarks may block the text content, causing some text to be unrecognizable; the graphics and text of seals and watermarks are often complex, and they may be mistakenly recognized as text, resulting in reduced recognition accuracy. Summary of the invention
[0003] The purpose of the embodiments of the present invention is to provide a model training, anti-counterfeiting mark removal method, electronic device and storage medium, which can enable the trained model to accurately remove anti-counterfeiting marks in document images while reducing the training cost of the model.
[0004] In order to solve the above technical problems, an embodiment of the present invention provides a model training method, comprising:
[0005] adding an anti-counterfeiting mark to the second document image sample not including the anti-counterfeiting mark to form a third document image sample;
[0006] The third document image sample is used as input to train a security mark removal network composed of two U-shaped neural networks connected in series, and the output of the security mark removal network is a target image after the security mark is removed from the third document image sample;
[0007] The loss function used in training the anti-counterfeiting mark removal network includes: a mean absolute error loss function constructed based on the second document image sample and the target image, and / or a structural similarity index loss function.
[0008] An embodiment of the present invention also provides a method for removing anti-counterfeiting marks, comprising: inputting an image to be processed containing an anti-counterfeiting mark into an anti-counterfeiting mark removal network trained by the above-mentioned model training method to obtain a target image corresponding to the image to be processed.
[0009] An embodiment of the present invention further provides an electronic device, including:
[0010] at least one processor; and,
[0011] a memory communicatively connected to the at least one processor; wherein,
[0012] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the model training method as described above, or the anti-counterfeiting mark removal method as described above.
[0013] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the model training method described above, or the anti-counterfeiting mark removal method described above.
[0014] Compared with the prior art, the embodiment of the present invention adds an anti-counterfeiting mark to a second document image sample that does not contain an anti-counterfeiting mark to form a third document image sample; the third document image sample is used as input to train a network for removing anti-counterfeiting marks, which is composed of two U-shaped neural networks connected in series, and the output of the network for removing anti-counterfeiting marks is a target image after the anti-counterfeiting mark is removed from the third document image sample; wherein the loss function used to train the network for removing anti-counterfeiting marks includes: a mean absolute error loss function constructed based on the second document image sample and the target image, and / or a structural similarity index loss function. This solution constructs a network for removing anti-counterfeiting marks by connecting two U-shaped neural networks in series, which can improve the accuracy of the network for removing anti-counterfeiting marks and removing anti-counterfeiting marks, while also reducing the redundancy of calculations and the cost of model training. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0016] Figure 1 The specific process of the model training method according to the embodiment of the present invention is Figure 1 ;
[0017] Figure 2 The specific process of the model training method according to the embodiment of the present invention is Figure 2 ;
[0018] Figure 3 is an architecture diagram of a network for removing anti-counterfeiting marks according to an embodiment of the present invention;
[0019] Figure 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0020] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the following will be described in detail with reference to the accompanying drawings. However, it will be appreciated by those skilled in the art that in the various embodiments of the present invention, many technical details are provided in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can be implemented.
[0021] One embodiment of the present invention relates to a model training method. Figure 1 As shown, the model training method provided in this embodiment includes the following steps.
[0022] Step 101: adding an anti-counterfeiting mark to a second document image sample that does not contain an anti-counterfeiting mark to form a third document image sample.
[0023] The second document image sample is a sample image without an anti-counterfeiting mark, such as a photo, a PDF scan file, etc. The anti-counterfeiting mark can be a seal, a watermark, etc. The anti-counterfeiting mark is added to the second document image sample to form a third document image sample. The third document image sample is the second document image sample containing the anti-counterfeiting mark.
[0024] Step 102: The third document image sample is used as input to train the security mark removal network composed of two U-shaped neural networks connected in series. The output of the security mark removal network is the target image after the security mark of the third document image sample is removed.
[0025] The loss function used in training the anti-counterfeiting mark removal network includes: a mean absolute error loss function constructed based on the second document image sample and the target image, and / or a structural similarity index loss function.
[0026] Specifically, in the two U-shaped neural networks connected in series to form the anti-counterfeiting mark removal network, the first U-shaped neural network can be called a rough removal stage network, and the second U-shaped neural network can be called a fine removal stage network. The third document image sample obtained through the above step 101 is input into the anti-counterfeiting mark removal network, and after the rough removal stage of the first U-shaped neural network, an intermediate result image with the anti-counterfeiting mark removed is obtained, and then the intermediate result image is passed through the fine removal stage of the second U-shaped neural network to obtain a target image with the anti-counterfeiting mark removed. That is, the anti-counterfeiting mark removal network is trained with the third document image sample as input, and the anti-counterfeiting mark removal network uses the target image with the anti-counterfeiting mark removed from the third document image sample as output.
[0027] Since the third document image sample is obtained by adding an anti-counterfeiting mark to the second document image sample, and the third document image sample is used to remove the anti-counterfeiting mark through the anti-counterfeiting mark removal network to obtain the target image, the higher the similarity between the target image and the second document image sample, the better the effect of the anti-counterfeiting mark removal network in removing the anti-counterfeiting mark. Therefore, the loss function used in the process of training the anti-counterfeiting mark removal network is the mean absolute error loss function constructed based on the second document image sample and the target image, and / or the structural similarity index loss function.
[0028] Compared with the related art, this embodiment adds an anti-counterfeiting mark to the second document image sample that does not contain the anti-counterfeiting mark to form a third document image sample; the third document image sample is used as input to train the anti-counterfeiting mark removal network composed of two U-shaped neural networks in series, and the output of the anti-counterfeiting mark removal network is the target image after the anti-counterfeiting mark is removed from the third document image sample; wherein the loss function used to train the anti-counterfeiting mark removal network includes: the mean absolute error loss function constructed based on the second document image sample and the target image, and / or the structural similarity index loss function. This solution constructs a anti-counterfeiting mark removal network by connecting two U-shaped neural networks in series, which can improve the recognition and removal accuracy of the anti-counterfeiting mark removal network, while also reducing the redundancy of the amount of calculation and the cost of model training.
[0029] Another embodiment of the present invention relates to a model training method, such as Figure 2 As shown, this embodiment is a refinement of step 101 of the above-mentioned embodiment.
[0030] like Figure 2 As shown, step 101 may include the following steps:
[0031] Step 1011: extracting a pixel-level instance region of the anti-counterfeiting mark from a first document image sample containing the anti-counterfeiting mark.
[0032] The first document image sample is a sample image containing an anti-counterfeiting mark, and the anti-counterfeiting mark instance area at the pixel level is extracted from the first document image sample. The extraction of the anti-counterfeiting mark instance area at the pixel level can use optical character recognition technology, and this application does not make specific restrictions on this. The anti-counterfeiting mark instance area at the pixel level can be a seal instance area or a watermark instance area at the pixel level.
[0033] Step 1012: Fusing the pixel-level instance region with the second document image sample to obtain a third document image sample.
[0034] Specifically, a pixel-level instance region of the anti-counterfeiting mark is extracted from the first document image sample, and the extracted pixel-level instance region is fused with the second document image sample that does not contain the anti-counterfeiting mark to obtain a third document image sample that is different from the first document image sample and the second document image sample.
[0035] Accordingly, in the specific implementation process, the present application does not limit the specific fusion process of fusing the pixel-level instance area with the second document image sample to obtain the third document image sample. For example, the image of the corresponding fusion position in the pixel-level instance area and the image of the corresponding fusion position of the second document image sample can be selected, and the pixels of the fusion position image can be directly subjected to the maximum value method, weighted average method, or median algorithm to obtain the fused image of the area; or, a model-based fusion method such as sparse representation SR, pulse coupled neural network PCNN, and a deep learning model based on convolutional neural network CNN can be used to achieve image fusion. The present application does not limit the specific fusion method of fusing the pixel-level instance area with the second document image sample to obtain the third document image sample. The fusion algorithm involved in the following embodiments is only an example for a better understanding of the present solution and does not constitute any limitation.
[0036] In one example, a fusion algorithm for fusion of the pixel-level instance region with the second document image sample to obtain a third document image sample may be: P =a P W P +(1-a P )Y P ;
[0037] Where P is the fusion position where the pixel-level instance area is fused with the second document image sample, and F P is the image corresponding to the fusion position in the third document image sample, W P is the image of the corresponding fusion position in the pixel-level instance area, Y P is the image corresponding to the fusion position in the second document image sample, a P is the weight coefficient and is a positive number less than 1.
[0038] Specifically, P = (i, j), where P is specifically represented by the position coordinates of the fusion position where the pixel-level instance region is fused with the second document image sample, and F P That is, the pixel value of the image corresponding to the fusion position in the third document image sample, Y Pis the pixel value of the image at the corresponding fusion position in the second document image sample. That is, the image at the fusion position corresponding to the second document image sample and the third document image sample is specifically represented by the pixel value of each pixel position. When the pixel-level instance region and the second document image sample are fused by the fusion algorithm, the pixels at the corresponding fusion position of the pixel-level instance region and the pixels at the corresponding fusion position of the second document image sample are fused one by one.
[0039] Compared with the related art, this embodiment extracts the anti-counterfeiting mark instance area at the pixel level from the first document image sample, and fuses the pixel-level instance area with the second document image sample through a fusion algorithm to obtain a third document image sample. The third text image is fused one by one by the pixels at the fusion position corresponding to the pixel-level instance area and the pixels at the fusion position corresponding to the second document image sample. The third text image obtained by the above method of this embodiment has a better fusion effect, which can not only obtain a clearer and more accurate third text image, but also can conveniently obtain a large number of model training samples, thereby improving the quality of samples for training the anti-counterfeiting mark removal network.
[0040] Another embodiment of the present invention relates to a model training method, which is a supplement to step 102 of the aforementioned embodiment: taking a third document image sample as input to train a security mark removal network composed of two U-shaped neural networks in series, wherein the output of the security mark removal network is a target image of the third document image sample after the security mark is removed.
[0041] In an example, the network architectures of two U-type neural networks are the same, and the search path in each U-type neural network is composed of several convolutional layers and residual layers that are interlaced in sequence, and the first layer is a convolutional layer and the last layer is a residual layer; the expansion path in each U-type neural network is composed of several transposed convolutional layers and residual layers that are interlaced in sequence, and the first layer is a residual layer and the last layer is a transposed convolutional layer; the output feature data of the convolutional layer at the same depth layer in each U-type neural network is superimposed with the output feature data of the transposed convolutional layer as the input feature data of the next residual layer of the transposed convolutional layer.
[0042] Specifically, Figure 3 As shown, in the two U-shaped neural networks connected in series to form a network for removing anti-counterfeiting marks, the first U-shaped neural network can be called a rough removal phase network, and the second U-shaped neural network can be called a fine removal phase network. The network architecture of each U-shaped neural network (U-Net) has a contracting path and an expansive path, and the contracting path and the expansive path are adjacently arranged.
[0043] like Figure 3As shown, the sample image with a seal or the sample image with a watermark is used as the input data of the anti-counterfeiting mark removal network. It should be noted that the third document image sample in the above embodiment is Figure 3 In the specific example, the sample image with a seal or the sample image with a watermark is used as input data and passes through the search path of the first U-type neural network. The last layer (residual layer) of the search path is output to the first layer (residual layer) of the extended path adjacent to the search path. After the extended path, the intermediate result image with the anti-counterfeiting mark roughly removed is obtained and the result is used as input to enter the second U-type neural network structure. The intermediate result image with the anti-counterfeiting mark roughly removed is input to the first layer (convolutional layer) of the search path of the second U-type neural network structure. After the search path, the last layer (residual layer) of the search path is output to the first layer (residual layer) of the extended path adjacent to the search path. After the extended path, the sample image after the seal is removed or the sample image after the watermark is removed, which is output by the final anti-counterfeiting mark removal network. It should be noted that in this embodiment, Figure 3 The sample image after the seal is removed or the sample image after the watermark is removed is the target image in the above embodiment.
[0044] Specifically, the loss function used in training the anti-counterfeiting mark removal network includes: a mean absolute error loss function constructed based on the second document image sample and the target image, and / or a structural similarity index loss function.
[0045] In one example, the mean absolute error loss function can be specifically expressed as the following function: L1 = ∑|F P -Y P |;
[0046] Where L1 is the mean absolute error loss value, P is the fusion position where the pixel-level instance area is fused with the second document image sample, and F P is the image corresponding to the fusion position in the third document image sample, Y P is the image corresponding to the fusion position in the second document image sample.
[0047] Specifically, the image corresponding to the fusion position in the third document image sample and the image corresponding to the fusion position in the second document image sample are specifically represented as the pixel values of the corresponding images at each position. When the pixel-level instance region and the second document image sample are fused by the fusion algorithm, the pixels at the fusion position corresponding to the pixel-level instance region and the pixels at the fusion position corresponding to the second document image sample are fused one by one. P = (i, j), P is specifically represented as the position coordinates of the fusion position where the pixel-level instance region and the second document image sample are fused, and F P That is, the pixel value of the image corresponding to the fusion position in the third document image sample, YP is the pixel value of the image corresponding to the fusion position in the second document image sample.
[0048] In an example, the structural similarity index loss function can be specifically expressed as the following function:
[0049] Among them, L2 is the structural similarity index loss value, μ f is the mean of the target image, μ g is the mean of the second document image sample, σ f is the variance of the target image, σ g is the variance of the second document image sample, σ fg is the covariance of the target image and the second document image samples, and C1 and C2 are constants.
[0050] Specifically, the structural similarity index loss function is constructed based on the second document image sample and the target image. The higher the similarity between the second document image sample and the target image, the better the effect of removing the anti-counterfeiting mark network is in removing the anti-counterfeiting mark.
[0051] Compared with the related art, this embodiment constructs a mean absolute error loss function based on the second document image sample and the target image, and uses the mean absolute error loss function and / or the structural similarity index loss function as the loss function to train the anti-counterfeiting mark removal network. The anti-counterfeiting mark removal network obtained by training with the loss function of this embodiment can effectively improve the recognition accuracy of anti-counterfeiting marks, reduce the redundancy of calculation, and improve the accuracy of anti-counterfeiting mark removal.
[0052] Another embodiment of the present invention relates to a method for removing anti-counterfeiting marks, specifically: inputting an image to be processed containing an anti-counterfeiting mark into an anti-counterfeiting mark removal network trained by a model training method as described in any of the above embodiments to obtain a target image corresponding to the image to be processed.
[0053] Among them, the specific training process and related implementation details of the model training method have been recorded in the above implementation method, and this application will not repeat them here.
[0054] Compared with the related art, the anti-counterfeiting mark removal method in the embodiment of the present invention can effectively remove anti-counterfeiting marks such as watermarks and seals in documents, and reduce the impact of anti-counterfeiting marks on optical character recognition.
[0055] Another embodiment of the present invention relates to an electronic device, such as Figure 4As shown, it includes at least one processor 202; and a memory 201 that is communicatively connected to the at least one processor 202; wherein the memory 201 stores instructions that can be executed by the at least one processor 202, and the instructions are executed by the at least one processor 202 so that the at least one processor 202 can execute any of the above method embodiments.
[0056] Among them, the memory 201 and the processor 202 are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects one or more processors 202 and various circuits of the memory 201 together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art, and therefore, are not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor 202 is transmitted on a wireless medium through an antenna, and further, the antenna also receives data and transmits the data to the processor 202.
[0057] The processor 202 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management and other control functions. The memory 201 can be used to store data used by the processor 202 when performing operations.
[0058] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.
[0059] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0060] Those skilled in the art will appreciate that the above-mentioned embodiments are specific examples for implementing the present invention, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.
Claims
1. A model training method, characterized in that: include: adding an anti-counterfeiting mark to the second document image sample not including the anti-counterfeiting mark to form a third document image sample; The third document image sample is used as input to train a security mark removal network composed of two U-shaped neural networks connected in series, and the output of the security mark removal network is a target image after the security mark is removed from the third document image sample; The loss function used in training the anti-counterfeiting mark removal network includes: a mean absolute error loss function constructed based on the second document image sample and the target image, and / or a structural similarity index loss function.
2. The method according to claim 1, characterized in that: The step of adding an anti-counterfeiting mark to a second document image sample that does not contain an anti-counterfeiting mark to form a third document image sample includes: Extracting a pixel-level instance region of the anti-counterfeiting mark from a first document image sample containing the anti-counterfeiting mark; The pixel-level instance region is fused with the second document image sample to obtain the third document image sample.
3. The method according to claim 2, characterized in that The step of fusing the pixel-level instance region with the second document image sample to obtain the third document image sample comprises: The third document image sample is obtained by using the following fusion algorithm: F P =a P W P +(1-a P )Y P Wherein, P is the fusion position where the pixel-level instance area is fused with the second document image sample, and F P is the image corresponding to the fusion position in the third document image sample, W P is the image corresponding to the fusion position in the pixel-level instance area, Y P is the image corresponding to the fusion position in the second document image sample, a P A positive number less than 1.
4. The method according to claim 1, characterized in that: The network architectures of the two U-shaped neural networks are the same, and the search path in each U-shaped neural network is formed by a plurality of convolutional layers and residual layers interlaced in series, with the first layer being a convolutional layer and the last layer being a residual layer; the expansion path in each U-shaped neural network is formed by a plurality of transposed convolutional layers and residual layers interlaced in series, with the first layer being a residual layer and the last layer being a transposed convolutional layer; The feature data obtained by superimposing the output feature data of the convolution layer at the same depth layer in each U-shaped neural network and the output feature data of the transposed convolution layer serves as the input feature data of the next residual layer of the transposed convolution layer.
5. The method according to claim 1, characterized in that The mean absolute error loss function is specifically expressed as follows: L1=∑|F P -AND P | Wherein, L1 is the mean absolute error loss value, P is the fusion position where the pixel-level instance area is fused with the second document image sample, and F P is the image corresponding to the fusion position in the third document image sample, Y P is the image corresponding to the fusion position in the second document image sample.
6. The method according to claim 1, characterized in that The structural similarity index loss function is specifically expressed as follows: Among them, L2 is the structural similarity index loss value, μ f is the mean of the target image, μ g is the mean of the second document image samples, σ f is the variance of the target image, σ g is the variance of the second document image sample, σ fg is the covariance of the target image and the second document image samples, and C1 and C2 are constants respectively.
7. The method according to any one of claims 1 to 6, characterized in that: The anti-counterfeiting mark includes a seal and / or a watermark.
8. A method for removing an anti-counterfeiting mark, characterized in that: include: An image to be processed containing an anti-counterfeiting mark is input into an anti-counterfeiting mark removal network trained by the model training method described in any one of claims 1 to 6 to obtain a target image corresponding to the image to be processed.
9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the model training method as described in any one of claims 1 to 7, or the anti-counterfeiting mark removal method as described in claim 8.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the model training method described in any one of claims 1 to 7, or the anti-counterfeiting mark removal method described in claim 8.