Document image detection method, system and device

Abstract

The application discloses a document image detection method, comprising the following steps: step S1, obtaining a to-be-detected document image; step S2, performing segmentation processing on the to-be-detected document image to obtain a background area and a to-be-detected area of the to-be-detected document image; step S3, performing sample collection on the background area by using a rectangular frame with a preset size to obtain a plurality of background area samples; step S4, performing data enhancement on each background area sample to respectively construct positive samples and negative samples of a character area, a seal area or a character and seal area; step S5, taking the obtained positive samples and negative samples as training data to train a document image forgery detection model to obtain a trained document image forgery detection model; and step S6, inputting each to-be-detected area into the trained document image forgery detection model to perform detection to determine whether each to-be-detected area is tampered with, and marking the to-be-detected area determined to be tampered with. The application can detect various tampering modes and has high accuracy.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a method, system and device for detecting document images. Background Technology

[0002] With the increasing application of e-commerce and e-government, the demand for online document authentication is also growing. For example, users need to upload photos or scans of documents such as business licenses, contracts, certificates, transcripts, etc., to various websites for authentication. Document images are usually formed by converting paper documents into image formats in some way. The content of document images can be tampered with using some image editing tools. Therefore, the first step in online authentication is to detect whether there is any tampering in the document images.

[0003] Currently, image detection methods used for tampering detection can be mainly divided into the following three categories: one is based on tampering trace detection, such as processing and detecting the edges of objects in the image, using filters to process the image and highlight the edge features, thereby determining the edge differences between the real area and the tampered area; another is based on the consistency of the inherent properties of the imaging device, such as the unique pattern noise that a camera usually adds during the image shooting process, and by analyzing the areas in the image where the noise pattern changes, the tampered area in the image can be determined; and the third is based on the inherent statistical features of the image, such as by analyzing the differences between the original image and the tampered image in terms of pixel average value, red-green-blue correlation, wavelet domain statistical features, etc., the authenticity of the image can be judged.

[0004] Document images are formatted documents, and although they are images, they differ significantly from ordinary photographs. Photographs have rich elements, while document images typically contain only text and tables, lacking complex edges, making it difficult to detect tampering based on edge features. Another characteristic of document images is their complex backgrounds. The same document captured by different devices may have vastly different levels of noise, making it difficult to detect tampered areas by analyzing noise patterns. Furthermore, document images may have been altered before being photographed, significantly weakening the inherent statistical characteristics between the original and altered images, making it difficult to determine image authenticity based on intrinsic statistical features. In summary, existing detection methods are not suitable for document images, resulting in low accuracy in document image forgery detection. Therefore, a detection method specifically for document images is urgently needed to improve the accuracy of document image forgery detection. Summary of the Invention

[0005] In order to overcome the shortcomings of the prior art, the present invention aims to provide a document image detection method, system and device, which solves the problem that the detection methods in the prior art are not applicable to document images, resulting in a low accuracy rate of document image forgery detection.

[0006] To achieve the above objectives, the present invention employs the following technical solution: a document image detection method, comprising the following steps:

[0007] Step S1: Obtain the image of the document to be detected;

[0008] Step S2: Segment the document image to be detected to obtain the background area and the area to be detected. The area to be detected is at least one of the following: text area, seal area, and text plus seal area.

[0009] Step S3: Collect samples from the background area using a rectangle of a preset size to obtain multiple background area samples;

[0010] Step S4: Perform data augmentation on each background area sample, and construct positive and negative samples for text area, stamp area, or text plus stamp area respectively.

[0011] Step S5: Use positive and negative samples as training data to train the document image forgery detection model and obtain the trained document image forgery detection model.

[0012] Step S6: Input each region to be detected into the trained document image forgery detection model for detection to determine whether each region to be detected has been tampered with, and mark the regions to be detected that have been tampered with.

[0013] Preferably, step S4, which constructs a positive sample of a text area, a stamp area, or a text plus stamp area, includes at least one of the following methods:

[0014] By printing text on a background area sample according to text parameters through virtual printing, a positive sample of the text area is obtained. The text parameters include text font, text color and text size.

[0015] A positive sample of the stamp area is obtained by printing a stamp on the background area sample according to the stamp parameters through virtual printing. The stamp parameters include stamp color and stamp transparency.

[0016] Text is printed on a background sample using virtual printing based on text parameters. Then, a stamp is printed on the printed text using virtual printing based on stamp parameters, resulting in a positive sample of text plus a stamp area. The text parameters include text font, text color, and text size, while the stamp parameters include stamp color and stamp transparency.

[0017] Preferably, step S4, which constructs a negative sample of a text area, a stamp area, or a text plus stamp area, specifically includes the following steps:

[0018] Step S41: As a background layer, after removing the background texture of a portion of the background layer, text and / or stamps are superimposed by copying and cutting to obtain a fake text image.

[0019] Step S42: Perform data augmentation on the forged text image using one or more methods such as edge blurring, style transfer, and re-photographing to obtain negative samples of the text area, negative samples of the seal area, or negative samples of text plus seal area.

[0020] Preferably, when printing text on the background layer via virtual printing, the color data of the obtained positive sample of the text area is as follows:

[0021] C = A × x% + B × (1 - x%)

[0022] Where A represents the color data of the text layer, B represents the color data of the background layer, x% represents the transparency, and C represents the color data of the positive sample of the text area formed by opaquely blending the text layer into the background layer.

[0023] Preferably, the document image forgery detection model is a general image forgery detection deep neural network model or is composed of an edge feature unit and a general image forgery detection deep neural network model; the edge feature unit is a rich hidden write filter module or is composed of a channel attention module, a spatial attention module and a rich hidden write filter module;

[0024] Edge feature units are used to output noise feature channels or output weight tensors and noise feature channels;

[0025] A general-purpose deep neural network model for image forgery detection is used to learn and train on the input data.

[0026] Preferably, the channel attention module and the spatial attention module are connected in parallel with the rich hidden write filter module.

[0027] Preferably, the rich hidden write filter module extracts noise features from positive and negative samples and outputs a noise feature channel;

[0028] The channel attention module extracts noise features from positive and negative samples, outputs a noise feature channel, and assigns different weight tensors to the red, green, and blue channels of positive and negative samples as well as the noise feature channel.

[0029] The spatial attention module assigns different weight tensors to the features of different location regions of positive and negative samples.

[0030] Preferably, the training of the document image forgery detection model includes the following steps:

[0031] Step S51: The input of the general image forgery detection deep neural network model is any one of the following three types: one is training data; another is training data that outputs noise feature channels through the rich hidden write filter module; and the third is training data that outputs weight tensors through the channel attention module and the spatial attention module in sequence.

[0032] Step S52: The general image forgery detection deep neural network model constructs a loss function based on the error between the sample labels of the training data and the judgment result output by the general image forgery detection deep neural network model, and uses the backpropagation algorithm to perform iterative training with the goal of minimizing the loss function, so as to obtain the trained document image forgery detection model.

[0033] The general image forgery detection deep neural network model learns and adjusts the parameters of each layer of the convolutional neural network based on the noise feature channels output by the rich hidden write filter module. It constructs a loss function based on the error between the sample labels of the training data and the judgment results output by the general image forgery detection deep neural network model, and uses the backpropagation algorithm to perform iterative training with the goal of minimizing the loss function, so as to obtain a well-trained document image forgery detection model.

[0034] The general image forgery detection deep neural network model focuses on the noise feature changes of text and seal edges based on the weight tensors output by the channel attention module and the spatial attention module, and adjusts the parameters of each layer of the convolutional neural network based on the noise feature channels output by the rich hidden write filter module. The loss function is constructed based on the error between the sample labels of the training data and the judgment results output by the general image forgery detection deep neural network model, and the backpropagation algorithm is used to perform iterative training with the goal of minimizing the loss function, so as to obtain the trained document image forgery detection model.

[0035] A document image detection system, comprising:

[0036] The acquisition module is used to acquire the document image to be detected;

[0037] The segmentation module is used to segment the document image to obtain the background area and the area to be detected. The area to be detected includes at least one of the following: text area, stamp area, and text plus stamp area.

[0038] The sample acquisition module is used to acquire samples from the background area using a rectangular frame of a preset size, thereby obtaining multiple background area samples.

[0039] The data augmentation module is used to augment the data of each background area sample, and to construct positive and negative samples of at least one of the following: text area, stamp area, and text plus stamp area.

[0040] The training module is used to train the document image forgery detection model with training data to obtain a trained document image forgery detection model;

[0041] The detection module is used to input each region to be detected into the trained document image forgery detection model for detection, so as to determine whether each region to be detected has been tampered with.

[0042] A document image detection device includes: a memory, a processor, and a computer program;

[0043] The computer program is stored in the memory and configured to be executed by the processor to implement a document image detection method.

[0044] Compared with the prior art, the present invention has the following beneficial effects: The present invention discloses a document image detection method, which involves segmenting the document image to be detected to obtain a background area and a detection area; collecting samples from the background area using a rectangle of a preset size to obtain multiple background area samples; performing data augmentation on each background area sample to construct positive and negative samples respectively; using the obtained positive and negative samples as training data to train a document image forgery detection model to obtain a trained document image forgery detection model; and inputting each detection area into the trained document image forgery detection model for detection to determine whether each detection area has been tampered with. This method utilizes the background area of ​​the document image to generate positive and negative samples of at least one of the following: text area, seal area, and text plus seal area, for training a document image forgery detection model. It fully leverages the noise consistency characteristics of the untampered document image background to perform forgery detection on text areas, seal areas, or text plus seal areas containing valuable information in the document image. It can solve the problems of large errors and low accuracy of traditional general image forgery detection methods when used for document image tampering detection. It can detect multiple tampering methods, and its accuracy is improved by 20% compared to existing technologies.

[0045] Furthermore, there are multiple ways to construct positive samples, which enriches the training samples and enables the trained document image forgery detection model to detect various tampering methods with high accuracy.

[0046] Furthermore, the document image forgery detection model is either a general image forgery detection deep neural network model or a combination of edge feature units and a general image forgery detection deep neural network model. This facilitates the detection of subtle tampering traces and further improves detection accuracy.

[0047] The present invention also discloses a document image detection system, which loads a pre-trained document image forgery detection model to detect the region to be detected in the document image to be detected, and outputs the tampered location in the document image to be detected in a clear and intuitive way.

[0048] The present invention also discloses a document image detection device, which can effectively block and deter attempts to tamper with or forge documents when used for online authentication in e-commerce and e-government applications. Attached Figure Description

[0049] Figure 1 This is a flowchart of the document image detection method of the present invention;

[0050] Figure 2 This is a framework diagram of the document image forgery detection model of the present invention;

[0051] Figure 3 This is a schematic diagram illustrating the document image segmentation process of the present invention;

[0052] Figure 4 This is a schematic diagram illustrating the construction of positive and negative samples in this invention;

[0053] Figure 5 This is a schematic diagram of the display interface for the detection results of the present invention;

[0054] Figure 6 This is a schematic diagram of the document image detection system of the present invention;

[0055] Figure 7 This is a schematic diagram illustrating the process of detecting document images according to the present invention;

[0056] Figure 8 This is a schematic diagram of the structure of the document image detection device of the present invention. Detailed Implementation

[0057] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0058] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0059] The present invention will now be described in further detail with reference to the accompanying drawings:

[0060] See Figure 1 A flowchart of a document image detection method, which includes the following steps:

[0061] Step S1: Obtain the image of the document to be detected;

[0062] The document image to be detected can be obtained by file, such as reading a pre-stored document image from memory; it can also be obtained by network transmission, such as receiving a certificate photo uploaded by a user during online authentication; there are no restrictions on the format of the document image to be detected.

[0063] Step S2: Segment the document image to be detected to obtain the background area and the area to be detected. The area to be detected is at least one of the following: text area, seal area, and text plus seal area.

[0064] After obtaining the document image to be detected, the document image is segmented; a preset segmentation algorithm or a pre-trained segmentation model is used to segment the document image.

[0065] Documents come in various forms. For example, some documents contain only text, while others contain both text and stamps, with the stamps possibly superimposed on the text. Therefore, when segmenting the document image, we can fully consider its characteristics. We can segment areas with only text in the background into text areas, areas with only stamps in the background into stamp areas, and areas with text and stamps superimposed into text and stamp areas. All other areas in the document image besides the text areas, stamp areas, and text and stamp areas mentioned above are background areas. The background areas of document images all have textures, including the texture patterns or watermarks of the document template, the paper texture of the paper document formed after being photographed or copied, or image noise formed during the photographing or copying process of the paper document due to the limitations of the device resolution.

[0066] Text image forgery typically involves counterfeiting text and / or seals. For text areas, seal areas, and text-plus-seal areas containing valuable information, it is necessary to detect any signs of tampering. Therefore, when segmenting a document image to be detected, the background area and the area to be detected are segmented based on the characteristics of the document image itself. The area to be detected includes at least one of the following: text area, seal area, and text-plus-seal area. See also Figure 3 ,right Figure 3 The document image to be detected is segmented to obtain 6 text areas, 1 background area, and 1 text-with-stamp area. In this embodiment, the area to be detected includes the text areas and the text-with-stamp area; for example... Figure 3 As shown, the size of each text area can be different;

[0067] Step S3: Collect samples from the background area using a rectangle of a preset size to obtain multiple background area samples;

[0068] After obtaining the background area, samples are collected from it. Specifically, the background area is cropped into a small document image using a rectangle of a preset size as the background area sample. The size of the rectangle should be large enough to hold the text or stamp. To obtain a sufficient number of background area samples, sampling is performed at various locations within the background area, meaning that the collected background area samples can cover various locations within the document image's background area. Each background area sample is a small block of document image with a background texture but no text or stamp.

[0069] Step S4: Perform data augmentation on each background area sample, and construct positive and negative samples for text area, stamp area, or text plus stamp area respectively.

[0070] Positive samples are document images generated by performing normal document operations on background area samples, while negative samples are document images generated by tampering with or forging background area samples.

[0071] After obtaining the background area sample, using it as a template, document images generated through normal document operations such as virtual printing are used as positive samples, while document images generated through tampering and forgery operations are used as negative samples. It should be noted that multiple data augmentations can be performed on a single background area sample. That is, multiple positive and / or negative samples can be obtained based on a single background area sample, thus further increasing the number of positive and negative samples beyond the existing background area sample count.

[0072] The area to be detected can include text areas, seal areas, and text plus seal areas. Therefore, positive samples for text areas, positive samples for seal areas, positive samples for text plus seal areas, negative samples for text areas, negative samples for seal areas, and negative samples for text plus seal areas can be constructed specifically. A rich set of training samples enables the trained document image forgery detection model to detect various tampering methods with high accuracy.

[0073] In one optional implementation, positive samples may include at least one of text area positive samples, stamp area positive samples, and text plus stamp area positive samples. Optionally, the type of positive samples can be determined based on the type of the region to be detected. This fully considers the characteristics of the document image to be detected while controlling the scale of the training samples and accelerating the training process. For example, when the region to be detected only includes text areas, the positive samples may only include text area positive samples; when the region to be detected includes both text areas and text plus stamp areas, the positive samples may include both text area positive samples and text plus stamp area positive samples. One or more of the following methods can be used to perform data augmentation on each background region sample to construct positive samples:

[0074] Using a background area sample as the background layer, text is printed on the background layer virtually according to text parameters, resulting in a positive text area sample. These text parameters include font, color, and size. By adjusting these parameters, various fonts, colors, and sizes of text can be virtually printed on the background area sample. For example, when printing text on the background layer virtually, the color data of the resulting positive text area sample can be determined using the following expression:

[0075] C = A × x% + B × (1 - x%)

[0076] Where A represents the color data of the text layer, B represents the color data of the background layer, x% represents the transparency, and C represents the color data of the positive sample of the text area formed by opaquely blending the text layer into the background layer.

[0077] Using a background area sample as the background layer, a stamp is virtually printed on the background layer according to the stamp parameters, resulting in a positive stamp area sample. The stamp parameters include stamp color and stamp transparency. By adjusting the stamp parameters, stamps of various colors and transparency can be virtually printed on the background area sample.

[0078] Using a background area sample as the background layer, text is printed onto the background layer based on text parameters using virtual printing. Then, a stamp is printed onto the printed text using virtual printing based on stamp parameters, resulting in a positive sample of the text plus stamp area. Text parameters include font, color, and size, while stamp parameters include color and transparency. It should be noted that the document image after printing text onto the background layer can be used as the positive sample of the text area. In other words, this method can sequentially obtain both a positive sample of the text area and a positive sample of the text plus stamp area.

[0079] In one optional implementation, the negative samples may include at least one of text area negative samples, stamp area negative samples, and text plus stamp area negative samples. Optionally, the type of negative sample can also be determined based on the type of the area to be detected. This fully considers the characteristics of the document image to be detected and controls the scale of the training samples, accelerating the training process. For example, when the area to be detected only includes text areas, the negative samples may only include text area negative samples; when the area to be detected includes both text areas and text plus stamp areas, the negative samples may include both text area negative samples and text plus stamp area negative samples. Specifically, the following methods can be used to data augment each background area sample to construct negative samples: using the background area sample as a background layer, removing the background texture of a portion of the background layer, and then overlaying text and / or stamps by copying and cutting to obtain a forged text image; data augmentation can be performed on the forged text image using one or more of the following methods: edge blurring, style transfer, and re-photographing, to obtain text area negative samples, stamp area negative samples, or text plus stamp area negative samples.

[0080] Figure 4 This is a schematic diagram illustrating the construction of positive and negative samples provided in an embodiment of the present invention, as shown below. Figure 4 As shown, for the background area sample on the left, printing the words "Notice" virtually yields positive sample 1, which is a positive sample of the text area. Overlaying a stamp onto positive sample 1 yields positive sample 2, which is a positive sample of the text and stamp area. For the background area sample on the left, overlaying the words "Notice" by copying and cutting creates a forged text image. Further processing of the forged text image using one or more methods such as edge blurring, style transfer, and re-photographing yields a negative sample, which is a negative sample of the text area.

[0081] Step S5: Use the obtained positive and negative samples as training data to train the document image forgery detection model, and obtain the trained document image forgery detection model.

[0082] Training a document image forgery detection model includes the following steps:

[0083] Step S51: The input of the general image forgery detection deep neural network model is divided into three types: one is training data, one is training data output through the rich hidden write filter module to output noise feature channel, and the other is training data output through the channel attention module and spatial attention module to output weight tensor.

[0084] The input to a general image forgery detection deep neural network model can be a region of a document image. This model includes a classifier as its output, which determines whether the input document image has been forged. In each training round, a certain amount of training data is input, and the model's output is compared to whether it matches a positive or negative sample. If the input is a positive sample, but the model's output determines that the sample is forged, then the model's output is incorrect, and there is a significant loss error between the output and the sample label. Similarly, if the input is a negative sample, but the model's output determines that the sample is not forged, then the model's output is also incorrect, and there is a significant loss error between the output and the sample label. If the input is a positive sample, the output of the general image forgery detection deep neural network model will determine that the sample has not been forged, and the output of the general image forgery detection deep neural network model will be consistent with the sample label. Similarly, if the input is a negative sample, the output of the general image forgery detection deep neural network model will determine that the sample has been forged, and the output of the general image forgery detection deep neural network model will be consistent with the sample label. The loss error between the output of the general image forgery detection deep neural network model and the sample label is very small. During each round of training, the backpropagation algorithm can be used to adjust the parameters in the general image forgery detection deep neural network model according to the loss error between the output of the general image forgery detection deep neural network model and the sample label, so as to minimize the loss error, that is, to make the output of the general image forgery detection deep neural network model as consistent as possible with the sample label. After training, the initial general image forgery detection model can be transformed into an optimized model for the document image to be detected, which can be used to detect the document image and improve the detection accuracy.

[0085] On the other hand, to facilitate the detection of subtle tampering traces and further improve detection accuracy, this embodiment can also construct a document image forgery detection model based on a convolutional neural network with an attention mechanism. In this case, the sequentially connected channel attention module, spatial attention module, and rich hidden write filter module are the edge feature units of the document image forgery detection model, and the convolutional neural network is a general deep neural network model for image forgery detection, such as... Figure 2 As shown. After obtaining a sufficient number of positive and negative samples, these are used as training data to train the document image forgery detection model, resulting in a well-trained model. The document image forgery detection model includes edge feature units and a general image forgery detection deep neural network model. The general image forgery detection deep neural network model is built based on deep learning neural networks.

[0086] On the other hand, in order to fully utilize the noise consistency characteristics of the untampered document image background and further improve the forgery detection accuracy of text areas, seal areas, or text plus seal areas containing valuable information in the document image, in another embodiment, the edge feature unit may also include a rich hidden write filter module connected in parallel with the channel attention module and the spatial attention module, such as... Figure 2 As shown.

[0087] The channel attention module extracts noise features from positive and negative samples, outputting a noise feature channel. It assigns different weight tensors to the red, green, and blue channels of both positive and negative samples, as well as the noise feature channel. The channel attention module assigns a weight tensor to the noise feature changes in the noise feature channel that is greater than the square root of the sum of the squares of all tensor elements in the weight tensors of the red, green, and blue channels. The spatial attention module assigns different weight tensors to features in different regions of both positive and negative samples. The spatial attention module assigns greater weight to noise feature changes near text edges or seal edges than to other regions. The parameters of both the channel and spatial attention modules are continuously updated with the input training data. After training stops, the channel attention module has the ability to assign greater weight to the noise feature channels of the text image to be detected, while the spatial attention module has the ability to assign greater weight to noise feature changes at the edges of text and / or seals in the text image to be detected, thus making it easier to detect noise feature inconsistencies disrupted by tampering. The channel attention module and spatial attention module are task-dependent and proactively and consciously focused on the surrounding areas of text and / or seals. By training on a large amount of training data, they are optimized for the document images to be detected, especially for magnifying and recognizing subtle differences at the edges of text and / or seals. This allows for the accurate detection of subtle traces caused by tampering and forgery operations, thereby further improving the detection accuracy of the document image forgery detection model.

[0088] In order to detect this corrupted noise feature in subsequent detection, the rich hidden write filter module extracts noise features from positive and negative samples and outputs a noise feature channel.

[0089] Step S52: The general image forgery detection deep neural network model constructs a loss function based on the error between the sample labels of the training data and the judgment results output by the general image forgery detection deep neural network model, and uses the backpropagation algorithm to perform iterative training with the goal of minimizing the loss function;

[0090] A general-purpose deep neural network model for image forgery detection focuses on noise feature changes at the edges of text and seals based on the weight tensors output by the channel attention and spatial attention modules. The rich hidden write filter module extracts noise features from positive and negative samples, outputting a noise feature channel. After learning and adjusting the parameters of each convolutional neural network layer based on the noise feature channel output by the rich hidden write filter module, a loss function is constructed based on the error between the sample labels of the training data and the judgment results output by the general-purpose deep neural network model. Backpropagation is used to iteratively train the model with the goal of minimizing the loss function, resulting in a well-trained document image forgery detection model. The edge feature unit, through the rich hidden write filter module, channel attention module, and spatial attention module, focuses on abnormal noise changes in negative samples relative to positive samples near the edges of text or seals. This allows the trained image forgery detection deep neural network model to prioritize detecting abnormal noise feature changes near the edges of text or seals during subsequent forgery detection. The general-purpose image forgery detection deep neural network model can be any of AlexNet, RestNet, or VGGNet, or a deep neural network model connected to the channel attention and spatial attention modules.

[0091] A general image forgery detection deep neural network model learns and adjusts the parameters of each layer of the convolutional neural network based on the noise feature channels output by the rich hidden write filter module. It then constructs a loss function based on the error between the sample labels of the training data and the judgment results output by the general image forgery detection deep neural network model. The model is then iteratively trained using a backpropagation algorithm with the goal of minimizing the loss function, resulting in a well-trained document image forgery detection model. Naturally captured text images have relatively smooth background pixel variations, and the noise possesses certain characteristic distributions. To make the forged text images deceptive, the forger needs to refine the edges of the forged text or seals to eliminate the abruptness and cracks in the forged edges, leaving numerous invisible traces of forgery in the text image. This inherent property of noise at the forged edges is destroyed.

[0092] During training, the general image forgery detection deep neural network model constructs a loss function based on the error between the sample labels of the training data and the judgment results output by the model. It then uses backpropagation to iteratively train the model with the goal of minimizing this loss function. For example, positive samples are labeled as 0, negative samples as 1, and the judgment results output by the model are values ​​between 0 and 1, representing the probability that the document image has been tampered with. A higher value indicates a higher likelihood of forgery. Loss functions can include, for example, L1-norm or L2-norm loss functions. After training with a sufficient amount of data, the trained model can then focus on detecting abnormal noise changes near text edges or seal edges during subsequent forgery detection. This approach aims to accelerate the training of the document image forgery detection model, shorten training time, and improve its detection accuracy. A general image forgery detection deep neural network model consists of multiple convolutional layers, pooling layers, and fully connected layers. It is trained using ordinary photographic images, such as AlexNet, RestNet, or VGGNet.

[0093] Step S6: Input each region to be detected into the trained document image forgery detection model for detection to determine whether each region to be detected has been tampered with. Mark the regions to be detected that have been tampered with in the document image by using semi-transparent, highlighted or conspicuous borders.

[0094] After obtaining the trained document image forgery detection model, the detection phase begins. The text image to be detected is input into the model according to each detection region obtained in step S2; Figure 3 Taking the document image to be detected as an example, the six text regions and one text plus stamp region obtained from the segmentation need to be input into the trained document image forgery detection model in sequence to determine whether each region to be detected has been tampered with; when one or more regions to be detected in the document image to be detected have been tampered with, the document image to be detected is determined to be forged.

[0095] Furthermore, appropriate detection models can be trained separately for text areas, seal areas, and text plus seal areas. Specifically, during the training phase, a sufficient number of positive and negative text area samples can be used to train the document image forgery detection model, resulting in a well-trained first document image forgery detection model. During the detection phase, the text areas of the document image to be detected are input one by one into the well-trained first document image forgery detection model for detection, and the determination results of whether each text area has been tampered with are obtained.

[0096] Similarly, during the training phase, a sufficient number of positive and negative samples of the stamp area can be used to train the document image forgery detection model, resulting in a well-trained second document image forgery detection model. During the detection phase, the stamp areas of the document image to be detected are input one by one into the well-trained second document image forgery detection model for detection, and the determination results of whether each stamp area has been tampered with are obtained.

[0097] Similarly, during the training phase, a sufficient number of positive and negative samples of text stamp areas can be used to train the document image forgery detection model, resulting in a well-trained third document image forgery detection model. During the detection phase, the text stamp areas of the document image to be detected are input one by one into the well-trained third document image forgery detection model for detection, and the determination results of whether each text stamp area has been tampered with are obtained.

[0098] In the above process, if the document image forgery detection model includes a rich hidden write filter module and a general image forgery detection deep neural network model, when detecting the area to be detected, the noise feature channel is first extracted from the area to be detected, and then the noise feature channel is input into the general image forgery detection deep neural network model for detection.

[0099] In the above process, if the document image forgery detection model includes a channel attention module, a spatial attention module, a rich hidden write filter module, and a general image forgery detection deep neural network model, when detecting the area to be detected, the weight tensor and noise feature channel are first extracted from the area to be detected, and then the weight tensor and noise feature channel are input into the general image forgery detection deep neural network model for detection.

[0100] To visually and intuitively display the detection results of document images, the areas to be detected that have been tampered with can be marked in the document image using semi-transparent, highlighted, or conspicuous borders. Figure 5 This is a schematic diagram of a display interface for detection results provided in an embodiment of the present invention. Figure 5 As shown, when the second text area from top to bottom is detected to have been tampered with, it is marked with a conspicuous border. This can accurately locate the tampered area in the document image, allowing users to easily distinguish the tampered area from the untampered or forged area.

[0101] This method utilizes the document image to generate positive and negative samples for training a document image forgery detection model. It fully leverages the characteristics of document images and solves the problems of large errors and low accuracy in traditional general image forgery detection methods when used for document image tampering detection. It can detect various tampering methods with high accuracy. When used for online authentication in e-commerce and e-government applications, it can effectively block and deter attempted document tampering and forgery.

[0102] The document image detection system 50 provided in this embodiment may include: an acquisition module 501, a segmentation module 502, a sample acquisition module 503, a data augmentation module 504, a training module 505, and a detection module 506, such as... Figure 6 As shown.

[0103] The acquisition module 501 is used to acquire the document image to be detected;

[0104] The segmentation module 502 is used to segment the document image to obtain the background area and the detection area of ​​the document image. The detection area includes at least one of the following: text area, stamp area, and text plus stamp area.

[0105] The sample acquisition module 503 is used to acquire samples from the background area using a rectangle of a preset size, thereby obtaining multiple background area samples.

[0106] The data augmentation module 504 is used to perform data augmentation on each background region sample, constructing positive and negative samples respectively.

[0107] The training module 505 is used to train the document image forgery detection model using the obtained positive and negative samples as training data, so as to obtain the trained document image forgery detection model.

[0108] The detection module 506 is used to input each region to be detected into the trained document image forgery detection model for detection, so as to determine whether each region to be detected has been tampered with.

[0109] like Figure 7 As shown, positive and negative samples are constructed based on the document image to be detected. The obtained positive and negative samples are used to train the document image forgery detection model. Finally, the trained document image forgery detection model is loaded to detect the region to be detected in the document image to be detected, and the tampering location in the document image to be detected is output.

[0110] like Figure 8 As shown, the document image detection device 70 includes a memory 701 and at least one processor 702. The memory 701 stores computer-executable instructions; the at least one processor 702 executes the computer-executable instructions stored in the memory 701, causing the at least one processor 702 to perform a document image detection method. The memory 701 and the at least one processor 702 are connected via a bus 703.

[0111] The memory 701 may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 701 stores programs, and the processor 702 executes the programs after receiving execution instructions. Furthermore, the software programs and modules within the memory 701 may also include an operating system, which may include various software components and / or drivers for managing system tasks (such as memory management, storage device control, power management, etc.) and can communicate with various hardware or software components to provide an operating environment for other software components.

[0112] Processor 702 can be an integrated circuit chip with signal processing capabilities. The aforementioned processor 702 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. It is understood that... Figure 8 The structure shown is for illustrative purposes only and may include more... Figure 8 The more or fewer components shown, or having the same Figure 8 The different configurations shown. Figure 8 The components shown can be implemented in hardware and / or software.

[0113] It should be noted that the document image detection device provided in this embodiment includes, but is not limited to, at least one of the following: user-side device and network-side device. User-side device includes, but is not limited to, computers, smartphones, tablets, digital broadcasting terminals, messaging devices, game consoles, personal digital assistants, etc. Network-side device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of computers or network servers. Cloud computing is a type of distributed computing, consisting of a super virtual computer composed of a group of loosely coupled computers.

[0114] This invention has been described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications can be made to the exemplary embodiments without departing from the scope of the invention. For example, various operational steps and components for performing operational steps can be implemented in different ways depending on the specific application or considering any number of cost functions associated with the operation of the system (e.g., one or more steps can be deleted, modified, or combined with other steps).

[0115] Furthermore, as those skilled in the art will understand, the principles of the present invention can be reflected in a computer program product on a computer-readable storage medium pre-loaded with computer-readable program code. Any tangible, non-transitory computer-readable storage medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu-ray discs, etc.), flash memory, and / or the like. These computer program instructions may be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to form a machine, such that instructions executing on the computer or other programmable data processing apparatus can generate means for implementing a specified function. These computer program instructions may also be stored in a computer-readable storage medium that can instruct the computer or other programmable data processing apparatus to operate in a particular manner, such that instructions stored in the computer-readable storage medium can form an article of manufacture including means for implementing the specified function. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to perform a series of operational steps on the computer or other programmable apparatus to produce a computer-implemented process, such that instructions executing on the computer or other programmable apparatus can provide steps for implementing the specified function.

[0116] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A method for detecting document images, characterized in that, Includes the following steps: Step S1: Obtain the image of the document to be detected; Step S2: Segment the document image to be detected to obtain the background area and the area to be detected. The area to be detected is at least one of the following: text area, seal area, and text plus seal area. Step S3: Collect samples from the background area using a rectangle of a preset size to obtain multiple background area samples; Step S4: Perform data augmentation on each background area sample, constructing positive and negative samples for text areas, stamp areas, or text plus stamp areas respectively; specific steps are as follows: Step S41: As a background layer, after removing the background texture of a portion of the background layer, text and / or stamps are superimposed by copying and cutting to obtain a fake text image. Step S42: Perform data augmentation on the forged text image using one or more methods such as edge blurring, style transfer, and re-photographing to obtain negative samples of the text area, negative samples of the seal area, or negative samples of the text plus seal area. Step S5: Use positive and negative samples as training data to train the document image forgery detection model and obtain the trained document image forgery detection model. Step S6: Input each region to be detected into the trained document image forgery detection model for detection to determine whether each region to be detected has been tampered with, and mark the regions to be detected that have been tampered with.

2. The document image detection method according to claim 1, characterized in that, The positive sample constructed in step S4, which consists of a text area, a stamp area, or a text plus stamp area, includes at least one of the following methods: By printing text on a background area sample according to text parameters through virtual printing, a positive sample of the text area is obtained. The text parameters include text font, text color and text size. A positive sample of the stamp area is obtained by printing a stamp on the background area sample according to the stamp parameters through virtual printing. The stamp parameters include stamp color and stamp transparency. Text is printed on a background sample using virtual printing based on text parameters. Then, a stamp is printed on the printed text using virtual printing based on stamp parameters, resulting in a positive sample of text plus a stamp area. The text parameters include text font, text color, and text size, while the stamp parameters include stamp color and stamp transparency.

3. The document image detection method according to claim 2, characterized in that, When printing text on the background layer using virtual printing, the color data of the positive sample of the text area is as follows: in, The color data represents the text layer. This represents the color data of the background layer. Indicates transparency, This represents the color data of the positive sample of the text area formed by opaquely blending the text layer into the background layer.

4. The document image detection method according to claim 1, characterized in that, The document image forgery detection model is a general image forgery detection deep neural network model or is composed of an edge feature unit and a general image forgery detection deep neural network model; the edge feature unit is a rich hidden write filter module or is composed of a channel attention module, a spatial attention module and a rich hidden write filter module. Edge feature units are used to output noise feature channels or output weight tensors and noise feature channels; A general-purpose deep neural network model for image forgery detection is used to learn and train on the input data.

5. The document image detection method according to claim 4, characterized in that, The channel attention module and spatial attention module are connected in parallel with the rich hidden write filter module.

6. The document image detection method according to claim 4, characterized in that, The rich hidden write filter module extracts noise features from positive and negative samples and outputs a noise feature channel. The channel attention module extracts noise features from positive and negative samples, outputs a noise feature channel, and assigns different weight tensors to the red, green, and blue channels of positive and negative samples as well as the noise feature channel. The spatial attention module assigns different weight tensors to the features of different location regions of positive and negative samples.

7. The document image detection method according to claim 4, characterized in that, The training of the document image forgery detection model includes the following steps: Step S51: The input of the general image forgery detection deep neural network model is any one of the following three types: one is training data; another is training data that outputs noise feature channels through the rich hidden write filter module; and the third is training data that outputs weight tensors through the channel attention module and the spatial attention module in sequence. Step S52: The general image forgery detection deep neural network model constructs a loss function based on the error between the sample labels of the training data and the judgment result output by the general image forgery detection deep neural network model, and uses the backpropagation algorithm to perform iterative training with the goal of minimizing the loss function, so as to obtain the trained document image forgery detection model. The general image forgery detection deep neural network model learns and adjusts the parameters of each layer of the convolutional neural network based on the noise feature channels output by the rich hidden write filter module. It constructs a loss function based on the error between the sample labels of the training data and the judgment results output by the general image forgery detection deep neural network model, and uses the backpropagation algorithm to perform iterative training with the goal of minimizing the loss function, so as to obtain a well-trained document image forgery detection model. The general image forgery detection deep neural network model focuses on the noise feature changes of text and seal edges based on the weight tensors output by the channel attention module and the spatial attention module, and adjusts the parameters of each layer of the convolutional neural network based on the noise feature channels output by the rich hidden write filter module. The loss function is constructed based on the error between the sample labels of the training data and the judgment results output by the general image forgery detection deep neural network model, and the backpropagation algorithm is used to perform iterative training with the goal of minimizing the loss function, so as to obtain the trained document image forgery detection model.

8. A document image detection system, based on the document image detection method of claim 1, characterized in that, include: The acquisition module is used to acquire the document image to be detected; The segmentation module is used to segment the document image to obtain the background area and the area to be detected. The area to be detected includes at least one of the following: text area, stamp area, and text plus stamp area. The sample acquisition module is used to acquire samples from the background area using a rectangular frame of a preset size, thereby obtaining multiple background area samples. The data augmentation module is used to augment the data of each background area sample, and to construct positive and negative samples of at least one of the following: text area, stamp area, and text plus stamp area. The training module is used to train the document image forgery detection model with training data to obtain a trained document image forgery detection model; The detection module is used to input each region to be detected into the trained document image forgery detection model for detection, so as to determine whether each region to be detected has been tampered with.

9. A document image detection device, characterized in that, include: Memory, processor, and computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the document image detection method as described in any one of claims 1-7.

Images