A certificate image photographing detection method and system based on multi-feature fusion

Abstract

The technical scheme of the present application provides a certificate image copying detection system based on multi-feature fusion, characterized by comprising a deep learning training module, an analysis module, a picture feature extraction module and a deep learning model discrimination module. The other technical scheme of the present application provides a certificate image copying detection method based on multi-feature fusion, characterized by comprising the following steps: the analysis module receives a base64 encoded picture sent by other systems; the picture feature extraction module extracts features from the decoded picture; the obtained 72-dimensional features are calculated together with the original picture by the deep learning discrimination module to obtain the result of whether the final picture is copied; and the final copying detection result is returned to the analysis module and then returned to the corresponding calling party by the analysis module.

Description

Technical Field

[0001] This invention relates to a method for detecting photocopied ID card images based on multi-feature fusion, and a system implemented based on this method, used to determine whether an ID card photo is a photocopy of a screen when uploading an ID card image, belonging to the field of image processing technology. Background Technology

[0002] In the field of government services, with increasing informatization and digitalization, electronic images, as a carrier of information, are being used more and more widely. For example, photos of various certificates are uploaded to government systems as supporting documentation. However, some criminals attempt to forge physical images by photographing them for illegal purposes. Therefore, the purpose of image forgery detection is to identify these violations as much as possible and ensure the authenticity of images.

[0003] Currently, most methods for detecting remakes fall into two main categories:

[0004] The first type is machine learning methods, which involve manually selecting features such as grayscale histograms, gradient features, and statistical features of DCT transform coefficients. These features are then classified using the machine learning method LibSVM to determine whether the film is a remake.

[0005] The second type is deep learning methods. The image undergoes certain preprocessing steps and is then fed into a deep learning network to automatically extract relevant features, ultimately determining whether it is a reproduction. Summary of the Invention

[0006] The purpose of this invention is to detect whether the certificate images uploaded or photographed by users in government services are reproduced images, thereby ensuring the authenticity of data within the government system as much as possible and increasing the cost of forgery for users.

[0007] To achieve the above objectives, one technical solution of the present invention is to provide a document image re-photographing detection system based on multi-feature fusion, characterized in that it includes:

[0008] The deep learning training module is used to train the deep learning model after it has been built, and to deploy the trained deep learning model to the deep learning model discrimination module.

[0009] The analysis module is used to: receive a base64 encoded image sent by the caller, decode the image to obtain the decoded image, and send the decoded image to the image feature extraction module; send the 72-dimensional features returned by the image feature extraction module and the original image to the deep learning model discrimination module; and return the result returned by the deep learning model discrimination module to the corresponding caller.

[0010] The image feature extraction module is used to extract 72-dimensional features from the received image and then send the extracted features back to the analysis module.

[0011] The deep learning model discrimination module is used to detect photocopied certificates based on 72-dimensional features and the original image using a trained deep learning model, and then returns the results to the analysis module.

[0012] Preferably, the deep learning model includes a residual module, an MLP module, and a loss module, wherein:

[0013] The residual module introduces a shortcut connection, which allows gradients to be backpropagated even when the deep learning model has many layers, thus solving the gradient vanishing problem.

[0014] The MLP module is used to: use a two-layer perceptron to cross-upgrade the 72-dimensional features to 256 dimensions, then concatenate the features with the 1000-dimensional features extracted by ResNet101 to form 1256-dimensional features; finally, use a single-layer perceptron to fuse the features.

[0015] The loss module is used to calculate the loss of the deep learning model using the cross-entropy function.

[0016] Preferably, the mathematical representation of the residual module is as follows:

[0017] y = σ(F(x,W) + x)

[0018] Where y represents the output of the residual block, σ() represents the activation function, F() represents the residual function, x represents the input, and W represents all weights within the residual block.

[0019] Preferably, the loss of the deep learning model is expressed as:

[0020]

[0021] Where: y i p represents the label of sample i, with 1 for positive class and 0 for negative class; i This represents the probability that sample i is predicted to be of the positive class; N is the number of samples.

[0022] Preferably, after the deep learning model is constructed, the deep learning training module trains the deep learning model using the open-source Moiré dataset.

[0023] Preferably, the 72-dimensional features include the mean of the R channel, the mean of the G channel, the mean of the B channel, the variance of the R channel, the variance of the G channel, the variance of the B channel, the grayscale variance, the grayscale mean, a 32-bit gradient histogram, and a 32-bit grayscale histogram.

[0024] Another technical solution of the present invention is to provide a method for detecting document image re-photographing based on multi-feature fusion, characterized by comprising the following steps:

[0025] Step S101: The analysis module receives a base64 encoded image sent by another system;

[0026] Step S102: The image feature extraction module extracts features from the decoded image;

[0027] Step S103: Based on the 72-dimensional features obtained in step S102, the deep learning discrimination module calculates together with the original image to determine whether the final image is a re-photograph.

[0028] Step S104: Return the final re-photographing detection result to the analysis module, which then returns it to the corresponding caller.

[0029] Compared with existing technical solutions, the present invention has the following beneficial effects:

[0030] This invention leverages the ability of deep learning models to extract low-level features from images at a shallow level and high-level features at a deep level, while also incorporating manually selected features to provide global statistical information about the images. By simultaneously using features automatically learned by deep learning and manually selected features, and fusing them for detecting reproduced images, this feature fusion approach better utilizes the complementarity between features compared to existing techniques that use only one of the two methods. By combining the advantages of different features, it helps the model integrate local and global feature information, thereby improving model performance. Attached Figure Description

[0031] Figure 1 This is a flowchart of a re-photographing detection process according to an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram of a re-photographing detection system according to an embodiment of the present invention;

[0033] Figure 3 The structure of the residual module is illustrated. Detailed Implementation

[0034] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0035] like Figure 1As shown, a document image re-photographing detection method according to an embodiment of the present invention includes the following steps:

[0036] Step S101: The analysis module receives a base64 encoded image sent by another system.

[0037] Step S102: Extract features from the decoded image.

[0038] In one embodiment of the present invention, the present invention achieves the extraction of 11 features, including R-channel mean, G-channel mean, B-channel mean, R-channel variance, G-channel variance, B-channel variance, gray-level variance, gray-level mean, 32-bit gradient histogram, and 32-bit gray-level histogram, for a total of 72-dimensional features.

[0039] Step S103: Based on the 72-dimensional features obtained in step S102, the deep learning discrimination module calculates together with the original image to determine whether the final image is a copy.

[0040] Step S104: Return the final re-photographing detection result to the analysis module, which then returns it to the corresponding caller.

[0041] The copy detection method of this invention adds a dedicated document image copy detection method to various government system processes. By detecting copy images based on multiple features of the document images, it significantly improves the copy recognition rate of user-uploaded document images in government systems and ensures the quality of image data.

[0042] To implement the above embodiments, the present invention also proposes a document image re-photographing detection system 210, such as... Figure 2 As shown, it includes a deep learning model training module 211, an analysis module 212, an image feature extraction module 213, and a deep learning model discrimination module 214.

[0043] Deep Learning Training Module 211

[0044] The deep learning model training module 211 is mainly responsible for the construction and training of the deep learning model structure. This deep learning model mainly includes the residual module, the MLP module, and the loss module.

[0045] 1) Residual module:

[0046] The main structure of the residual module is as follows: Figure 3 As shown, this module primarily addresses the issue of decreased model accuracy when the model has too many layers. The residual module introduces a shortcut connection, which allows gradients to be propagated back even with a large number of model layers, thus resolving the gradient vanishing problem.

[0047] The mathematical representation of the residual module is as follows:

[0048] y = σ(F(x,W) + x)

[0049] Where y represents the output of the residual block, σ() represents the activation function, F() represents the residual function, x represents the input (i.e., the three-channel image), and W represents all weights within the residual block.

[0050] 2) MLP module:

[0051] The MLP module mainly implements two functions:

[0052] A two-layer perceptron was used to cross-enlarge the 72-dimensional features to 256 dimensions, and then these features were concatenated with the 1000-dimensional features extracted by ResNet101 to obtain 1256-dimensional features. Finally, a single-layer perceptron was used to fuse the features.

[0053] 3) Loss Module

[0054] The model's loss module uses the cross-entropy function to calculate the model's loss, as shown in the following formula:

[0055]

[0056] Where: y i p represents the label of sample i, with 1 for positive class and 0 for negative class; i This represents the probability that sample i is predicted to be of the positive class; N is the number of samples.

[0057] After the deep learning model is built, it is trained using the open-source Moiré dataset. The deep learning model can then be deployed to the deep learning model discrimination module 214 for use.

[0058] Analysis Module 212

[0059] In one embodiment of the present invention, the analysis module 212 is used to receive a base64 encoded image sent by the caller, and can decode the data to obtain a decoded image. The analysis module 212 sends the decoded image to the image feature extraction module 213 to obtain the extracted features, and then sends the original image and features to the deep learning model discrimination module 214 to obtain the final result, which is then returned to the caller.

[0060] Image feature extraction module 213

[0061] In one embodiment of the present invention, the image feature extraction module 213 extracts 11 statistical features of the image, namely, R-channel mean, G-channel mean, B-channel mean, R-channel variance, G-channel variance, B-channel variance, gray-level variance, gray-level mean, 32-bit gradient histogram, and 32-bit gray-level histogram, for a total of 72 dimensions. The extracted features are then sent back to the analysis module 212.

[0062] Deep learning model discrimination module 214

[0063] In one embodiment of the present invention, the deep learning model discrimination module 214 uses a trained deep learning model to detect document image re-photographing, including the following steps:

[0064] Step 1: Receive the original image and features sent by the image feature extraction module 213 and calculate the results.

[0065] Step 2: Return the results to the analysis module 212.

[0066] It should be noted that, unless otherwise specified, one or more embodiments and features described in this specification can be combined with each other. Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the invention. The advantages of the present invention have been fully and effectively realized. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments; without departing from the stated principles, the implementation of the present invention can be modified or altered in any way.

Claims

1. A document image re-photographing detection system based on multi-feature fusion, characterized in that, include: The deep learning training module is used to train the deep learning model after it has been built, and to deploy the trained deep learning model to the deep learning model discrimination module. The deep learning model includes a residual module, an MLP module, and a loss module. The residual module introduces a shortcut connection, which allows gradients to be backpropagated even when the deep learning model has many layers, thus solving the gradient vanishing problem. The MLP module is used to: use a two-layer perceptron to cross-upgrade the 72-dimensional features to 256 dimensions, then concatenate the features with the 1000-dimensional features extracted by ResNet101 to form 1256-dimensional features; finally, use a single-layer perceptron to fuse the features. The loss module is used to calculate the loss of the deep learning model using the cross-entropy function. The analysis module is used to: receive a base64 encoded image sent by the caller, decode the image to obtain the decoded image, and send the decoded image to the image feature extraction module; send the 72-dimensional features returned by the image feature extraction module and the original image to the deep learning model discrimination module; and return the result returned by the deep learning model discrimination module to the corresponding caller. The image feature extraction module is used to extract 72-dimensional features from the received image and then send the extracted features back to the analysis module. The deep learning model discrimination module is used to detect document image re-photographing based on 72-dimensional features and the original image using a trained deep learning model, and returns the results to the analysis module.

2. The document image re-photographing detection system based on multi-feature fusion as described in claim 1, characterized in that, The mathematical representation of the residual module is as follows: in, This represents the output of the residual block. Represents the activation function. Represents the residual function. Represents input, This represents all weights within the residual block.

3. The document image re-photographing detection system based on multi-feature fusion as described in claim 1, characterized in that, The loss of the deep learning model is expressed as: in: Indicates sample The label is 1 for positive classes and 0 for negative classes; Indicates sample The probability of predicting it as positive; This represents the number of samples.

4. The document image re-photographing detection system based on multi-feature fusion as described in claim 1, characterized in that, After the deep learning model is built, the deep learning training module uses the open-source Moiré dataset to train the deep learning model.

5. The document image re-photographing detection system based on multi-feature fusion as described in claim 1, characterized in that, The 72-dimensional features include the mean of the R channel, the mean of the G channel, the mean of the B channel, the variance of the R channel, the variance of the G channel, the variance of the B channel, the grayscale variance, the grayscale mean, the 32-bit gradient histogram, and the 32-bit grayscale histogram.

6. A method for detecting document image re-photographing based on multi-feature fusion, employing the document image re-photographing detection system as described in claim 1, characterized in that, Includes the following steps: Step S101: The analysis module receives a base64 encoded image sent by another system; Step S102: The image feature extraction module extracts features from the decoded image; Step S103: Based on the 72-dimensional features obtained in step S102, the deep learning discrimination module calculates together with the original image to determine whether the final image is a re-photograph. Step S104: Return the final re-photographing detection result to the analysis module, which then returns it to the corresponding caller.

Images