Methods, devices, electronic equipment, and storage media for identifying altered ID photo images

Abstract

This invention relates to a method, apparatus, electronic device, and storage medium for identifying tampered ID photo images. The method includes the following steps: S1, inputting an ID photo image to be detected as tampered with; S2, performing pixel-based analysis of the input ID photo to determine if it has been tampered with, using a residual structure neural network to locate the tampered area, predicting based on the classification results on the combined blocks, and then using a threshold-based method to determine if the image has been tampered with; S3, analyzing the spectral signal to determine if the ID photo has been tampered with, inputting the signal to a GAN simulator, where the GAN simulator acquires the RGB3 channel spectrum of the image, performs spectral separation, calculates the logarithmic spectrum, normalizes it to [-1, 1], and performs artifact detection to determine if the image has been tampered with; S4, outputting the detection result. According to this invention, the method for identifying tampered ID photo faces created by GAN network forgery can be effectively identified.

Description

Technical Field

[0001] This invention relates to the field of communications, and more specifically, to a method, apparatus, electronic device, and storage medium for identifying tampering with ID photo images. Background Technology

[0002] The primary purpose of altering ID photo images is to tamper with identity information. Therefore, ID photo alteration detection is crucial for obtaining accurate information, and existing patents document numerous technologies for identifying whether an image has been altered, such as:

[0003] Chinese patent CN114677670A: A method for automatic identification and localization of ID card tampering. The method inputs a detection image A into a trained Rankiqa model, performs transformation processing, and then uses a CAT-Net model to determine whether the image has been tampered with. However, CAT-Net's detection results are significantly reduced when some images are rotated more than 180 degrees.

[0004] Chinese patent CN110443804B: A method, apparatus, and computer device for identifying resampling tampering of JPEG images. This method involves converting the JPEG image to grayscale, obtaining the non-pure color blocks in the JPEG image, and filtering them to eliminate quantization noise. The new JPEG image is divided into multiple sub-images, and the resampling spectrum of each sub-image is obtained. Based on the resampling spectrum, a resampling factor estimate for each sub-image is calculated, and a resampling factor interval estimate is performed. The estimation results determine whether the JPEG image has been resampling tampered with. However, this method only applies to JPEG images; converting the original image to grayscale inevitably results in pixel loss.

[0005] Chinese patent CN115907781A: A method, apparatus, computer-readable medium, and electronic device for determining fake images. The method involves extracting features from an image to be determined, obtaining its features, and then matching these features with existing features in a historical image feature database to obtain target existing features. If the total number of objects associated with the target existing features exceeds a first preset number, the image is determined to be a fake image. However, this patent relies on existing features in a historical image feature database.

[0006] None of the methods proposed in the above patents can effectively detect ID photos that have been tampered with by Generative Adversarial Networks (GANs).

[0007] Artificial intelligence technology can be used to automatically generate realistic fake ID photos. GAN (Generative Adversarial Network) is a combination of two networks: a generator network (Generator) generates simulated data, and a discriminator network (Discriminator) determines whether the input data is real or generated. The generator network continuously optimizes its generated data to make it difficult for the discriminator network to detect, while the discriminator network also optimizes itself to make its judgments more accurate. The two networks are in an adversarial relationship, hence the name "adversarial network." Currently, in the field of deep learning, GANs can be used to generate high-quality fake ID photos. The generator learns to generate fake images, and the discriminator judges the authenticity of the images. The two networks compete against each other, ultimately resulting in a generator that produces realistic images. The images generated by GANs are of very high quality, making it difficult for the naked eye or simple recognition algorithms to determine their authenticity. Because the features of GAN-generated ID photos vary greatly and the visual effect is very realistic, detecting ID photos tampered with by GANs is very difficult, and currently there is no good method for detecting ID photos tampered with by GAN networks. Summary of the Invention

[0008] The technical problem this invention aims to solve is how to detect and identify ID photos that have been tampered with by GAN.

[0009] To solve the above-mentioned technical problems, according to one aspect of the present invention, a method for identifying tampered ID photo images is provided, comprising the following steps: S1, inputting an ID photo image, wherein the input image is an ID photo to be identified as tampered with; S2, performing pixel tampering analysis on the input ID photo using a neural network, locating the tampered area using a residual structure neural network, predicting based on the classification results on the combined blocks, and then performing a first judgment on whether the image has been tampered with using a threshold-based method; S3, performing a second detection based on spectral signal analysis on the image that was not detected as tampered after the first detection in step S2. The input is fed into the GAN simulator. The GAN simulator determines whether the ID photo has been tampered with by performing spectral separation and artifact revealing. The GAN simulator obtains the RGB3 channel spectrum of the image, performs spectral separation, calculates the logarithmic spectrum and normalizes it to [-1, 1], and performs artifact revealing to determine whether the image has been tampered with. S4, Output the detection results and summarize the pixel tampering analysis and spectral signal tampering analysis. Output the final detection result: If step S2 detects that the photo has been tampered with, or if step S2 does not detect tampering but step S3 detects that the photo has been tampered with, then output the result that the ID photo has been tampered with; if steps S2 and S3 detect that the photo has not been tampered with, then output the result that the ID photo has not been tampered with.

[0010] According to an embodiment of the present invention, in step S3, the GAN simulator may include an encoder and a decoder, wherein the encoder includes a downsampling layer for extracting high-level information from the input image and generating a low-resolution feature vector; the decoder includes an upsampling layer for taking the low-resolution features as input and outputting a high-resolution image.

[0011] According to an embodiment of the present invention, in step S3, the frequency domain is further analyzed in the method for detecting image tampering based on spectrum analysis. Based on the property that the spectrum of a low-resolution image is mainly concentrated in the low-frequency part in the discrete Fourier transform, zeros are inserted in the high-frequency part of the low-resolution image, which is equivalent to copying the spectrum of the original low-resolution image to the high-frequency part. Fourier transform is then performed to finally obtain the high-resolution image spectrum, thereby revealing the artifacts in the image.

[0012] According to an embodiment of the present invention, the Convolutional Neural Network (CNN) architecture in step S2 may include 6 hidden convolutional layers and 2 fully connected layers. The convolutional layers use a kernel size of (3, 3, D), where D is the depth of the filter. Small convolutional kernels are used to facilitate the learning of local features. First, non-overlapping blocks of size (64, 64, 3) or (128, 128, 3) are extracted from the image for use as input to the convolutional neural network. Non-overlapping blocks can accelerate the processing speed.

[0013] According to an embodiment of the present invention, in step S2, the neural network pixel analysis can extract non-overlapping blocks in the image as input to the convolutional neural network, and use the residual network concept with the assistance of the pooling layer to add the output of the second layer to the fifth layer. This cross-layer connection method expands the network's optimization space and improves the gradient backpropagation effect, preserving low-level information while also transmitting high-level semantic knowledge. At this point, the added output is the output feature map of the convolutional block.

[0014] The residual connection mathematical representation used is as follows:

[0015] (1)

[0016] Residual connections preserve low-level detailed features and fuse high-level semantic features, thereby enhancing the network's ability to learn and express the local structure of images.

[0017] Use Focal loss to guide model training.

[0018] (2)

[0019] (3)

[0020] Among them, a tTo modify the weighting factor of the tampered region; to reduce the impact of negative sample weights on the loss, the adjustable parameter γ is set to 5 and a t With a smaller setting, more granular weight adjustment is performed on samples of different categories to locate objects, solve the problem of class imbalance, and locate or distinguish objects and textures in the tampered area, accurately locating the tampered parts in the image;

[0021] Finally, the classification results on the combined blocks are used to classify the original blocks using a convolutional neural network, and a specific method is introduced to post-process the predictions of the convolutional neural network (CNN). Finally, a threshold-based method is used to make a decision. If the tampered part in the image exceeds the threshold, it is classified as modified or forged.

[0022] According to embodiments of the present invention, post-processing can be performed on predictions made by convolutional neural networks (CNNs):

[0023] (4)

[0024] Using the ratio of the number of predicted tampered patches to the total number of patches ensures that images of different sizes obtain a common threshold, and all values ​​are normalized for effective classification.

[0025] According to an embodiment of the present invention, after normalization, the optimal threshold can be searched in the range of 1 to 10 using grid search, and the optimal threshold is found to be 4; a decision is made that if the proportion of tampered blocks contained in the image is greater than the threshold, it is classified as modified, forged or tampered.

[0026] According to a second aspect of the present invention, an apparatus for identifying altered ID photo images is provided, comprising:

[0027] The input module is used for inputting ID photo images, and inputting the ID photo image to be detected and identified as tampered with;

[0028] The neural network analysis module is used to combine neural network pixel analysis to determine whether the ID photo has been tampered with. It performs pixel tampering analysis on the input ID photo, uses residual structure neural network to locate the tampered area, combines the classification results on the block for prediction, and then uses a threshold-based method to determine whether the image has been tampered with.

[0029] The spectrum signal analysis module is used to analyze whether the ID photo has been tampered with based on the spectrum signal. It performs a second detection based on the spectrum signal analysis on images that have not been detected as tampered after the first detection by the neural network analysis module. The images are then input to the GAN simulator. The GAN simulator determines whether the ID photo has been tampered with by performing spectrum separation and artifact revealing. The GAN simulator obtains the RGB3 channel spectrum of the image, performs spectrum separation, performs logarithmic spectrum calculation and normalizes it to [-1, 1], and performs artifact revealing to determine whether the image has been tampered with.

[0030] The output module is used to output the detection results, summarizing the pixel tampering analysis and spectral signal tampering analysis results to output the final detection result: If the neural network analysis module detects tampering, or if the neural network analysis module does not detect tampering but the spectral signal analysis module does, then the output result is "ID photo has been tampered with"; if both the neural network analysis module and the spectral signal analysis module detect no tampering, then the output result is "ID photo has not been tampered with".

[0031] The spectrum signal analysis module includes a GAN simulator comprising an encoder and a decoder. The encoder includes a downsampling layer, which extracts high-level information from the input image and generates a low-resolution feature vector. The decoder includes an upsampling layer, which takes the low-resolution features as input and outputs a high-resolution image. Further analysis in the frequency domain is performed in the spectrum analysis-based image tampering detection method. Based on the property that the spectrum of a low-resolution image in the Discrete Fourier Transform is mainly concentrated in the low-frequency part, zeros are inserted in the high-frequency part of the low-resolution image. This is equivalent to copying the spectrum of the original low-resolution image to the high-frequency part, performing a Fourier Transform, and finally obtaining the high-resolution image spectrum, thereby revealing artifacts in the image.

[0032] In the neural network analysis module, the neural network CNN architecture includes 6 hidden convolutional layers and 2 fully connected layers. The convolutional layers use kernels of size (3, 3, D) and residual connections are used to add the output of the second layer to the output of the fifth layer, thereby detecting whether the ID photo has been tampered with by the GAN network.

[0033] According to a third aspect of the present invention, an electronic device is provided, comprising: a memory, a processor, and an ID photo tampering identification program stored in the memory and executable on the processor, wherein the ID photo tampering identification program, when executed by the processor, implements the steps of the above-described ID photo tampering identification method.

[0034] According to a fourth aspect of the present invention, a computer storage medium is provided, wherein the computer storage medium stores a document photo tampering identification program, which, when executed by a processor, implements the steps of the document photo tampering identification method described above.

[0035] Compared with the prior art, the technical solution provided by the embodiments of the present invention can achieve at least the following beneficial effects:

[0036] 1) This invention provides a method and apparatus for identifying and detecting tampering of ID photos by combining spectral signal analysis and pixel analysis. It targets ID photo facial images forged and modified using GAN networks. After inputting the image, pixel analysis is first used to detect whether the ID photo has been tampered with, followed by spectral signal analysis to detect the tampering. The method utilizes a neural network architecture based on residual network concepts, adjustable Focal loss parameters, CNN prediction post-processing, and a GAN simulator based on spectral signal input to determine whether the ID photo has been tampered with, and then outputs the detection result.

[0037] 2) This invention proposes a pixel-based signal spectrum analysis method to identify GAN-tampered ID photos. By combining neural network pixel analysis with random spectrum signal analysis, it detects whether ID photos have been tampered with by GAN. The proposed GAN simulator based on spectrum input does not require any fake images during training; it only simulates the artifacts generated by GAN and merges them into the real image. This invention can effectively detect whether ID photos have been tampered with by the GAN network, achieving a recognition rate of up to 99% for tampered ID photos. It can be used for identification after the collection of enterprise and financial ID photos, and can be applied in fields such as anti-fraud, achieving a high recognition rate for ID photo tampering.

[0038] 3) The image tampering detection method based on spectrum analysis provided by this invention, from the perspective of signal spectrum analysis, shows that spectrum-based classifiers have better performance than pixel-based classifiers. At the same time, combined with the tampering detection results based on neural network pixel analysis, it can effectively detect whether ID photos have been tampered with by GAN networks, and the detection and recognition rate is higher than other models.

[0039] 4) This invention uses normalization as a preprocessing step to ensure data comparability across all features. Normalization is performed on each mini-batch of data to prevent internal covariate bias during training. Model weights are initialized using the Xavier method, and L1 norm regularization is used to improve model robustness and feature selection ability.

[0040] 5) Although ID photos altered by GAN networks may appear visually very similar to real images, artifacts will occur during the reconstruction process. This invention proposes a GAN simulator based on spectral input. By using real images and reconstructed images to train a classifier, the classifier will focus on artifacts, thus distinguishing GAN-altered ID photos with high quality and efficiency.

[0041] 6) Compared to other models that require careful selection of images of a pair of categories, the method proposed in this invention can use images of any semantic category as input, which greatly simplifies the data collection process and has better recognition performance for single semantic images. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of the present invention and are not intended to limit the present invention.

[0043] Figure 1 This is a flowchart illustrating a method for identifying tampering with ID photo images according to an embodiment of the present invention;

[0044] Figure 2 This is a flowchart illustrating the detection of whether an ID photo has been tampered with using neural network pixel analysis according to an embodiment of the present invention;

[0045] Figure 3 This is a diagram illustrating the processing steps of a convolutional neural network according to an embodiment of the present invention;

[0046] Figure 4 This is a flowchart illustrating the analysis of whether an ID photo has been tampered with based on spectral signals according to an embodiment of the present invention;

[0047] Figure 5 This is a diagram illustrating the architecture of a GAN simulator according to an embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the described embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains. The terms “first,” “second,” and similar terms used in the specification and claims of this patent application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an” or “a” and similar terms do not indicate a limitation of quantity, but rather indicate the presence of at least one.

[0050] Figure 1 This is a flowchart illustrating a method for identifying tampering with ID photo images according to an embodiment of the present invention; Figure 2This is a flowchart illustrating the detection of whether an ID photo has been tampered with using neural network pixel analysis according to an embodiment of the present invention; Figure 3 This is a diagram illustrating the processing steps of a convolutional neural network according to an embodiment of the present invention; Figure 4 This is a flowchart illustrating the analysis of whether an ID photo has been tampered with based on spectral signals according to an embodiment of the present invention.

[0051] like Figures 1 to 4 As shown, the method for identifying tampering with ID photo images includes the following steps:

[0052] S1. Input ID photo image: Input the ID photo image to be detected and identified as tampered with.

[0053] S2. Combine neural network pixel analysis to determine whether the ID photo has been tampered with. Perform pixel tampering analysis on the input ID photo, use residual structure neural network to locate the tampered area, use the classification results on the combined block to make predictions, and then use a threshold-based method to determine whether the image has been tampered with.

[0054] S3. Based on spectral signal analysis, determine whether the ID photo has been tampered with. For images that were not detected as tampered after the first detection in step S2, perform a second detection based on spectral signal analysis and input the data into the GAN simulator. The GAN simulator determines whether the ID photo has been tampered with by performing spectral separation and artifact revealing. The GAN simulator obtains the RGB3 channel spectrum of the image, performs spectral separation, performs logarithmic spectrum calculation and normalizes it to [-1, 1], and performs artifact revealing to determine whether the image has been tampered with.

[0055] S4. Output the detection results and summarize the pixel tampering analysis and spectrum signal tampering analysis. Output the final detection results: If step S2 detects that the photo has been tampered with, or if step S2 does not detect tampering but step S3 detects that the photo has been tampered with, then output the result that the photo has been tampered with; if both steps S2 and S3 detect that the photo has not been tampered with, then output the result that the photo has not been tampered with.

[0056] The image tampering detection method based on spectrum analysis provided by this invention analyzes the signal spectrum from the perspective that the spectrum-based classifier has better performance than the pixel-based classifier. At the same time, combined with the tampering detection results based on neural network pixel analysis, it can effectively detect whether the ID photo has been tampered with by the GAN network, and the detection and recognition rate is higher than other models. Figure 5 This is a diagram illustrating the architecture of a GAN simulator according to an embodiment of the present invention.

[0057] like Figure 5As shown, in step S3, the GAN simulator includes an encoder and a decoder. The encoder includes a downsampling layer, which is used to extract high-level information from the input image and generate a low-resolution feature vector. The decoder includes an upsampling layer, which is used to take the low-resolution features as input and output a high-resolution image.

[0058] Although ID photos altered by GAN networks are visually very similar to real images, artifacts will be generated during the reconstruction process. This invention proposes a GAN simulator based on spectral input. By using real images and reconstructed images to train a classifier, the classifier will focus on artifacts and distinguish ID photos altered by GANs with high quality and efficiency.

[0059] According to one or more embodiments of the present invention, in step S3, the frequency domain is further analyzed in the method for detecting image tampering based on spectrum analysis. Based on the property that the spectrum of a low-resolution image is mainly concentrated in the low-frequency part in the discrete Fourier transform, zeros are inserted in the high-frequency part of the low-resolution image, which is equivalent to copying the spectrum of the original low-resolution image to the high-frequency part. Fourier transform is then performed to finally obtain the high-resolution image spectrum, thereby revealing the artifacts in the image.

[0060] like Figure 3 As shown, the neural network CNN architecture in step S2 includes 6 hidden convolutional layers and 2 fully connected layers. The convolutional layers use kernels of size (3, 3, D), where D is the depth of the filter. Small convolutional kernels are used to facilitate the learning of local features. First, non-overlapping blocks of size (64, 64, 3) or (128, 128, 3) are extracted from the image for use as input to the convolutional neural network. Non-overlapping blocks can accelerate the processing speed.

[0061] According to one or more embodiments of the present invention, in step S2, the neural network pixel analysis extracts non-overlapping blocks in the image as input to the convolutional neural network, and uses the residual network idea to add the output of the second layer to the fifth layer with the assistance of the pooling layer. The cross-layer connection method expands the optimization space of the network and improves the gradient backpropagation effect, which not only preserves the low-level information but also transmits high-level semantic knowledge. At this time, the added output is the output feature map of the convolutional block.

[0062] The residual connection mathematical representation used is as follows:

[0063] (1)

[0064] Residual connections preserve low-level detailed features and fuse high-level semantic features, thereby enhancing the network's ability to learn and represent the local structure of images.

[0065] Use Focal loss to guide model training.

[0066] (2)

[0067] (3)

[0068] Among them, a t To modify the weighting factor of the tampered region; to reduce the impact of negative sample weights on the loss, the adjustable parameter γ is set to 5 and a t With a smaller setting, more granular weight adjustment is performed on samples of different categories to locate objects, solve the problem of class imbalance, and locate or distinguish objects and textures in the tampered area, accurately locating the tampered parts in the image.

[0069] Finally, the classification results on the combined blocks are used to classify the original blocks using a convolutional neural network, and a specific method is introduced to post-process the predictions of the convolutional neural network (CNN). Finally, a threshold-based method is used to make a decision. If the tampered part in the image exceeds the threshold, it is classified as modified or forged.

[0070] According to an embodiment of the present invention, post-processing is performed on the predictions of the convolutional neural network (CNN):

[0071] (4)

[0072] Using the ratio of the number of predicted tampered patches to the total number of patches ensures that images of different sizes obtain a common threshold, and all values ​​are normalized for effective classification.

[0073] According to one or more embodiments of the present invention, after normalization, a grid search is used to search for the optimal threshold in the range of 1 to 10, and the optimal threshold is found to be 4; a decision is made that if the proportion of tampered blocks contained in the image is greater than the threshold, it is classified as modified, forged or tampered.

[0074] This invention uses normalization as a preprocessing step to ensure data comparability across all features. Normalization is performed on each mini-batch of data to prevent internal covariate bias during training. Model weights are initialized using the Xavier method, and L1 norm regularization is used to improve model robustness and feature selection ability.

[0075] According to a second aspect of the present invention, an apparatus for identifying tampering of ID photo images is provided, comprising: an input module, a neural network analysis module, a spectrum signal analysis module, and an output module.

[0076] The input module is used for inputting ID photo images, which are then used to detect whether the ID photo has been tampered with.

[0077] The neural network analysis module is used to combine neural network pixel analysis to determine whether the ID photo has been tampered with. It performs pixel tampering analysis on the input ID photo, uses residual structure neural network to locate the tampered area, combines the classification results on the block for prediction, and then uses a threshold-based method to determine whether the image has been tampered with.

[0078] The spectrum signal analysis module is used to analyze whether the ID photo has been tampered with based on the spectrum signal. For images that were not detected as tampered after the first detection by the neural network analysis module, a second detection based on spectrum signal analysis is performed. The image is then input to the GAN simulator. The GAN simulator determines whether the ID photo has been tampered with by performing spectrum separation and artifact revealing. The GAN simulator obtains the RGB3 channel spectrum of the image, performs spectrum separation, calculates the logarithmic spectrum and normalizes it to [-1, 1], and performs artifact revealing to determine whether the image has been tampered with.

[0079] The output module is used to output the detection results, summarizing the pixel tampering analysis and the spectrum signal tampering analysis results to output the final detection result: if the neural network analysis module detects that the photo has been tampered with, or if the neural network analysis module does not detect tampering but the spectrum signal analysis module does, then the output result is that the photo has been tampered with; if both the neural network analysis module and the spectrum signal analysis module detect that the photo has not been tampered with, then the output result is that the photo has not been tampered with.

[0080] In the spectrum signal analysis module, the GAN simulator includes an encoder and a decoder. The encoder includes a downsampling layer, which extracts high-level information from the input image and generates a low-resolution feature vector. The decoder includes an upsampling layer, which takes the low-resolution features as input and outputs a high-resolution image. Further analysis in the frequency domain is conducted in the spectrum analysis-based image tampering detection method. Based on the property that the spectrum of a low-resolution image is mainly concentrated in the low-frequency part of the Discrete Fourier Transform, zeros are inserted in the high-frequency part of the low-resolution image. This is equivalent to copying the spectrum of the original low-resolution image to the high-frequency part, performing a Fourier Transform, and finally obtaining the high-resolution image spectrum, thereby revealing artifacts in the image.

[0081] In the neural network analysis module, the neural network CNN architecture includes 6 hidden convolutional layers and 2 fully connected layers. The convolutional layers use kernels of size (3, 3, D) and residual connections are used to add the output of the second layer to the output of the fifth layer, thereby detecting whether the ID photo has been tampered with by the GAN network.

[0082] The present invention provides a method and apparatus for identifying and detecting tampering of ID photos by combining spectral signal and pixel analysis. It identifies ID photo faces that have been forged and modified by GAN networks. After inputting the image, the pixel analysis method is first used to detect whether the ID photo has been tampered with, and then the spectral signal analysis method is used to detect the tampering. The method uses a neural network architecture based on residual network ideas, adjustable Focal loss parameters, CNN prediction post-processing method, and a GAN simulator based on spectral signal input to determine whether the ID photo has been tampered with, and then outputs the detection result.

[0083] Compared to other models that require careful selection of a pair of images of different categories, the method proposed in this invention can use images of any semantic category as input, greatly simplifying the data collection process and achieving better recognition results for single semantic images.

[0084] This invention proposes a pixel-based signal spectrum analysis method to identify GAN-tampered ID photos. By combining neural network pixel analysis with random spectrum signal analysis, it detects whether ID photos have been tampered with by GAN. The proposed GAN simulator based on spectrum input does not require any fake images during training; it only simulates the artifacts generated by GAN and merges them into the real image. This invention can effectively detect whether ID photos have been tampered with by GAN networks, achieving a recognition rate of up to 99% for tampered ID photos. It can be used for identification after the collection of enterprise and financial ID photos, and can be applied in fields such as anti-fraud, achieving a high recognition rate for ID photo tampering.

[0085] According to another aspect of the present invention, an apparatus for identifying tampering of ID photo images is provided, comprising: a memory, a processor, and an ID photo image tampering identification program stored in the memory and executable on the processor, wherein when the ID photo image tampering identification program is executed by the processor, it implements the steps of the above-described ID photo image tampering identification method.

[0086] The present invention also provides a computer storage medium.

[0087] The computer storage medium stores a program for identifying tampering with ID photos. When the processor executes the program, it implements the steps of the aforementioned method for identifying tampering with ID photos.

[0088] The method implemented when the ID photo tampering recognition program running on the processor is executed can be referred to in various embodiments of the ID photo tampering recognition method of the present invention, and will not be repeated here.

[0089] The present invention also provides a computer program product.

[0090] The computer program product of the present invention includes an ID photo tampering recognition program, which, when executed by a processor, implements the steps of the ID photo tampering recognition method as described above.

[0091] The method implemented when the ID photo tampering recognition program running on the processor is executed can be referred to in various embodiments of the ID photo tampering recognition method of the present invention, and will not be repeated here.

[0092] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0093] The above description is merely an exemplary embodiment of the present invention and is not intended to limit the scope of protection of the present invention, which is determined by the appended claims.

Claims

1. A method for identifying tampering with ID photo images, comprising the following steps: S1. Input ID photo image: Input the ID photo image to be detected and identified as tampered with; S2. Combine neural network pixel analysis to determine whether the ID photo has been tampered with. Perform pixel tampering analysis on the input ID photo, use residual structure neural network to locate the tampered area, use the classification results on the combined block to make predictions, and then use a threshold-based method to determine whether the image has been tampered with. S3. Based on spectral signal analysis, determine whether the ID photo has been tampered with. For images that were not detected as tampered after the first detection in step S2, perform a second detection based on spectral signal analysis and input the data to the GAN simulator. The GAN simulator determines whether the ID photo has been tampered with by performing spectral separation and artifact revealing. The GAN simulator obtains the RGB3 channel spectrum of the image, performs spectral separation, performs logarithmic spectrum calculation and normalizes it to [-1, 1], and performs artifact revealing to determine whether the image has been tampered with. S4. Output the detection results and summarize the pixel tampering analysis and spectrum signal tampering analysis. Output the final detection result: If step S2 detects tampering, or if step S2 does not detect tampering but step S3 detects tampering, then output the result that the ID photo has been tampered with; if both steps S2 and S3 detect no tampering, then output the result that the ID photo has not been tampered with. in, In step S3, the GAN simulator includes an encoder and a decoder. The encoder includes a downsampling layer, which is used to extract high-level information from the input image and generate a low-resolution feature vector. The decoder includes an upsampling layer, which takes the low-resolution features as input and outputs a high-resolution image. In step S3, the frequency domain is further analyzed in the spectrum analysis-based image tampering detection method. Based on the property that the spectrum of low-resolution images in discrete Fourier transform is mainly concentrated in the low-frequency part, zeros are inserted in the high-frequency part of the low-resolution image. This is equivalent to copying the spectrum of the original low-resolution image to the high-frequency part, performing Fourier transform, and finally obtaining the high-resolution image spectrum, thereby revealing the artifacts in the image.

2. The method for identifying altered ID photo images as described in claim 1, wherein, The neural network CNN architecture in step S2 includes 6 hidden convolutional layers and 2 fully connected layers. The convolutional layers use kernels of size (3, 3, D), where D is the depth of the filter. Small convolutional kernels are used to facilitate the learning of local features. First, non-overlapping blocks of size (64, 64, 3) or (128, 128, 3) are extracted from the image for use as input to the convolutional neural network. Non-overlapping blocks can accelerate the processing speed.

3. The method for identifying altered ID photo images as described in claim 2, wherein, In step S2, the neural network pixel analysis extracts non-overlapping blocks from the image as input to the convolutional neural network. Using a residual network approach with pooling layer assistance, the output of the second layer is added to the fifth layer. This cross-layer connection expands the network's optimization space and improves gradient backpropagation, preserving low-level information while transmitting high-level semantic knowledge. The added output at this point becomes the output feature map of the convolutional block. The residual connection mathematical representation used is as follows: （1） Residual connections preserve low-level detailed features and fuse high-level semantic features, thereby enhancing the network's ability to learn and express the local structure of images. Use Focal loss to guide model training. （2） （3） Among them, a t To modify the weighting factor of the tampered region; to reduce the impact of negative sample weights on the loss, the adjustable parameter γ is set to 5 and a t With a smaller setting, more granular weight adjustment is performed on samples of different categories to locate objects, solve the problem of class imbalance, and locate or distinguish objects and textures in the tampered area, accurately locating the tampered parts in the image; Finally, the classification results on the combined blocks are used to classify the original blocks using a convolutional neural network, and a specific method is introduced to post-process the convolutional neural network predictions; finally, a threshold-based method is used to make a decision, and if the tampered part in the image exceeds the threshold, it is classified as modified or forged.

4. The method for identifying altered ID photo images as described in claim 3, wherein, Post-processing of convolutional neural network predictions: （4） Using the ratio of the number of predicted tampered patches to the total number of patches ensures that images of different sizes obtain a common threshold, and all values ​​are normalized for effective classification.

5. The method for identifying altered ID photo images as described in claim 4, wherein, After normalization, the optimal threshold is searched in the range of 1 to 10 using gridsearch, and the optimal threshold is found to be 4. A decision is made that if the proportion of tampered blocks in an image exceeds the threshold, it is classified as modified, forged, or tampered.

6. A device for identifying altered ID photo images, comprising: The input module is used for inputting ID photo images, and inputting the ID photo image to be detected and identified as tampered with; The neural network analysis module is used to combine neural network pixel analysis to determine whether the ID photo has been tampered with. It performs pixel tampering analysis on the input ID photo, uses residual structure neural network to locate the tampered area, combines the classification results on the block for prediction, and then uses a threshold-based method to determine whether the image has been tampered with. The spectrum signal analysis module is used to analyze whether the ID photo has been tampered with based on the spectrum signal. It performs a second detection based on the spectrum signal analysis on images that have not been detected as tampered after the first detection by the neural network analysis module. The images are then input to the GAN simulator. The GAN simulator determines whether the ID photo has been tampered with by performing spectrum separation and artifact revealing. The GAN simulator obtains the RGB3 channel spectrum of the image, performs spectrum separation, performs logarithmic spectrum calculation and normalizes it to [-1, 1], and performs artifact revealing to determine whether the image has been tampered with. The output module is used to output the detection results, summarize the pixel tampering analysis and the spectrum signal tampering analysis output results, and output the final detection result: if the neural network analysis module detects that it has been tampered, or if the neural network analysis module does not detect that it has been tampered but the spectrum signal analysis module does detect that it has been tampered, then the output result is that the ID photo has been tampered with. If the neural network analysis module and the spectrum signal analysis module detect that the photo has not been tampered with, then the output will show that the ID photo has not been tampered with. In the spectrum signal analysis module, the GAN simulator includes an encoder and a decoder. The encoder includes a downsampling layer, which extracts high-level information from the input image and generates a low-resolution feature vector. The decoder includes an upsampling layer, which takes the low-resolution features as input and outputs a high-resolution image. Further analysis in the frequency domain is performed in the spectrum analysis-based image tampering detection method. Based on the property that the spectrum of a low-resolution image is mainly concentrated in the low-frequency part of the discrete Fourier transform, zeros are inserted in the high-frequency part of the low-resolution image. This is equivalent to copying the spectrum of the original low-resolution image to the high-frequency part, performing a Fourier transform, and finally obtaining the high-resolution image spectrum, thereby revealing artifacts in the image. In the neural network analysis module, the neural network CNN architecture includes 6 hidden convolutional layers and 2 fully connected layers. The convolutional layers use kernels of size (3, 3, D) and residual connections are used to add the output of the second layer to the output of the fifth layer, thereby detecting whether the ID photo has been tampered with by the GAN network.

7. An electronic device, comprising: The document includes a memory, a processor, and a document photo tampering identification program stored in the memory and executable on the processor, wherein the document photo tampering identification program, when executed by the processor, implements the steps of the document photo tampering identification method as described in any one of claims 1 to 5.

8. A computer storage medium, wherein, The computer storage medium stores a program for identifying tampering with ID photos. When the processor executes the program, it implements the steps of the method for identifying tampering with ID photos as described in any one of claims 1 to 5.

Images