Tamper detection model training method, tamper detection method and tamper detection device

The method enhances document tampering detection accuracy and robustness by training neural networks with samples under varying lighting conditions and simulating tampering scenarios, improving the model's ability to detect tampering accurately across different lighting environments.

CN119068320APending Publication Date: 2024-12-03ANT BLOCKCHAIN TECHNOLOGY (SHANGHAI) CO LTD

Patent Information

Application Number
CN202410940563.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-12
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing document tampering detection methods based on deep learning neural networks lack accuracy and robustness, particularly under varying lighting conditions, which compromises the security of online document verification systems.

Method used

A tampering detection model training method that collects samples under different lighting conditions, trains a neural network model using these samples, and applies transfer learning to enhance the model's generalization and robustness by simulating tampering scenarios, including physical and digital manipulations.

Benefits of technology

The method improves the accuracy and robustness of document tampering detection by enhancing the model's ability to identify tampering under diverse lighting conditions and unknown tampering types, ensuring secure online document verification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068320A_ABST
    Figure CN119068320A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a tampering detection model training method, a tampering detection method and a tampering detection device, and the method comprises the steps: firstly obtaining multiple frames of sample images of a target certificate in different illumination environments, training a preset first neural network model through the multiple frames of sample images, and obtaining a certificate pre-training model; and then tampering the multiple frames of sample images to obtain multiple frames of tampered images. And further migrating the model parameters of the certificate pre-training model to a preset second neural network model, and training the second neural network model by using the multiple frames of tampering images to obtain a tampering detection model. And finally, accurately identifying whether the certificate to be detected is tampered or not by utilizing the trained tampering detection model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to computer technology, and in particular to a tampering detection model training method, a tampering detection method and a tampering detection device. Background Art

[0002] In recent years, with the rapid development of the Internet, people can handle various affairs online and verify relevant documents online.

[0003] Document tampering detection is a key detection link in the online document verification process to ensure document security and prevent personal information leakage. In related technologies, document tampering detection can be implemented based on a deep learning neural network model. The accuracy of document tampering detection is directly related to the security of online document verification. In order to achieve more secure online document verification, it is imperative to improve the accuracy of document tampering detection. Summary of the invention

[0004] The embodiment of this specification provides a tampering detection model training method, which enhances the generalization ability of the model by collecting samples under variable lighting conditions, and then uses the tampering clues in the document image to train the model, effectively improving the accuracy and robustness of document tampering detection. The method includes:

[0005] Acquire multiple frames of sample images of the target certificate under different lighting environments;

[0006] Using the multiple frames of sample images to train a preset first neural network model to obtain a document pre-training model;

[0007] Performing tampering processing on the multiple frames of sample images to obtain multiple frames of tampered images;

[0008] The model parameters of the document pre-training model are migrated to a preset second neural network model, and the second neural network model is trained using the multiple frames of tampered images to obtain a tampering detection model.

[0009] Further, in some embodiments, the second neural network includes a feature extraction network and a target detection network;

[0010] The method of training the second neural network model using the multiple frames of tampered images to obtain a tampering detection model includes:

[0011] Inputting the multiple frames of tampered images into the feature extraction network to obtain multi-scale intermediate features;

[0012] The intermediate features of multiple scales are fused to obtain fused features, and the target detection network is used to perform target segmentation and classification prediction on the fused features to obtain a target prediction result;

[0013] The feature extraction network and the target detection network are iteratively trained based on the target prediction result until an iteration termination condition is met, thereby obtaining the tampering detection model.

[0014] Further, in some embodiments, the feature extraction network includes at least a feature embedding network and a feature extraction network, and the intermediate feature includes a first intermediate feature and a second intermediate feature;

[0015] The step of inputting the multi-frame tampered images into the feature extraction network to obtain multi-scale intermediate features includes:

[0016] Segmenting each of the tampered images into at least one image block through the feature embedding network, and embedding each of the image blocks to obtain a plurality of embedded features;

[0017] Each of the embedded features and the position code corresponding to each of the embedded features are input into the feature extraction network to obtain multiple layers of first intermediate features of the same scale, and convolution processing is performed on the target intermediate features in the multiple layers of first intermediate features to obtain multi-scale second intermediate features.

[0018] Further, in some embodiments, the first neural network model includes an encoder and a decoder;

[0019] The step of training a preset first neural network model using the multiple frames of sample images includes:

[0020] Performing random mask processing on the multiple frames of sample images to obtain multiple frames of input images;

[0021] Inputting unmasked image blocks in the multiple frames of input images into the encoder to obtain sample image features;

[0022] The decoder is used to splice and decode the sample image features and the mask image blocks in the multiple frames of input images to obtain a reconstructed image;

[0023] A loss value between the sample image and the reconstructed image is calculated in pixel space using a preset loss function, and model parameters of the encoder and model parameters of the decoder are adjusted according to the loss value.

[0024] Furthermore, in some embodiments, the multiple frames of sample images of the target document in different lighting environments include a first reference frame image of the target document in a normal lighting environment and a first target frame image of the target document in a flash lighting environment.

[0025] Furthermore, in some embodiments, after acquiring multiple frames of sample images of the target document under different lighting environments, the method further includes:

[0026] The first reference frame image and the first target frame image are aligned.

[0027] Further, in some implementations, aligning the first reference frame image with the first target frame image includes:

[0028] Performing corner point detection on the first reference frame image and the first target frame image respectively, to determine reference corner points in the first reference frame image and target corner points in the first target frame image;

[0029] A transformation matrix is ​​calculated according to the reference corner point and the target corner point, and the first target frame image is perspective transformed using the transformation matrix to align the reference corner point with the target corner point.

[0030] Furthermore, in some implementations, tampering the multiple frames of sample images to obtain multiple frames of tampered images includes:

[0031] According to a preset tampering type, the aligned first reference frame image and the first target frame image are tampered with to obtain the multi-frame tampered image.

[0032] Furthermore, in some implementations, the preset tampering type includes at least one or more of rectangular tampering, text overlay, and foreign object occlusion.

[0033] The present specification also provides a tampering detection method, which includes:

[0034] Acquire multiple frames of document images of the document to be detected under different lighting environments;

[0035] Inputting the multiple frames of document images into a trained tampering detection model to obtain tampering detection results;

[0036] Wherein, the tampering detection model is trained by any one of the tampering detection model training methods.

[0037] Furthermore, in some embodiments, after acquiring multiple frames of document images of the document to be detected under different lighting environments, the method further includes:

[0038] The multiple frames of document images are aligned to input the aligned multiple frames of document images into a trained tampering detection model.

[0039] Furthermore, in some embodiments, the multiple frames of document images of the document to be detected in different lighting environments include a second reference frame image of the document to be detected in a normal lighting environment and a second target frame image of the document to be detected in a flash lighting environment.

[0040] Furthermore, in some implementations, the tampering detection result includes tampering location information and tampering type corresponding to the document to be detected.

[0041] The embodiment of this specification also proposes a tamper detection model training device, the device comprising:

[0042] A sample image acquisition module, used to acquire multiple frames of sample images of the target certificate under different lighting environments;

[0043] A first model training module, used to train a preset first neural network model using the multiple frames of sample images to obtain a document pre-training model;

[0044] A multi-frame image tampering module, used for tampering the multi-frame sample image to obtain a multi-frame tampered image;

[0045] The second model training module is used to migrate the model parameters of the document pre-training model to a preset second neural network model, and use the multiple frames of tampered images to train the second neural network model to obtain a tampering detection model.

[0046] The present specification also provides a tamper detection device, the device comprising:

[0047] The document image acquisition module is used to acquire multiple frames of document images of the document to be detected under different lighting environments;

[0048] An image tampering detection module, used for inputting the multiple frames of document images into a trained tampering detection model to obtain tampering detection results;

[0049] Wherein, the tampering detection model is trained by any one of the tampering detection model training methods.

[0050] The embodiments of the present specification also provide a storage medium, wherein the storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the steps of the above method.

[0051] An embodiment of the present specification also provides an electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the above method.

[0052] The embodiments of the present specification also provide a computer program product, wherein the computer program product stores at least one instruction, and the at least one instruction is suitable for being loaded by a processor and executing the above method steps.

[0053] In the embodiment of this specification, firstly, multiple sample images of the target certificate under different lighting environments are obtained, and the preset first neural network model is trained using the multiple sample images to obtain a certificate pre-training model. Then, the multiple sample images are tampered to obtain multiple tampered images. The model parameters of the certificate pre-training model are further transferred to the preset second neural network model, and the second neural network model is trained using the multiple tampered images to obtain a tampering detection model. Finally, the trained tampering detection model is used to accurately identify whether the certificate to be detected has been tampered with. By adopting the method proposed in the embodiments of this specification, on the one hand, by collecting multiple frames of sample images under different lighting environments for training, the differences between real and forged certificates under different lighting conditions can be explored, thereby enhancing the generalization ability of the model and the robustness of the model to lighting changes, ensuring that certificate information can be accurately identified under various lighting conditions; on the other hand, by tampering with multiple frames of sample images to generate tampered images, the model can learn the tampered information in the tampered images, thereby improving the model's sensitivity and recognition accuracy to the tampered information; on the other hand, the use of the certificate pre-training model and the subsequent transfer learning strategy can effectively reuse the learned feature representations, accelerate model convergence, and at the same time enable the model to better understand and capture key information in the certificate image, thereby improving the detection capability of unknown tampering types. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic diagram of a system architecture for applying the embodiments of this specification;

[0055] Figure 2 A flowchart of a tamper detection model training method provided in an embodiment of this specification;

[0056] Figure 3 A schematic diagram of a process for aligning multiple sample frames provided in an embodiment of this specification;

[0057] Figure 4 A flowchart of another tamper detection model training method provided in an embodiment of this specification;

[0058] Figure 5 A flowchart of another tamper detection model training method provided in an embodiment of this specification;

[0059] Figure 6 A schematic diagram of a process for extracting a multi-scale second intermediate feature provided in an embodiment of this specification;

[0060] Figure 7 A flowchart of a tamper detection method provided in an embodiment of this specification;

[0061] Figure 8A schematic diagram of the structure of a tamper detection model provided in an embodiment of this specification;

[0062] Fig. 9 A schematic diagram of the structure of a tamper detection model training device provided in an embodiment of this specification;

[0063] Fig.10 A schematic diagram of the structure of a tamper detection device provided in an embodiment of this specification;

[0064] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0066] Figure 1 A schematic diagram of an architecture to which the embodiments of this specification can be applied is shown.

[0067] like Figure 1 As shown, the system architecture 100 may include one or more of terminal devices such as a smart phone 101, a portable computer 102, a desktop computer 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the terminal device and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The terminal device may be various electronic devices with data processing functions, and the electronic device may have a display screen, which is used for input sample images, tampered images, document images, and tampering detection results, etc.

[0068] It is understandable that the terminal device can also be various electronic devices with image acquisition functions. For example, the sample images used in the tampering detection model training process can be obtained by real-time acquisition by electronic devices. Of course, the sample images can also be pre-acquired by other image acquisition devices and stored in the terminal device or server 105, and then called when training the model. This specification does not limit this. After the document tampering detection is completed, the tampering detection results can be directly displayed to the detection personnel through the display screen on the terminal device.

[0069] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

[0070] See also Figure 2 , which is a flowchart of a tamper detection model training method provided in an embodiment of this specification. In an embodiment of this specification, the tamper detection model training method is applied to a tamper detection model training device or an electronic device equipped with a tamper detection model training device. Figure 2 The process shown is described in detail, and the tampering detection model training method may specifically include the following steps:

[0071] S202, obtaining multiple frames of sample images of the target certificate under different lighting environments;

[0072] In one or more embodiments of this specification, the target certificate refers to the certificate used for training the tamper detection model. For example, it can be a personal ID card, passport, driver's license, or other certificate containing personal information. The embodiments of this specification do not limit the type of target certificate, and any certificate that has a tamper detection requirement and is suitable for training the tamper detection model can be used.

[0073] Different lighting environments may include normal lighting environments and flash lighting environments. Among them, normal lighting environments refer to lighting generated by natural light or a stable indoor light source. Of course, other environments such as dynamic lighting and colored lighting may also be included, which are not limited in the embodiments of this specification. Normal lighting environments generally refer to environments under natural light or a stable indoor light source, where the light is relatively uniform and soft. Flash lighting environments refer to instantaneous strong light environments generated when a flash is used, where the light is more concentrated and intense. Experiments have shown that there are significant differences in the images obtained under these two lighting environments. For example, in a normal lighting environment, the anti-counterfeiting marks on the certificate may present a uniform reflective effect; while in a flash lighting environment, these anti-counterfeiting marks may produce special optical reactions, such as local strong reflections or the appearance of specific patterns. This difference is an important basis for detecting the authenticity of certificates.

[0074] In the tampering detection scenario of document anti-counterfeiting, the document information is usually tampered by sticking a note, and the note is not reflective under the flash light environment, which makes the forged document and the real document have obvious differences. Therefore, this specification takes the normal light environment and the flash light environment as examples for explanation. Accordingly, the frame image of the target document in the normal light environment is collected and recorded as the first reference frame image, and the frame image of the document in the flash light environment is collected and recorded as the first target frame image.

[0075] It is understandable that the target certificate may be any number of certificates, and correspondingly, the multi-frame sample images used for training the tampering detection model may be a set of images containing any number, which is not limited in the present disclosure.

[0076] S204, training a preset first neural network model using the multiple frames of sample images to obtain a document pre-training model;

[0077] In one or more embodiments of the present specification, the multi-frame sample images are a double-frame image dataset including a first reference frame image and a first target frame image, ensuring that different vertical scenes can be covered and that there are obvious dynamic changes between the two frames, which helps the model to learn a wide range of lighting conditions and scene changes and improve the generalization ability of the model.

[0078] Before inputting multiple frames of sample images into the first neural network model for training, the multiple frames of sample images can be standardized, such as scaling, cropping, normalization, etc., to ensure that the size and format of the sample images are suitable for the input requirements of the model to improve the training efficiency of the model.

[0079] Among them, the preset first neural network model can be a convolutional neural network model, or a deep learning network model, or a combination model of a convolutional neural network model and a deep learning network model. The first neural network model can process a double-frame image sequence and extract dynamic features from the double-frame image sequence. The present disclosure does not limit this.

[0080] During the model training process, according to the requirements of the target task, a corresponding loss function is defined to measure the reconstruction error, such as the mean square error loss function or the mean absolute error loss function. The first reference frame image and the first target frame image in the dual-frame image dataset are input into the first neural network model to obtain a reconstructed image. The loss function is used to quantify the difference between the reconstructed image and the original image, and optimize the first neural network model by minimizing the reconstruction error, guiding the model to learn the dynamic features of the dual-frame image sequence. After the training is completed, the document pre-training model is obtained.

[0081] Through pre-training with a large amount of dual-frame image data, the model can learn the influence of lighting changes on images and the dynamic change patterns in the scene. The pre-training process can not only improve the initialization performance of the model, but also accelerate the fine-tuning process of subsequent specific tasks.

[0082] S206, performing tampering processing on the multiple frames of sample images to obtain multiple frames of tampered images;

[0083] In order to train the model to recognize and detect image tampering, multiple sample images can be tampered in the same way to obtain multiple tampered images. Tampered images can simulate various image manipulation behaviors that may occur in reality, so as to help the model learn to distinguish between real images and tampered images.

[0084] Optionally, the tampering process includes physical tampering, digital tampering, synthesis and splicing, etc. Among them, physical tampering includes scratching, smearing and reprinting, etc., digital tampering includes using image editing software to modify images, face fusion, digital watermark removal, etc., synthesis and splicing include image splicing, background replacement and other methods.

[0085] In this embodiment, the tampering processing performed on multiple frames of sample images mainly includes the following types:

[0086] a) Local replacement: Randomly select an area of ​​the image (5% to 20% of the original image) and replace it with the corresponding area of ​​another document image. This tampering simulates the behavior of criminals trying to change the document information.

[0087] b) Text overlay: Text is added to the image at random locations with a font size of 12 to 24 pixels and a color that contrasts with the background. This tampering simulates the addition of additional information to the document.

[0088] c) Image stitching: stitching two images of different documents together, blurring the borders to hide the traces. This tampering simulates the complexity of making fake documents.

[0089] d) Digital manipulation: Digital manipulation of the image, such as adjusting brightness, contrast, or applying filter effects. This type of tampering simulates an attempt to change the overall appearance of the document.

[0090] Each tampering type is applied to images under both normal lighting and flash lighting to ensure that the model can identify various tampering under different lighting conditions.

[0091] It should be noted that in the embodiments of this specification, before the aligned multi-frame sample images are tampered with, the multi-frame sample images need to be aligned in advance. Image frame alignment can ensure the consistency of target positions between different frames and improve the accuracy and stability of the algorithm.

[0092] S208, migrating the model parameters of the document pre-training model to a preset second neural network model, and using the multiple frames of tampered images to train the second neural network model to obtain a tampering detection model.

[0093] The second neural network model can be a network model for implementing classification and target detection. The trained second neural network model, i.e., the tampering detection model, can detect whether the certificate has been tampered with, as well as the corresponding tampering location information and tampering type. It should be noted that the structure of the second neural network model can receive multiple frames of input.

[0094] Before inputting multiple frames of tampered images into the second neural network model, transfer learning is required. Specifically, the model parameters of the certificate pre-training model are transferred to the second neural network model, that is, the model parameters of the certificate pre-training model are used as the initial parameters of the second neural network model. The transfer learning strategy can effectively reuse the feature representations learned by the certificate pre-training model, accelerate the convergence of the second neural network model, and enable the second neural network model to better understand and capture the key information in the certificate image, and improve the detection ability of unknown tampering types.

[0095] It should be noted that the ID card pre-training model and the second neural network model have a high degree of compatibility in model architecture, so that the model parameters in the two can correspond to each other, thereby realizing the migration of model parameters. For example, the weights of the ID card pre-training model are loaded into the corresponding layers of the second neural network model, especially the feature extraction network of the second neural network model will directly inherit the weights learned by the ID card pre-training model during the pre-training process, rather than randomly initialized.

[0096] After the second neural network model is loaded with the model parameters of the document pre-training model, it is fine-tuned on a specific target detection dataset. At this stage, the second neural network model adjusts the model parameters according to the new data and task objectives to optimize the performance of target detection.

[0097] In this embodiment, by capturing the difference information of the tampered area in the normal frame document image and the flash frame document image, the accuracy of document tampering detection can be improved.

[0098] During training, the following strategies can be used to improve the performance of the model:

[0099] a) Data enhancement: In addition to tampering, we also rotated, flipped, and scaled the training data to increase data diversity.

[0100] b) Learning rate scheduling: The cosine annealing learning rate scheduling strategy is used, with an initial learning rate of 0.001 and an adjustment every 50 epochs.

[0101] c) Loss function: A weighted multi-task loss function is used, including reconstruction loss (L1 loss), classification loss (cross entropy loss) and feature consistency loss (based on features extracted by VGG16). The loss function form is as follows:

[0102] L_total=α*L_recon+β*L_class+γ*L_feature

[0103] Among them, α, β, and γ are 0.5, 1.0, and 0.1 respectively.

[0104] d) Early stopping strategy: When the loss on the validation set does not decrease for 10 consecutive epochs, stop training to prevent overfitting.

[0105] e) Model Ensemble: The final tampering detection model is an ensemble of five models trained under different random seeds, using soft voting for prediction.

[0106] In one or more embodiments of the present specification, after executing step S202 to obtain multiple frames of sample images of the target document under different lighting environments, the multiple frames of sample images need to be aligned so that the aligned multiple frames of sample images can be tampered with, thereby ensuring the consistency of the tampering operation.

[0107] For example, the first reference frame image and the first target frame image may be aligned, such as strictly aligning the image corners of the two frames of image. Figure 3 , is a schematic diagram of a process for aligning multiple frame sample images provided in an embodiment of this specification.

[0108] S302, performing corner point detection on the first reference frame image and the first target frame image respectively, to determine reference corner points in the first reference frame image and target corner points in the first target frame image;

[0109] The embodiments of this specification do not limit the number of reference corner points and target corner points. For example, the reference corner points and the target corner points each include 4 corner points.

[0110] Before corner detection, the image needs to be preprocessed to reduce noise and improve the accuracy of corner detection. Preprocessing includes grayscale, smoothing filtering, etc. Grayscale processing can simplify subsequent calculations, and smoothing filtering can reduce image noise and help find corners more accurately.

[0111] Methods for performing corner detection on the first reference frame image and the first target frame image include Harris corner detection, FAST (Features From Accelerated Segment Test) corner detection, ORB (Oriented FAST and Rotated BRIEF) feature detection, etc.

[0112] Optionally, Harris corner detection is used to determine the reference corner points in the first reference frame image as an example. Specifically, the gradient of the image in the x and y directions is calculated at each pixel point of the first reference frame image. Based on the calculated gradient, a 2×2 structure matrix M is constructed, which reflects the direction and magnitude of the intensity change in the neighborhood around each pixel point. The corner response function R is calculated using the structure matrix M, and points with higher R values ​​are used as candidate corner points. Then, local maximum points are searched in the corner response function R to eliminate duplicate corner points and retain the strongest corner points. Finally, an R threshold is set to determine the final reference corner point.

[0113] S304, calculating a transformation matrix according to the reference corner point and the target corner point, and performing perspective transformation on the first target frame image using the transformation matrix to align the reference corner point with the target corner point.

[0114] After determining the reference corner points and the target corner points, a transformation matrix is ​​determined according to the reference corner points and the target corner points, and the transformation matrix can describe the geometric transformation from the first reference frame image to the first target frame image. Optionally, the transformation matrix is ​​a 3x3 matrix, wherein the matrix elements can be calculated by solving a linear equation system or using functions in a library such as OpenCV (computer vision library).

[0115] The first target frame image is perspective transformed using a transformation matrix, such as applying the perspective transformation matrix to each pixel in the first target frame image, so that the target corner point in the first target frame image is aligned with the reference corner point in the first reference frame image.

[0116] In one or more embodiments of the present specification, after aligning multiple frames of sample images, when executing step S206, specifically, the aligned first reference frame image and the first target frame image can be tampered with according to a preset tampering type to obtain multiple frames of tampered images.

[0117] Optionally, physical tampering handwriting generation is simulated by digital generation, and the corresponding tampering types include at least one or more of rectangular tampering, text overlay and foreign object occlusion. Among them, rectangular tampering refers to the use of randomly generated rectangles to cover the sample image area, and the position, size, color and other parameters of the rectangle can be pre-specified to simulate different occlusion situations. Text overlay refers to superimposing some randomly generated text on the sample image area to cover the original image information, such as using different fonts, colors, transparency and other parameters to simulate different types of occlusion effects. Foreign object occlusion refers to randomly cutting foreign objects from other images and sticking them to the same areas of the two sample images. It can be understood that when the aligned first reference frame image and the first target frame image are tampered with, it is necessary to ensure that the tampered areas and attack contents of the two sample images are exactly the same.

[0118] In the embodiments of the present specification, by tampering with multiple frames of sample images to generate tampered images, the model can subsequently learn the tampered information in the tampered images, thereby improving the model's sensitivity and recognition accuracy to the tampered information.

[0119] See also Figure 4 , is a flow chart of another tamper detection model training method provided in an embodiment of this specification. The first neural network model may include an encoder and a decoder, and the encoder and the decoder are asymmetric. Figure 4 As shown, the method comprises the following steps:

[0120] S402, obtaining multiple frames of sample images of the target certificate under different lighting environments;

[0121] For step S402, please refer to the detailed description of step S202 in the first embodiment of this specification, which will not be repeated here.

[0122] S404, performing random mask processing on the multiple frames of sample images to obtain multiple frames of input images;

[0123] Taking the first reference frame image in multiple sample frames as an example, the first reference frame image can be divided into non-overlapping image blocks of size 16×16, and a part of the image blocks are randomly masked with a higher mask ratio (such as 75%). At this time, there are two types of image blocks, namely masked image blocks and unmasked image blocks. By simulating the situation of missing information in the image, the model can learn how to fill in the missing parts and improve the model performance.

[0124] Moreover, each image block is converted into a high-dimensional feature through a block embedding operation, and an input image corresponding to the first reference frame image is obtained. Similarly, an input image corresponding to each frame of sample image can be obtained.

[0125] S406, inputting the unmasked image blocks in the multiple frames of input images into the encoder to obtain sample image features;

[0126] After the masking operation, the unmasked image blocks in each frame of the input image are sent to the encoder for feature extraction to obtain sample image features. For example, the encoder can be composed of a convolutional neural network, which is not limited in the embodiments of this specification.

[0127] S408, splicing and decoding the sample image features and the mask image blocks in the multiple frames of input images through the decoder to obtain a reconstructed image;

[0128] Next, the sample image features extracted from the encoder are concatenated with the mask image blocks in the corresponding frame input image to obtain a fused feature with the same size as the original input image. Finally, a lightweight decoder is used to reconstruct the original image based on the fused feature, such as decoding the fused feature to obtain a reconstructed image.

[0129] S410, calculating a loss value between the sample image and the reconstructed image in a pixel space using a preset loss function, and adjusting a model parameter of the encoder and a model parameter of the decoder according to the loss value;

[0130] The reconstructed image is compared with the original unmasked sample image, and the difference between the two is measured by a preset loss function (such as a mean square error loss function, a cross entropy loss function, etc.). In this illustrated embodiment, the goal of the loss function is to minimize the pixel-level difference between the reconstructed image and the sample image. Based on the calculated loss value, the model parameters of the encoder and decoder are updated using a back propagation algorithm so that the model gradually learns to more accurately predict and reconstruct the masked image portion.

[0131] S412, performing tampering processing on the multiple frames of sample images to obtain multiple frames of tampered images;

[0132] For step S412, please refer to the detailed description of step S206 in the first embodiment of this specification, which will not be repeated here.

[0133] S414, migrating the model parameters of the document pre-training model to a preset second neural network model, and using the multiple frames of tampered images to train the second neural network model to obtain a tampering detection model.

[0134] For step S414, please refer to the detailed description of step S208 in another embodiment of this specification, which will not be repeated here.

[0135] It should be noted that, after executing step S402 to obtain multiple frames of sample images of the target certificate under different lighting environments, the following steps are also required: Figure 3 The steps S302 and S304 are shown to align the multiple frames of sample images, so as to facilitate the tampering process (ie, step S412) of the aligned multiple frames of sample images, thereby ensuring the consistency of the tampering operation.

[0136] As mentioned above, the first neural network model in this embodiment adopts an encoder-decoder structure. The encoder part can use an improved ResNet-50 architecture, for example, including 5 convolution blocks, each of which contains multiple residual units. This design helps to extract deep features of the image while alleviating the gradient vanishing problem. The decoder part uses a stack of transposed convolution layers, specifically including 4 transposed convolution layers, each followed by BatchNormalization and ReLU activation functions. This design can effectively reconstruct the encoded features to the original image size. Between the encoder and the decoder, an attention mechanism module can also be added to highlight important features and suppress irrelevant information. This attention mechanism module can be set to include a self-attention layer and a cross-attention layer, which helps the model better understand the contextual relationship of the image content.

[0137] In this embodiment, the document pre-training model shows greater robustness in handling illumination changes and can effectively identify and compensate for image features under different illumination conditions, which is particularly important for identity authentication in eKYC (electronic identity authentication) scenarios, because in eKYC scenarios it is usually necessary to accurately identify individuals under various illumination conditions. Importantly, pre-trained models are more easily adapted to new tasks quickly with small amounts of data, such as specific types of eKYC verification, reducing the large amount of time and resources required to train models from scratch. In summary, the document pre-training model obtained by pre-training on dual-frame data of the eKYC scenario can significantly improve the performance in handling dynamic illumination changes, scene analysis, and image reconstruction, thereby achieving more efficient and accurate identity authentication and security monitoring in the eKYC scenario.

[0138] See also Figure 5 , is a flow chart of another tamper detection model training method provided in an embodiment of this specification. The second neural network model may include a feature extraction network and a target detection network. Figure 5 As shown, the method comprises the following steps:

[0139] S502, obtaining multiple frames of sample images of the target certificate under different lighting environments;

[0140] For step S502, please refer to the detailed description of step S202 in the first embodiment of this specification, which will not be repeated here.

[0141] S504, training a preset first neural network model using the multiple frames of sample images to obtain a document pre-training model;

[0142] For step S504, please refer to the detailed description of step S204 in one embodiment of this specification, or refer to the detailed description of steps S404 to S410 in another embodiment of this specification, which will not be repeated here.

[0143] S506, performing tampering processing on the multiple frames of sample images to obtain multiple frames of tampered images;

[0144] For step S506, please refer to the detailed description of step S206 in the first embodiment of this specification, which will not be repeated here.

[0145] S508, migrating the model parameters of the certificate pre-training model to a preset second neural network model;

[0146] Before inputting multiple frames of tampered images into the second neural network model, transfer learning is required. Specifically, the model parameters of the certificate pre-training model are transferred to the second neural network model, that is, the model parameters of the certificate pre-training model are used as the initial parameters of the second neural network model to initialize the model parameters of the second neural network model.

[0147] S510, inputting the multiple frames of tampered images into the feature extraction network to obtain multi-scale intermediate features;

[0148] Among them, the feature extraction network at least includes a feature embedding network and a feature extraction network, the intermediate features include first intermediate features and second intermediate features, the first intermediate features refer to multiple layers of intermediate features of the same scale output from the feature extraction network, and the second intermediate features refer to multiple intermediate features of different scales obtained by convolution processing of a certain layer of the first intermediate features.

[0149] See also Figure 6 , is a schematic diagram of a multi-scale feature extraction process provided in the embodiments of this specification. Figure 6 As shown, the method comprises the following steps:

[0150] S602, dividing each of the tampered images into at least one image block through the feature embedding network, and embedding each of the image blocks to obtain a plurality of embedded features;

[0151] The feature embedding network can be used to segment each tampered image into at least one image block, and embed each image block to obtain multiple embedded features. For example, the feature embedding network may include at least one convolution layer, and the segmentation of the tampered image is achieved through the convolution kernel in the convolution layer, and at least one image block obtained after the segmentation is embedded to obtain the corresponding embedded features. For another example, in the feature embedding network, each image block is converted into a corresponding image block vector, and each image block vector is input into a linear projection layer, and the embedded features corresponding to each image block are output. Of course, the feature embedding network can also be other types of networks that can achieve the segmentation of the tampered image into at least one image block and embedding representation. The embodiment of this specification does not specifically limit the structure of the feature embedding network.

[0152] S604, input each of the embedded features and the position code corresponding to each of the embedded features into the feature extraction network to obtain multiple layers of first intermediate features of the same scale, and perform convolution processing on the target intermediate features in the multiple layers of first intermediate features to obtain multi-scale second intermediate features.

[0153] In order to preserve the position information, each embedded feature is added with a position encoding. The position encoding is a fixed vector that is used to distinguish the relative positions of different image patches in the original image. It is understandable that the position encoding is usually learned so that the model can optimize the expression of position information according to the specific task.

[0154] Optionally, each embedded feature and the corresponding position code are input into a feature extraction network consisting of a multi-layer Transformer encoder. Each layer of Transformer encoder contains two submodules: a multi-head self-attention mechanism and a feedforward neural network. In the multi-head self-attention mechanism, the embedded features of each image block interact with the features of all other image blocks. By calculating the attention weights, the model can pay attention to other image blocks related to the current image block, thereby capturing global context information.

[0155] Of course, the feature extraction network may also be other types of networks that can realize the extraction of image features contained in the image block. This example embodiment does not specifically limit the structure of the feature extraction network.

[0156] For example, each embedded feature and the position code corresponding to each embedded feature are input into the feature extraction network to obtain multiple layers of the same scale of the first intermediate feature, the size of the first intermediate feature is 1 / 16 scale. The target intermediate feature can refer to the last layer of features of the feature extraction network, and the target intermediate feature is convolved using convolution and deconvolution of different step lengths to obtain a multi-scale second intermediate feature.

[0157] Specifically, when the step size is set to 2, a 2x2 maximum pooling operation is used to slide a 2x2 window on the first intermediate feature map, moving two pixels each time, and selecting the maximum value in each window as the output, so that the size of the output feature becomes 1 / 2 of the first intermediate feature map, thereby obtaining a second intermediate feature with a feature size of 1 / 32. When the step size is set to 1, it means that the size of the feature map is not changed, no downsampling or upsampling operations are performed, and the first intermediate feature is directly passed to the next network layer to obtain a second intermediate feature with a feature size of 1 / 16. When the step size is set to 1 / 2, a 2x2 transposed convolution is used to obtain a second intermediate feature with a feature size of 1 / 8. When the step size is set to 1 / 4, two 2x2 transposed convolutions are used in succession to obtain a second intermediate feature with a feature size of 1 / 4. Adding LN (Layer Normalization) and GELU (Gaussian Error Linear Units) activation functions between two transposed convolutions can improve gradient flow and nonlinear transformation capabilities, further improving model performance.

[0158] S512, fusing the intermediate features of multiple scales to obtain fused features, and using the target detection network to perform target segmentation and classification prediction on the fused features to obtain a target prediction result;

[0159] Optionally, the second intermediate features of different scales obtained by step S602 and step S604 are fused to obtain fused features. For example, the second intermediate features of different scales are spliced ​​together along the channel dimension to obtain fused features. This method can retain the details of all features. For another example, weights are dynamically assigned to the second intermediate features of different scales through the attention mechanism, and then weighted summation is performed to obtain fused features. The importance of features of different scales can be flexibly adjusted according to task requirements.

[0160] After obtaining the fused features, the fused features can be input into the target detection network for target segmentation and classification prediction to obtain a target prediction result, which includes the tampering location information and tampering type in the tampered image. Among them, the target detection network can be a Faster R-CNN (Fast Region-based Convolutional Neural Network) network, a YOLO network, a Mask R-CNN (Mask Region-based Convolutional Neural Network) network, etc., and the embodiments of this specification do not limit this.

[0161] S514, iteratively training the feature extraction network and the target detection network based on the target prediction result until an iteration termination condition is met, thereby obtaining the tampering detection model.

[0162] Based on the target prediction results, the back-propagation algorithm is used to jointly optimize the feature extraction network and the target detection network, and a loss function that comprehensively considers the classification, positioning accuracy and tampering detection characteristics is constructed. That is, the loss function includes classification loss, positioning loss and additional loss terms in tampering detection (such as consistency loss or regularization term) to penalize the difference between the prediction result and the true label.

[0163] For example, the model parameters of the feature extraction network and the model parameters of the target detection network can be updated using the stochastic gradient descent algorithm. According to the principle of back propagation, the loss function is continuously calculated, and the model parameters of each model are simultaneously updated according to the calculated loss value. When the loss function converges to the minimum value, the training of all model parameters is completed. The model parameters can also be updated iteratively in reverse, and the training of all model parameters is completed when the preset number of iterations is met. After the iteration is completed, the optimized model parameters can be obtained. In other examples, the alternating least squares method, Adam optimization algorithm, etc. can also be used to minimize the loss function, update the model parameters from back to front, and optimize the model parameters. The optimized tampering detection model can be used for new images or video frames, which can automatically identify and locate possible tampering areas and help determine the authenticity of the image.

[0164] In this embodiment, the transfer learning strategy can effectively reuse the feature representations learned by the document pre-training model and accelerate the convergence of the second neural network model. At the same time, the pre-training parameters contain rich prior knowledge, which helps the second neural network model learn more complex features on limited training data, so that the second neural network model can better understand and capture key information in the document image and improve the detection ability of unknown tampering types. Moreover, the document pre-training model is trained on a large-scale data set, which means that the data requirements and computing resources can be reduced during the training process of the second neural network model, reducing the training cost.

[0165] See also Figure 7 , is a flow chart of a tamper detection method provided in an embodiment of this specification. In an embodiment of this specification, the tamper detection method is applied to a tamper detection device or an electronic device equipped with a tamper detection device. Figure 7 The process shown is described in detail, and the tampering detection method may specifically include the following steps:

[0166] S702, obtaining multiple frames of document images of the document to be detected under different lighting environments;

[0167] In one or more embodiments of the present specification, the certificate to be detected refers to a certificate that needs to be detected for tampering. The certificate to be detected may be the same as the target certificate, such as one of the target certificates, or may be different from the target certificate. The present specification does not limit this.

[0168] After obtaining multiple frames of document images of the document to be detected under different lighting environments, the multiple frames of document images can be aligned to input the aligned multiple frames of document images into a trained tampering detection model.

[0169] Optionally, the multiple frames of document images of the document to be detected under different lighting environments include a second reference frame image of the document to be detected under a normal lighting environment and a second target frame image of the document to be detected under a flash lighting environment. For example, the reference corner points in the second reference frame image are strictly aligned with the target corner points in the second target frame image. For specific implementation, please refer to the detailed description of step S302 and step S304 in the first embodiment of this specification, which will not be repeated here.

[0170] S704, inputting the multiple frames of document images into a trained tampering detection model to obtain a tampering detection result.

[0171] The tampering detection result includes the tampering location information and tampering type corresponding to the certificate to be detected.

[0172] It should be noted that the tampering detection model is trained by the method described in detail in steps S202 to S208 of an embodiment of this specification, or is trained by the method described in detail in steps S402 to S414 of another embodiment of this specification, or is trained by the method described in detail in steps S502 to S514 of yet another embodiment of this specification, which will not be repeated here.

[0173] In one possible embodiment, see Figure 8 , is a schematic diagram of a tamper detection model provided in an embodiment of this specification. Figure 8 As shown in the figure, the tampering detection model includes two structures: Vit (Vision Transformer, computer vision) network and Mask R-CNN network.

[0174] Specifically, when the aligned second reference frame image 801 and the second target frame image 802 are input into the Vit network, each frame image is divided into multiple image blocks, and each image block is converted into a one-dimensional vector through a block embedding operation, and then the category vector and the position vector are added as model input.

[0175] The Block structure in the Vit network (such as Figure 8803-807) is an encoder structure based on Transfommer, which includes a multi-head self-attention network structure and a feedforward neural network structure.

[0176] The ViT network can only obtain features of one scale (such as 1 / 16). In order to obtain finer-grained information, an additional convolutional layer is used to downsample the output features to obtain scale feature maps of 1 / 32 and 1 / 16. At the same time, in order to obtain higher-resolution feature maps, a deconvolution operation can be used to upsample the output features to obtain scale feature maps of 1 / 8 and 1 / 4.

[0177] Then, the feature maps of different scales are fused together to form a feature pyramid, and the feature pyramid is input into the Mask R-CNN network. The Mask R-CNN network uses the input feature map to perform target detection and segmentation, and obtains the outputs corresponding to the second reference frame image 801 and the second target frame image 802. Further, the difference between the two outputs is compared, such as comparing the feature vectors, bounding box positions, masks, etc. of the second reference frame image 801 and the second target frame image 802. If there is a difference between the two, the area with the difference may be the tampered part, and the specific method of tampering can also be determined based on the result of feature comparison and difference analysis.

[0178] In this embodiment, the combination of the ViT network and the Mask R-CNN network enables the model to not only identify whether there is tampering in the document image, but also accurately locate the specific location of the tampering. In addition, the ViT network uses the self-attention mechanism to capture the long-distance dependencies between different parts of the document image, which is suitable for processing complex patterns and texts that may exist in the document image, and improves the feature expression ability of the model.

[0179] It should also be noted that the basic framework of the tampering detection model in the embodiments of this specification is highly versatile and can be extended to other types of image tampering detection fields, such as currency anti-counterfeiting, image content authenticity verification, etc.

[0180] See also Fig. 9 , is a schematic diagram of the structure of a tamper detection model training device provided in an embodiment of this specification. Fig. 9 As shown, the tampering detection model training device 1 can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the tampering detection model training device 1 includes a sample image acquisition module 11, a first model training module 12, a multi-frame image tampering module 13, and a second model training module 14, specifically including:

[0181] The sample image acquisition module 11 is used to acquire multiple frames of sample images of the target certificate under different lighting environments;

[0182] A first model training module 12, used to train a preset first neural network model using the multiple frames of sample images to obtain a document pre-training model;

[0183] A multi-frame image tampering module 13 is used to tamper with the multi-frame sample image to obtain a multi-frame tampered image;

[0184] The second model training module 14 is used to migrate the model parameters of the document pre-training model to a preset second neural network model, and use the multiple frames of tampered images to train the second neural network model to obtain a tampering detection model.

[0185] Optionally, the second neural network includes a feature extraction network and a target detection network;

[0186] When the second model training module 14 trains the second neural network model using the multiple frames of tampered images to obtain the tampering detection model, it is specifically used to:

[0187] Inputting the multiple frames of tampered images into the feature extraction network to obtain multi-scale intermediate features;

[0188] The intermediate features of multiple scales are fused to obtain fused features, and the target detection network is used to perform target segmentation and classification prediction on the fused features to obtain a target prediction result;

[0189] The feature extraction network and the target detection network are iteratively trained based on the target prediction result until an iteration termination condition is met, thereby obtaining the tampering detection model.

[0190] Optionally, the feature extraction network includes at least a feature embedding network and a feature extraction network, and the intermediate feature includes a first intermediate feature and a second intermediate feature; when the second model training module 14 inputs the multi-frame tampered image into the feature extraction network to obtain multi-scale intermediate features, it is specifically used to:

[0191] Segmenting each of the tampered images into at least one image block through the feature embedding network, and embedding each of the image blocks to obtain a plurality of embedded features;

[0192] Each of the embedded features and the position code corresponding to each of the embedded features are input into the feature extraction network to obtain multiple layers of first intermediate features of the same scale, and convolution processing is performed on the target intermediate features in the multiple layers of first intermediate features to obtain multi-scale second intermediate features.

[0193] Optionally, the first neural network model includes an encoder and a decoder; when the first model training module 12 executes the training of the preset first neural network model using the multiple frames of sample images, it is specifically used to:

[0194] Performing random mask processing on the multiple frames of sample images to obtain multiple frames of input images;

[0195] Inputting unmasked image blocks in the multiple frames of input images into the encoder to obtain sample image features;

[0196] The decoder is used to splice and decode the sample image features and the mask image blocks in the multiple frames of input images to obtain a reconstructed image;

[0197] A loss value between the sample image and the reconstructed image is calculated in pixel space using a preset loss function, and model parameters of the encoder and model parameters of the decoder are adjusted according to the loss value.

[0198] Optionally, in the sample image acquisition module 11, the multiple frames of sample images of the target document in different lighting environments include a first reference frame image of the target document in a normal lighting environment and a first target frame image of the target document in a flash lighting environment.

[0199] Optionally, the tampering detection model training device 1 further includes a sample image alignment module 15, which is specifically used to:

[0200] After acquiring multiple frames of sample images of the target certificate under different lighting environments, the first reference frame image and the first target frame image are aligned.

[0201] Optionally, when the sample image alignment module 15 aligns the first reference frame image with the first target frame image, it is specifically used to:

[0202] Performing corner point detection on the first reference frame image and the first target frame image respectively, to determine reference corner points in the first reference frame image and target corner points in the first target frame image;

[0203] A transformation matrix is ​​calculated according to the reference corner point and the target corner point, and the first target frame image is perspective transformed using the transformation matrix to align the reference corner point with the target corner point.

[0204] Optionally, when the multi-frame image tampering module 13 performs tampering processing on the multi-frame sample images to obtain the multi-frame tampered images, it is specifically used to:

[0205] According to a preset tampering type, the aligned first reference frame image and the first target frame image are tampered with to obtain the multi-frame tampered image.

[0206] Optionally, in the multi-frame image tampering module 13, the preset tampering type includes at least one or more of rectangular tampering, text overlay and foreign object occlusion.

[0207] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0208] See also Fig.10 , is a schematic diagram of the structure of a tamper detection device provided in an embodiment of this specification. Fig.10 As shown, the tampering detection device 2 can be implemented as all or part of the electronic device through software, hardware or a combination of both. According to some embodiments, the tampering detection device 2 includes a document image acquisition module 21 and an image tampering detection module 22, specifically including:

[0209] The document image acquisition module 21 is used to acquire multiple frames of document images of the document to be detected under different lighting environments;

[0210] An image tampering detection module 22, used for inputting the multiple frames of document images into a trained tampering detection model to obtain a tampering detection result;

[0211] Wherein, the tampering detection model is trained by the method described in the tampering detection model training device.

[0212] Optionally, the tampering detection device 2 further includes a document image alignment module 23, which is specifically used to:

[0213] After acquiring multiple frames of document images of the document to be detected under different lighting environments, the multiple frames of document images are aligned, so as to input the aligned multiple frames of document images into a trained tampering detection model.

[0214] Optionally, in the document image acquisition module 21, the multiple frames of document images of the document to be detected in different lighting environments include a second reference frame image of the document to be detected in a normal lighting environment and a second target frame image of the document to be detected in a flash lighting environment.

[0215] Optionally, in the image tampering detection module 22, the tampering detection result includes tampering location information and tampering type corresponding to the document to be detected.

[0216] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0217] The present specification also provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figures 2 to 7 The method of the embodiment shown in the figure can be specifically executed by referring to Figures 2 to 7 The specific description of the illustrated embodiment will not be repeated here.

[0218] This specification also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded by the processor and executes the method of the embodiment shown above. The specific execution process can be seen in Figures 2 to 7 The specific description of the illustrated embodiment will not be repeated here.

[0219] The embodiments of this specification also provide Fig.11 The structural diagram of the electronic device shown in FIG. Fig.11 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned voice activity detection method.

[0220] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0221] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0222] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0223] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0224] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0225] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0226] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0227] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0228] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0229] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0230] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0231] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0232] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0233] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0234] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0235] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0236] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A tamper detection model training method, the method comprising: Acquire multiple frames of sample images of the target certificate under different lighting environments; Using the multiple frames of sample images to train a preset first neural network model to obtain a document pre-training model; Performing tampering processing on the multiple frames of sample images to obtain multiple frames of tampered images; The model parameters of the document pre-training model are migrated to a preset second neural network model, and the second neural network model is trained using the multiple frames of tampered images to obtain a tampering detection model.

2. According to the tampering detection model training method of claim 1, the second neural network includes a feature extraction network and a target detection network; The method of training the second neural network model using the multiple frames of tampered images to obtain a tampering detection model includes: Inputting the multiple frames of tampered images into the feature extraction network to obtain multi-scale intermediate features; The intermediate features of multiple scales are fused to obtain fused features, and the target detection network is used to perform target segmentation and classification prediction on the fused features to obtain a target prediction result; The feature extraction network and the target detection network are iteratively trained based on the target prediction result until an iteration termination condition is met, thereby obtaining the tampering detection model.

3. According to the tampering detection model training method of claim 2, the feature extraction network at least includes a feature embedding network and a feature extraction network, and the intermediate feature includes a first intermediate feature and a second intermediate feature; The step of inputting the multi-frame tampered images into the feature extraction network to obtain multi-scale intermediate features includes: Segmenting each of the tampered images into at least one image block through the feature embedding network, and embedding each of the image blocks to obtain a plurality of embedded features; Each of the embedded features and the position code corresponding to each of the embedded features are input into the feature extraction network to obtain multiple layers of first intermediate features of the same scale, and convolution processing is performed on the target intermediate features in the multiple layers of first intermediate features to obtain multi-scale second intermediate features.

4. According to the tampering detection model training method of claim 1, the first neural network model includes an encoder and a decoder; The step of training a preset first neural network model using the multiple frames of sample images includes: Performing random mask processing on the multiple frames of sample images to obtain multiple frames of input images; Inputting unmasked image blocks in the multiple frames of input images into the encoder to obtain sample image features; The decoder is used to splice and decode the sample image features and the mask image blocks in the multiple frames of input images to obtain a reconstructed image; A loss value between the sample image and the reconstructed image is calculated in pixel space using a preset loss function, and model parameters of the encoder and model parameters of the decoder are adjusted according to the loss value.

5. According to the tampering detection model training method of claim 1, the multiple frames of sample images of the target certificate under different lighting environments include a first reference frame image of the target certificate under a normal lighting environment and a first target frame image of the target certificate under a flash lighting environment.

6. According to the tampering detection model training method of claim 1, the tampering processing of the multiple frames of sample images to obtain multiple frames of tampered images comprises: According to a preset tampering type, the aligned first reference frame image and the first target frame image are tampered with to obtain the multi-frame tampered image.

7. A tampering detection method, the method comprising: Acquire multiple frames of document images of the document to be detected under different lighting environments; Inputting the multiple frames of document images into a trained tampering detection model to obtain tampering detection results; Wherein, the tampering detection model is trained by the method described in any one of claims 1-8.

8. According to the tampering detection method of claim 7, the multiple frames of document images of the document to be detected under different lighting environments include a second reference frame image of the document to be detected under a normal lighting environment and a second target frame image of the document to be detected under a flash lighting environment.

9. The tampering detection method according to claim 7, wherein the tampering detection result includes tampering location information and tampering type corresponding to the certificate to be detected.

10. A computer program product having at least one instruction stored thereon, wherein when the at least one instruction is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Citation Information

Patent Citations

  • Certificate image defect detection method and device

    CN114581359A

  • Certificate identification model training method and device, storage medium and electronic equipment

    CN117912046A

  • Methods and apparatuses for detecting tamper using heuristic models

    US20200025789A1

Cited By

  • Industrial equipment indicating lamp state identification method based on image sequence and Transform network

    CN120808031A