Certificate anti-counterfeiting detection model training method, certificate anti-counterfeiting detection method and device
By combining the training method of the document anti-counterfeiting detection model with sub-networks of segmentation and block consistency comparison learning, the problem of accurate detection of tampered areas in document images is solved, thereby improving the anti-counterfeiting detection capability of document images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANT BLOCKCHAIN TECHNOLOGY (SHANGHAI) CO LTD
- Filing Date
- 2023-03-29
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies struggle to accurately detect tampered areas in document images, leading to a risk of deception during the identity authentication process.
A document anti-counterfeiting detection model is adopted, which combines a first sub-network based on segmentation and a second sub-network based on block consistency contrast learning. By training a loss function, the tampered areas in the document image are located and distinguished.
It improves the accuracy and detection capability of locating tampered areas in document images, enabling the identification of tampered areas in document images and reducing the risk of deception in identity authentication.
Smart Images
Figure CN116310370B_ABST
Abstract
Description
Technical Field
[0001] This specification relates to electronic information technology in one or more embodiments, and more particularly to a training method for a document anti-counterfeiting detection model, a document anti-counterfeiting detection method, and an apparatus. Background Technology
[0002] Currently, document anti-counterfeiting detection is frequently required in various business scenarios. For example, various applications (such as financial applications) need to verify electronic ID cards for identity authentication or account opening; another example is verifying employee work cards. For instance, during the electronic ID card account opening process, key information such as the facial image, ID number, and name on the document can be easily tampered with, deceiving the application system to achieve the goal of successful account opening. Therefore, how to accurately detect whether documents have been tampered with is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention
[0003] This specification describes one or more embodiments of a training method for a document anti-counterfeiting detection model, a document anti-counterfeiting detection method, and an apparatus, which can more accurately identify tampered areas in document images.
[0004] According to the first aspect, a training method for a document anti-counterfeiting detection model is provided, wherein the structure of the document anti-counterfeiting detection model includes: a first sub-network based on segmentation and a second sub-network based on block consistency contrastive learning; and the loss function of the document anti-counterfeiting detection model includes: a weighted sum of a first loss function corresponding to the first sub-network and a second loss function corresponding to the second sub-network; the method includes:
[0005] Obtain a sample set of document images; this sample set includes multiple document images and a label for each document image; wherein, the label is used to mark the tampered areas in the document images;
[0006] The set of document image samples is input into the first sub-network, and the first sub-network is trained using the loss function of the document anti-counterfeiting detection model. The first sub-network is then used to train the document anti-counterfeiting detection model’s ability to locate tampered areas and detect targets.
[0007] The sample set of document images is input into the second sub-network, and the loss function of the document anti-counterfeiting detection model is used to train the second sub-network, so as to train the document anti-counterfeiting detection model to distinguish the tampered area from the image background area.
[0008] The training of the first sub-network includes:
[0009] For each document image in the document image sample set, the following is performed:
[0010] The first feature map is obtained by downsampling encoding the current document image using the first segmentation-based sub-network;
[0011] The first feature map is decoded based on upsampling to obtain the second feature map;
[0012] Based on the prediction results of the first sub-network of the tampered region in the second feature map and the label of the current document image, the first loss function corresponding to the first sub-network is adjusted.
[0013] During the training of the segmentation-based first sub-network, the upsampling-based decoding includes:
[0014] By upsampling the first feature map of the ID card image at different multiples in successive stages, the relationship between the first feature map of the ID card image at different feature layers is learned.
[0015] For the features of the first feature map of the document image at each feature layer, the features at that feature layer are divided into N*N image feature regions. An attention mechanism is used to learn the relationship between different image feature regions of the first feature map of the document image at the same feature layer and the weights of different image regions at the same feature layer.
[0016] The learning through the attention mechanism includes: performing a dot product between an N*N attention matrix and the N*N image feature regions.
[0017] The training of the second sub-network includes:
[0018] For each document image in the document image sample set, the following is performed:
[0019] The third feature map of the current document image is obtained by the second sub-network based on block consistency contrastive learning;
[0020] Each element in the third feature map is compared with all elements in the third feature map in pairs. If, based on the label of the current document image, it is determined that the image regions of the elements being compared in pairs are either both tampered with or both are not tampered with, then the comparison result is the first category. If, based on the label of the current document image, it is determined that one image region of the elements being compared in pairs is tampered with and the other image region is not tampered with, then the comparison result is the second category. After the comparison is completed, a block consistency category map is obtained.
[0021] Based on the prediction results of the tampered region of the third feature map by the second sub-network and the block consistency category map, the second loss function corresponding to the second sub-network is adjusted.
[0022] The first sub-network is a UNet network;
[0023] And / or,
[0024] The loss function of the document anti-counterfeiting detection model is L, L = a*L1 + L2, where L1 is the first loss function corresponding to the first sub-network, L2 is the second loss function corresponding to the second sub-network, and a is the weight of the first sub-network based on the segmentation.
[0025] The segmentation-based first sub-network and the block consistency-based contrastive learning-based second sub-network share the backbone of the document anti-counterfeiting detection model; wherein, the backbone uses a pre-trained large graph-text model.
[0026] Training the segmentation-based first sub-network and the block consistency contrastive learning-based second sub-network further includes: freezing the encoding parameters of the pre-trained graph-text large model and training the parameters of the first sub-network and the second sub-network.
[0027] According to the second aspect, a method for detecting counterfeit documents is provided, wherein the method includes:
[0028] Obtain the image of the document to be inspected;
[0029] Input the image of the document to be detected into the document anti-counterfeiting detection model;
[0030] The anti-counterfeiting recognition result of the document anti-counterfeiting detection model is obtained; wherein, if the output document image to be detected includes a prominent area, the prominent area represents the tampered area.
[0031] The anti-counterfeiting recognition result of the document anti-counterfeiting detection model outputting the document image to be detected includes:
[0032] The output of the first sub-network in the document anti-counterfeiting detection model is used as the anti-counterfeiting recognition result of the document image to be detected.
[0033] or,
[0034] The intersection of the outputs of the first sub-network and the second sub-network in the document anti-counterfeiting detection model is obtained, and this intersection is used as the anti-counterfeiting recognition result of the document image to be detected.
[0035] or,
[0036] The union of the outputs of the first sub-network and the second sub-network in the document anti-counterfeiting detection model is obtained, and this union is used as the anti-counterfeiting recognition result of the document image to be detected.
[0037] According to the third aspect, a training device for a document anti-counterfeiting detection model is provided, wherein the structure of the document anti-counterfeiting detection model includes: a first sub-network based on segmentation and a second sub-network based on block consistency contrastive learning; the loss function of the document anti-counterfeiting detection model includes: a weighted sum of a first loss function corresponding to the first sub-network and a second loss function corresponding to the second sub-network; the device includes:
[0038] The sample acquisition module is configured to obtain a set of document image samples; the set of document image samples includes multiple document images and a label for each document image; wherein, the label is used to mark the tampered areas in the document images;
[0039] The first sub-network training module is configured to input the document image sample set into the first sub-network and train the first sub-network using the loss function of the document anti-counterfeiting detection model, so as to train the document anti-counterfeiting detection model's ability to locate tampered areas and detect targets by training the first sub-network;
[0040] The second sub-network training module is configured to input the document image sample set into the second sub-network, and train the second sub-network using the loss function of the document anti-counterfeiting detection model, so as to train the document anti-counterfeiting detection model to distinguish the tampered area from the image background area by training the second sub-network.
[0041] According to the fourth aspect, a document anti-counterfeiting detection device is provided, wherein the device includes:
[0042] The document image acquisition module is configured to obtain the document image to be detected;
[0043] The input module is configured to input the image of the document to be detected into the document anti-counterfeiting detection model;
[0044] The detection result acquisition module is configured to obtain the anti-counterfeiting recognition result of the document anti-counterfeiting detection model output by the document anti-counterfeiting detection model; wherein, if the output document image to be detected includes a prominent area, the prominent area represents the tampered area.
[0045] According to a fifth aspect, a computer-readable storage medium is provided, having a computer program stored thereon that, when executed in a computer, causes the computer to perform the methods described in any embodiment of this specification.
[0046] According to a sixth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method described in any embodiment of this specification.
[0047] The training methods, methods, and devices for document anti-counterfeiting detection provided in the various embodiments of this specification, individually or in combination, have at least the following beneficial effects:
[0048] 1. This specification describes a novel document anti-counterfeiting detection model. The model comprises two parts: a first sub-network based on segmentation and a second sub-network based on block consistency comparison learning. On one hand, the first sub-network is trained using document image samples with marked tampered areas. Based on the first sub-network's image segmentation learning ability, the model's ability to locate tampered areas and detect objects is trained. On the other hand, the second sub-network is trained using document image samples with marked tampered areas. Based on the second sub-network's block consistency comparison learning ability, the model's ability to distinguish tampered areas from the image background is trained. By setting a loss function, the first and second sub-networks can be jointly trained. Thus, the final trained document anti-counterfeiting detection model can accurately locate tampered areas in document images from both localization and object detection perspectives, as well as block consistency comparison perspectives.
[0049] 2. When an ID document image is tampered with, whether through Photoshop or methods such as replacing stickers, the tampered area will have a different texture than the untampered area. Based on this characteristic, in the embodiments of this specification, the ID document anti-counterfeiting detection model includes a second sub-network based on block consistency, thereby distinguishing between tampered and untampered areas in the ID document image by differentiating between areas of similar origin and areas of different origin.
[0050] 3. In actual business implementation, the number of samples for document images is very small. Therefore, the document anti-counterfeiting detection model in this embodiment may include a pre-trained large image-text model, so that a good training effect can still be obtained even with a small number of document image samples. Furthermore, in this embodiment, the encoding parameters of the pre-trained large image-text model are frozen. This allows the large image-text model, which has already been trained on large datasets of other types, to retain its strong representational ability and better generalization ability, thereby improving the accuracy of the second sub-network based on block consistency contrastive learning.
[0051] 4. For document images, the tampered areas are usually the face region and the text region. Therefore, in the embodiments of this specification, when training the segmentation sub-network, a module is added between the backbone network and the decoding module. This module uses the attention mechanism to learn the relationship between different image feature regions within the same feature layer. During the learning process, the weights of the face region and the text region are increased, so that the model can better understand which image region is more important in the same feature layer, thereby improving the training effect of the first sub-network based on segmentation. Attached Figure Description
[0052] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0053] Figure 1 This is a schematic diagram of the system architecture used in one embodiment of this specification.
[0054] Figure 2 This is a flowchart of a training method for a document anti-counterfeiting detection model in one embodiment of this specification.
[0055] Figure 3 This is a schematic diagram of the network structure of a document anti-counterfeiting detection model in one embodiment of this specification.
[0056] Figure 4 This is a flowchart of a document anti-counterfeiting detection method in one embodiment of this specification.
[0057] Figure 5 This is a schematic diagram of the structure of a training device for a document anti-counterfeiting detection model in one embodiment of this specification.
[0058] Figure 6 This is a schematic diagram of the structure of a document anti-counterfeiting detection device in one embodiment of this specification. Detailed Implementation
[0059] The solution provided in this specification will now be described with reference to the accompanying drawings.
[0060] First, it should be noted that the terminology used in the embodiments of this invention is for the purpose of describing specific embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.
[0061] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.
[0062] To facilitate understanding of the methods provided in this manual, the system architecture involved and applicable to this manual is first described. For example... Figure 1 As shown, the system architecture mainly includes three types of network nodes: web crawler module, data processing module, and training execution module.
[0063] The web crawler module is used to obtain a large number of document images from the internet, such as ID card images, employee ID card images, and driver's license images. The data processing module is used to obtain a sample set of document images generated from the document images. The training and execution module is used to train a document anti-counterfeiting detection model using the sample set of document images.
[0064] It should be understood that Figure 1 The number of web crawler modules, data processing modules, and training execution modules shown is merely illustrative. Any number can be selected and deployed according to implementation needs.
[0065] Figure 2 This is a flowchart of a training method for a document anti-counterfeiting detection model in one embodiment of this specification. The main entity executing this method is the training device for the document anti-counterfeiting detection model. It is understood that this method can also be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities. In this training method, see [link to relevant documentation]. Figure 3 The pre-defined structure of the document anti-counterfeiting detection model includes two parts: a first sub-network based on segmentation and a second sub-network based on block consistency contrastive learning; and the loss function of the document anti-counterfeiting detection model includes a weighted sum of the first loss function corresponding to the first sub-network and the second loss function corresponding to the second sub-network; see [link to relevant documentation]. Figure 2 The training method includes the following steps in the specific training process:
[0066] Step 201: Obtain a sample set of document images; the sample set of document images includes multiple document images and a label for each document image; wherein, the label is used to mark the tampered areas in the document images;
[0067] Step 203: Input the document image sample set into the first sub-network, and train the first sub-network using the loss function of the document anti-counterfeiting detection model, so as to train the document anti-counterfeiting detection model's ability to locate tampered areas and detect targets by training the first sub-network;
[0068] Step 205: Input the document image sample set into the second sub-network, and train the second sub-network using the loss function of the document anti-counterfeiting detection model, so as to train the document anti-counterfeiting detection model to distinguish the tampered area from the image background area.
[0069] According to the above Figure 2 As shown in the flowchart, this embodiment of the specification designs a novel document anti-counterfeiting detection model. This model comprises two parts: a first sub-network based on segmentation and a second sub-network based on block consistency comparison learning. On one hand, the first sub-network is trained using document image samples with marked tampered areas. Based on the first sub-network's image segmentation learning ability, the model's ability to locate tampered areas and detect objects is trained. On the other hand, the second sub-network is trained using document image samples with marked tampered areas. Based on the second sub-network's block consistency comparison learning ability, the model's ability to distinguish tampered areas from the image background is trained. By setting the loss function, the first and second sub-networks can be jointly trained. Thus, the finally trained document anti-counterfeiting detection model can accurately locate tampered areas in the document image from both the perspectives of localization and object detection, as well as block consistency comparison.
[0070] The following will use specific examples to illustrate... Figure 2 Each step shown is explained separately.
[0071] See Figure 3 The pre-defined structure of the document anti-counterfeiting detection model comprises two parts: a first sub-network based on segmentation and a second sub-network based on block consistency contrastive learning. Furthermore, the loss function of the document anti-counterfeiting detection model is defined as a weighted sum of the first loss function corresponding to the first sub-network and the second loss function corresponding to the second sub-network. The expression for the loss function L is: L = a * L1 + L2, where L1 is the first loss function corresponding to the first sub-network, L2 is the second loss function corresponding to the second sub-network, and a is the weight of the segmentation-based first sub-network.
[0072] In the specific training, firstly, for step 201: obtain the document image sample set; the document image sample set includes multiple document images and a label for each document image; wherein, the label is used to mark the tampered areas in the document image.
[0073] In step 201, a web crawler module can obtain a large number of required document images from the network, such as a large number of employee ID card images, and tamper with one area, such as the headshot, and then mark the tampered area. This generates a document image sample set including multiple document images and labels for each image.
[0074] Next, for step 203: input the document image sample set into the first sub-network, and train the first sub-network using the loss function of the document anti-counterfeiting detection model, so as to train the document anti-counterfeiting detection model's ability to locate tampered areas and detect targets by training the first sub-network.
[0075] In step 203, after inputting the document image sample set into the first sub-network, the process of training the first sub-network includes:
[0076] For each document image in the document image sample set, the following is performed:
[0077] Step 2031: The current document image is encoded based on downsampling by the first sub-network based on segmentation to obtain a feature map, denoted as the first feature map;
[0078] Step 2033: Decode the first feature map based on upsampling to obtain a new feature map, denoted as the second feature map;
[0079] Step 2035: Adjust the first loss function of the corresponding first sub-network based on the prediction result of the tampered region in the second feature map by the first sub-network and the label of the current document image.
[0080] In step 2033 above, the upsampling-based decoding process during the training of the segmentation-based first sub-network includes:
[0081] On the one hand, by upsampling the first feature map of the document image at different multiples, such as 16 times upsampling, 8 times upsampling, 4 times upsampling, etc., the relationship between the first feature map of the document image in different feature layers is learned.
[0082] On the other hand, for the features of the first feature map of the ID card image at each feature layer, the features at that feature layer are divided into N*N image feature regions. An attention mechanism is used to learn the relationships between different image feature regions of the first feature map of the ID card image at the same feature layer, as well as the weights of different image regions at the same feature layer. For example, N can be 10. Specifically, when learning through the attention mechanism, an N*N attention matrix can be used to perform a dot product with the N*N image feature regions.
[0083] During the upsampling process, the 16th upsampling step is convolved with the 8th upsampling step, and the 8th upsampling step is convolved with the 4th upsampling step. At each convolutional connection, a pre-set small module processes the data, using an attention mechanism to learn and uncover the connections between different image feature regions within the same feature layer. This continuously adjusts the weights for different regions, giving higher weights to more important image regions, such as face or text regions. When using the attention mechanism, a feature map can be segmented into 10x10 image feature regions, generating a 10x10 attention matrix. A dot product is then performed between each 10x10 image region and the 10x10 attention matrix, causing the model to focus more on which image region within the same feature layer is more important.
[0084] In the embodiments described in this specification, the first sub-network can be a UNet network.
[0085] In step 203, we first downsample the feature map of each document image. The purpose of downsampling is: 1. to save computational resources; 2. to increase the receptive field, that is, to obtain higher-level features. Higher-level features contain more semantic information, while lower-level features contain more detailed information. To know whether there is a segmentation object, we need to look at the higher-level features; to know where the segmentation object is, we need to look at the lower-level features. Therefore, through downsampling, we can obtain more high-level semantic information in the image. Next, we upsample the feature map of the document image. The purpose of upsampling is: 1. to restore the original input image size; 2. to concatenate the features of the corresponding encoding and decoding layers, thus considering both high-level and low-level features, resulting in better segmentation performance.
[0086] Step 205: Input the document image sample set into the second sub-network, and train the second sub-network using the loss function of the document anti-counterfeiting detection model, so as to train the document anti-counterfeiting detection model to distinguish the tampered area from the image background area.
[0087] In step 205, after inputting the document image sample set into the first sub-network, the process of training the second sub-network includes:
[0088] For each document image in the document image sample set, the following is performed:
[0089] Step 2051: Obtain the feature map of the current document image from the second sub-network based on block consistency contrastive learning, denoted as the third feature map;
[0090] Step 2053: Compare each element in the third feature map with all elements in the third feature map in pairs. If, based on the label of the current document image, it is determined that the image regions of the elements being compared in pairs are either both tampered with or both are not tampered with, then the comparison result is homogeneous and is denoted as the first category, for example, denoted as 0. If, based on the label of the current document image, it is determined that one image region of the elements being compared in pairs is tampered with and the other image region is not tampered with, then the comparison result is dissimilar and is denoted as the second category, for example, denoted as 1. After the comparison is completed, a block consistency category map is obtained.
[0091] For example, in step 2053, the feature map of the document image can be downsampled to obtain a third feature map. This third feature map can be regarded as a 10*10 feature matrix. For each element in the feature matrix, the element is compared with all elements in the feature matrix including the element. That is, each element can obtain 10*10, or 100 comparison results. Therefore, a 100*100 block consistency category map can be obtained in the end.
[0092] When an ID document image is tampered with, whether through Photoshop or methods such as replacing stickers, the tampered area will have a different texture than the untampered area. Based on this characteristic, the ID document anti-counterfeiting detection model in this embodiment includes a second sub-network based on block consistency, which distinguishes between tampered and untampered areas in the ID document image by differentiating between areas of similar origin and areas of different origin.
[0093] Step 2055: Adjust the second loss function of the corresponding second sub-network based on the prediction results of the tampered region of the third feature map by the second sub-network and the block consistency category map.
[0094] In one embodiment of this specification, in Figure 2 In the process shown, the first sub-network based on segmentation and the second sub-network based on block consistency contrastive learning share the backbone of the document anti-counterfeiting detection model; wherein, the backbone uses a pre-trained large graph and text model such as MAE or simMIN;
[0095] When training the first sub-network based on segmentation and the second sub-network based on block consistency contrastive learning, the encoding parameters of the pre-trained graph-text large model can be frozen, and the parameters of the first and second sub-networks can be trained.
[0096] In actual business implementations, the number of samples for document images is very small. Therefore, the document anti-counterfeiting detection model in this embodiment may include a pre-trained large-scale image-text model, thereby achieving good training results even with a small number of document image samples. Furthermore, in this embodiment, the encoding parameters of the pre-trained large-scale image-text model are frozen, and the decoding parameters are used for training. This allows the large-scale image-text model, which has already been trained on large datasets of other types, to retain its stronger representational capabilities and better generalization ability, thereby improving the accuracy of the second sub-network based on block consistency contrastive learning.
[0097] This specification provides an embodiment of a method for detecting counterfeit documents. (See also...) Figure 4 The method includes:
[0098] Step 401: Obtain the image of the document to be inspected.
[0099] Step 403: Input the image of the document to be detected into the document anti-counterfeiting detection model.
[0100] The document anti-counterfeiting detection model in step 403 is trained using the document anti-counterfeiting detection model training method of any embodiment of this specification.
[0101] Step 405: Obtain the anti-counterfeiting recognition result of the document anti-counterfeiting detection model outputting the document anti-counterfeiting image to be detected; wherein, if the output document image to be detected includes a prominent area, the prominent area represents the tampered area.
[0102] The implementation of step 405 can include at least the following:
[0103] Method 1: Only the output of the first sub-network in the document anti-counterfeiting detection model is used as the anti-counterfeiting recognition result of the document image to be detected.
[0104] Referring to the training method in any embodiment of this specification, in the document anti-counterfeiting detection model, the first sub-network and the second sub-network are jointly trained. Therefore, after training, even the recognition result output by the first sub-network will not only have the characteristics of the recognition result of the first sub-network, but also the characteristics of the recognition result of the second sub-network. Therefore, this method one can be adopted. This method one is more suitable for business scenarios with higher requirements for detection efficiency, such as those where it is desirable to obtain anti-counterfeiting detection results faster.
[0105] Method 2: Obtain the intersection of the output of the first sub-network and the output of the second sub-network in the document anti-counterfeiting detection model, and use this intersection as the anti-counterfeiting recognition result of the document image to be detected.
[0106] Method 2 is more suitable for business scenarios that require higher accuracy in detection.
[0107] Method 3: Obtain the union of the outputs of the first sub-network and the second sub-network in the document anti-counterfeiting detection model, and use this union as the anti-counterfeiting recognition result of the document image to be detected.
[0108] Method 3 is more suitable for business scenarios that want to provide more possibilities for tampering with the area.
[0109] In the embodiments of this specification, a training device for a document anti-counterfeiting detection model is also proposed. The structure of the document anti-counterfeiting detection model includes: a first sub-network based on segmentation and a second sub-network based on block consistency contrastive learning; the loss function of the document anti-counterfeiting detection model includes: a weighted sum of a first loss function corresponding to the first sub-network and a second loss function corresponding to the second sub-network; see also... Figure 5 The device includes:
[0110] The sample acquisition module 501 is configured to obtain a set of document image samples; the set of document image samples includes multiple document images and a label for each document image; wherein, the label is used to mark the tampered areas in the document images;
[0111] The first sub-network training module 502 is configured to input the document image sample set into the first sub-network, and train the first sub-network using the loss function of the document anti-counterfeiting detection model, so as to train the document anti-counterfeiting detection model's ability to locate tampered areas and detect targets by training the first sub-network;
[0112] The second sub-network training module 503 is configured to input the document image sample set into the second sub-network, and train the second sub-network using the loss function of the document anti-counterfeiting detection model, so as to train the document anti-counterfeiting detection model to distinguish the tampered area from the image background area by training the second sub-network.
[0113] exist Figure 5 In one embodiment of the apparatus shown in this specification, the first sub-network training module 502 is configured to perform:
[0114] For each document image in the document image sample set, the following is performed:
[0115] The first feature map is obtained by downsampling encoding the current document image using the first segmentation-based sub-network;
[0116] The first feature map is decoded based on upsampling to obtain the second feature map;
[0117] Based on the prediction results of the first sub-network of the tampered region in the second feature map and the label of the current document image, the first loss function corresponding to the first sub-network is adjusted.
[0118] exist Figure 5 In one embodiment of the apparatus shown in this specification, the first sub-network training module 502 is configured to perform:
[0119] By upsampling the first feature map of the ID card image at different multiples in successive stages, the relationship between the first feature map of the ID card image at different feature layers is learned.
[0120] For the features of the first feature map of the document image at each feature layer, the features at that feature layer are divided into N*N image feature regions. An attention mechanism is used to learn the relationship between different image feature regions of the first feature map of the document image at the same feature layer and the weights of different image regions at the same feature layer.
[0121] exist Figure 5 In one embodiment of the apparatus shown in this specification, the first sub-network training module 502 is configured to perform: dot product of the N*N attention matrix with the N*N image feature regions.
[0122] exist Figure 5 In one embodiment of the apparatus shown in this specification, the second sub-network training module 503 is configured to perform:
[0123] For each document image in the document image sample set, the following is performed:
[0124] The third feature map of the current document image is obtained by the second sub-network based on block consistency contrastive learning;
[0125] Each element in the third feature map is compared with all elements in the third feature map in pairs. If, based on the label of the current document image, it is determined that the image regions of the elements being compared in pairs are either both tampered with or both are not tampered with, then the comparison result is the first category. If, based on the label of the current document image, it is determined that one image region of the elements being compared in pairs is tampered with and the other image region is not tampered with, then the comparison result is the second category. After the comparison is completed, a block consistency category map is obtained.
[0126] Based on the prediction results of the tampered region of the third feature map by the second sub-network and the block consistency category map, the second loss function corresponding to the second sub-network is adjusted.
[0127] The segmentation-based first sub-network and the block consistency-based contrastive learning-based second sub-network share the backbone of the document anti-counterfeiting detection model; wherein, the backbone uses a pre-trained large graph-text model.
[0128] The first sub-network training module 502 and the second sub-network training module 503 are configured to freeze the encoding parameters of the pre-trained graph-text large model and train the parameters of the first sub-network and the second sub-network when training the segmentation-based first sub-network and the block consistency contrastive learning-based second sub-network.
[0129] exist Figure 5 In one embodiment of the device shown in this specification, the first sub-network is a UNet network.
[0130] exist Figure 5 In one embodiment of the device shown in this specification, the loss function of the document anti-counterfeiting detection model is L, L = a*L1 + L2, where L1 is the first loss function corresponding to the first sub-network, L2 is the second loss function corresponding to the second sub-network, and a is the weight of the first sub-network based on the segmentation.
[0131] This specification also provides an anti-counterfeiting detection device for identification documents, wherein, see [link to documentation]. Figure 6 The device includes:
[0132] The document image acquisition module 601 is configured to obtain the document image to be detected;
[0133] Input module 602 is configured to input the image of the document to be detected into the document anti-counterfeiting detection model;
[0134] The detection result acquisition module 603 is configured to obtain the anti-counterfeiting recognition result of the document anti-counterfeiting detection model output by the document anti-counterfeiting detection model; wherein, if the output document image to be detected includes a prominent area, the prominent area represents the tampered area.
[0135] exist Figure 6 In one embodiment of the apparatus shown in this specification, the detection result acquisition module 603 is configured to perform:
[0136] The output of the first sub-network in the document anti-counterfeiting detection model is used as the anti-counterfeiting recognition result of the document image to be detected.
[0137] or,
[0138] The intersection of the outputs of the first sub-network and the second sub-network in the document anti-counterfeiting detection model is obtained, and this intersection is used as the anti-counterfeiting recognition result of the document image to be detected.
[0139] or,
[0140] The union of the outputs of the first sub-network and the second sub-network in the document anti-counterfeiting detection model is obtained, and this union is used as the anti-counterfeiting recognition result of the document image to be detected.
[0141] It should be noted that the above-mentioned devices are typically implemented on the server side. They can be set up on independent servers, or some or all of the devices can be combined and installed on the same server. This server can be a single server or a server cluster consisting of multiple servers. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system. The above-mentioned devices can also be implemented on computer terminals with strong computing capabilities.
[0142] This specification provides, in one embodiment, a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the methods of any embodiment in the specification.
[0143] This specification provides a computing device according to one embodiment, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to perform the method of any embodiment of the specification.
[0144] It is understood that the structures illustrated in the embodiments of this specification do not constitute a specific limitation on the apparatus of the embodiments of this specification. In other embodiments of the specification, the above-described apparatus may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
[0145] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0146] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, widgets, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.
[0147] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.
Claims
1. Training method for document anti-counterfeiting detection model, wherein, The structure of the document anti-counterfeiting detection model includes: a first sub-network based on segmentation and a second sub-network based on block consistency contrastive learning; and the loss function of the document anti-counterfeiting detection model includes: a weighted sum of the first loss function corresponding to the first sub-network and the second loss function corresponding to the second sub-network; the method includes: Obtain a sample set of document images; this sample set includes multiple document images and a label for each document image; wherein, the label is used to mark the tampered areas in the document images; The set of document image samples is input into the first sub-network, and the first sub-network is trained using the loss function of the document anti-counterfeiting detection model. The first sub-network is then used to train the document anti-counterfeiting detection model’s ability to locate tampered areas and detect targets. The sample set of document images is input into the second sub-network, and the loss function of the document anti-counterfeiting detection model is used to train the second sub-network so as to train the document anti-counterfeiting detection model to distinguish the tampered area from the image background area. The training of the second sub-network includes: For each document image in the document image sample set, the following is executed: The third feature map of the current document image is obtained by the second sub-network based on block consistency contrastive learning; Each element in the third feature map is compared with all elements in the third feature map in pairs. If, based on the label of the current document image, it is determined that the image regions of the elements being compared in pairs are either both tampered with or both are not tampered with, then the comparison result is the first category. If, based on the label of the current document image, it is determined that one image region of the elements being compared in pairs is tampered with and the other image region is not tampered with, then the comparison result is the second category. After the comparison is completed, a block consistency category map is obtained. Based on the prediction results of the tampered region of the third feature map by the second sub-network and the block consistency category map, the second loss function corresponding to the second sub-network is adjusted.
2. The method according to claim 1, wherein, The training of the first sub-network includes: For each document image in the document image sample set, the following is executed: The first feature map is obtained by downsampling encoding the current document image using the first segmentation-based sub-network; The first feature map is decoded based on upsampling to obtain the second feature map; Based on the prediction results of the first sub-network of the tampered region in the second feature map and the label of the current document image, the first loss function corresponding to the first sub-network is adjusted.
3. The method according to claim 2, wherein during the training of the segmentation-based first sub-network, the upsampling-based decoding includes: By upsampling the first feature map of the ID card image at different multiples in successive stages, the relationship between the first feature map of the ID card image at different feature layers is learned. For the features of the first feature map of the document image at each feature layer, the features at that feature layer are divided into N*N image feature regions. An attention mechanism is used to learn the relationship between different image feature regions of the first feature map of the document image at the same feature layer and the weights of different image regions at the same feature layer.
4. The method according to claim 3, wherein, The learning through attention mechanisms includes: The N*N attention matrix is multiplied by the N*N image feature regions.
5. The method according to claim 1, wherein, The segmentation-based first sub-network and the block consistency-based contrastive learning-based second sub-network share the backbone of the document anti-counterfeiting detection model; wherein, the backbone uses a pre-trained large graph-text model. Training the segmentation-based first sub-network and the block consistency contrastive learning-based second sub-network further includes: freezing the encoding parameters of the pre-trained graph-text large model and training the parameters of the first sub-network and the second sub-network.
6. Methods for detecting counterfeit documents, including, The method includes: Obtain the image of the document to be inspected; The image of the document to be detected is input into the document anti-counterfeiting detection model; wherein the document anti-counterfeiting detection model is trained using the training method of any one of the document anti-counterfeiting detection models in claims 1 to 5; The anti-counterfeiting recognition result of the document anti-counterfeiting detection model is obtained; wherein, if the output document image to be detected includes a prominent area, the prominent area represents the tampered area.
7. The method according to claim 6, wherein, The anti-counterfeiting recognition result of the document anti-counterfeiting detection model outputting the document image to be detected includes: The output of the first sub-network in the document anti-counterfeiting detection model is used as the anti-counterfeiting recognition result of the document image to be detected. or, The intersection of the outputs of the first sub-network and the second sub-network in the document anti-counterfeiting detection model is obtained, and this intersection is used as the anti-counterfeiting recognition result of the document image to be detected. or, The union of the outputs of the first sub-network and the second sub-network in the document anti-counterfeiting detection model is obtained, and this union is used as the anti-counterfeiting recognition result of the document image to be detected.
8. A training device for a document anti-counterfeiting detection model, wherein, The structure of the document anti-counterfeiting detection model includes: a first sub-network based on segmentation and a second sub-network based on block consistency contrastive learning; the loss function of the document anti-counterfeiting detection model includes: a weighted sum of the first loss function corresponding to the first sub-network and the second loss function corresponding to the second sub-network; the device includes: The sample acquisition module is configured to obtain a set of document image samples; the set of document image samples includes multiple document images and a label for each document image; wherein, the label is used to mark the tampered areas in the document images; The first sub-network training module is configured to input the document image sample set into the first sub-network and train the first sub-network using the loss function of the document anti-counterfeiting detection model, so as to train the document anti-counterfeiting detection model's ability to locate tampered areas and detect targets by training the first sub-network; The second sub-network training module is configured to input the document image sample set into the second sub-network and train the second sub-network using the loss function of the document anti-counterfeiting detection model, so as to train the document anti-counterfeiting detection model to distinguish the tampered area from the image background area by training the second sub-network. The second sub-network training module is configured to perform: For each document image in the document image sample set, the following is executed: The third feature map of the current document image is obtained by the second sub-network based on block consistency contrastive learning; Each element in the third feature map is compared with all elements in the third feature map in pairs. If, based on the label of the current document image, it is determined that the image regions of the elements being compared in pairs are either both tampered with or both are not tampered with, then the comparison result is the first category. If, based on the label of the current document image, it is determined that one image region of the elements being compared in pairs is tampered with and the other image region is not tampered with, then the comparison result is the second category. After the comparison is completed, a block consistency category map is obtained. Based on the prediction results of the tampered region of the third feature map by the second sub-network and the block consistency category map, the second loss function corresponding to the second sub-network is adjusted.
9. Document anti-counterfeiting detection device, among which, The device includes: The document image acquisition module is configured to obtain the document image to be detected; The input module is configured to input the image of the document to be detected into the document anti-counterfeiting detection model; wherein the document anti-counterfeiting detection model is trained using the training device of the document anti-counterfeiting detection model of claim 8; The detection result acquisition module is configured to obtain the anti-counterfeiting recognition result of the document anti-counterfeiting detection model output by the document anti-counterfeiting detection model; wherein, if the output document image to be detected includes a prominent area, the prominent area represents the tampered area.
10. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-7.
11. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-7.