Certificate anti-counterfeiting detection method and device, computer readable storage medium, electronic equipment and computer program product

Abstract

The embodiment of the invention discloses a certificate anti-counterfeiting detection method and device, a computer readable storage medium, electronic equipment and a computer program product, and the method comprises the steps: obtaining a first certificate image obtained by carrying out image collection for a target certificate of a user, and carrying out the processing of the first certificate image, and taking the generated voucher noise feature map or the voucher spectrogram as a second voucher image, and carrying out collaborative anti-counterfeiting detection on the first voucher image, the second voucher image and other multi-modal voucher images by using a voucher anti-counterfeiting detection model to obtain an anti-counterfeiting detection result for the first voucher image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a credential anti-counterfeiting detection method, device, computer-readable storage medium, electronic device, and computer program product. Background Art

[0002] With the rapid development of science and technology, the threshold for image forgery is getting lower and lower. In the process of obtaining services, some users may submit forged credential images to service providers in order to achieve personal goals. The behavior of these users submitting forged credential images may involve risks such as fraud and illegality, thereby causing potential risks and losses to society, enterprises or others. At present, service providers usually arrange staff to carefully detect and identify the authenticity of credential images submitted by users. Some service providers have also begun to use deep learning algorithms to train models to analyze the entire credential image submitted by users to distinguish between real credential images and forged credential images. However, the accuracy of the anti-counterfeiting detection results generated for credential images by the above scheme is limited and may not be able to protect the rights and interests of all parties.

[0003] Based on this, how to improve the accuracy of anti-counterfeiting detection results for user credential images has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of this specification provide a method, device, computer-readable storage medium, electronic device and computer program product for credential anti-counterfeiting detection. Anti-counterfeiting detection is performed by combining credential images in the spatial domain and corresponding credential noise feature maps or credential spectrum maps and other multi-modal credential images, which is beneficial to improving the accuracy and effectiveness of the credential anti-counterfeiting detection solution.

[0005] An embodiment of the present specification provides a method for detecting credential anti-counterfeiting, the method comprising:

[0006] Acquire a first credential image obtained by performing image acquisition on a target credential of a user;

[0007] Processing the first voucher image to obtain a second voucher image, wherein the second voucher image includes: any one of a voucher noise feature image and a voucher spectrum image;

[0008] Performing anti-counterfeiting detection processing on the first credential image and the second credential image using a credential anti-counterfeiting detection model to obtain a first anti-counterfeiting detection result for the first credential image output by the credential anti-counterfeiting detection model;

[0009] Among them, the credential anti-counterfeiting detection model includes: a multimodal feature encoder and a classifier connected in sequence.

[0010] The embodiment of this specification also provides a credential anti-counterfeiting detection device, including:

[0011] A first acquisition module is used to acquire a first credential image obtained by performing image acquisition on a target credential of a user;

[0012] A second acquisition module is used to process the first voucher image to obtain a second voucher image, where the second voucher image includes: any one of a voucher noise feature image and a voucher spectrum image;

[0013] The anti-counterfeiting detection module is used to perform anti-counterfeiting detection processing on the first credential image and the second credential image using a credential anti-counterfeiting detection model to obtain a first anti-counterfeiting detection result for the first credential image output by the credential anti-counterfeiting detection model; wherein the credential anti-counterfeiting detection model includes: a multimodal feature encoder and a classifier connected in sequence.

[0014] The embodiments of this specification also provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0015] An embodiment of the present specification also provides an electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the above method.

[0016] The embodiments of the present specification also provide a computer program product, on which at least one instruction is stored, and when the at least one instruction is executed by a processor, the steps of the above method are implemented.

[0017] In the embodiment of the present specification, when anti-counterfeiting detection is performed on the first credential image of the user's target credential in the spatial domain, the first credential image can be processed to generate a credential noise feature map or a credential spectrum map as the second credential image. Since the credential noise feature map and the credential spectrum map can reflect the forgery traces in the first credential image,

[0018] Therefore, the credential anti-counterfeiting detection model can be used to collaboratively process multimodal credential images such as the first credential image and the second credential image, which is conducive to improving the accuracy and effectiveness of the anti-counterfeiting detection results generated for the first credential image, reducing the risks brought by users using forged credential images, and protecting the rights and interests of all parties. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of an application scenario of a credential anti-counterfeiting detection solution provided in an embodiment of this specification;

[0020] Figure 2A schematic diagram of a flow chart of a credential anti-counterfeiting detection method provided in an embodiment of this specification;

[0021] Figure 3 A schematic diagram of the structure of a credential anti-counterfeiting detection model provided in an embodiment of this specification;

[0022] Figure 4 A schematic diagram of the structure of another credential anti-counterfeiting detection model provided in the embodiments of this specification;

[0023] Figure 5 A schematic diagram of the structure of another credential anti-counterfeiting detection model provided in the embodiments of this specification;

[0024] Figure 6 A schematic diagram of the structure of another credential anti-counterfeiting detection model provided in the embodiments of this specification;

[0025] Figure 7 A schematic diagram of the structure of a credential anti-counterfeiting detection device provided in an embodiment of this specification;

[0026] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0028] When service providers provide services to users, there are cases where users submit forged credential images to achieve personal purposes. For example, users may forge identity certificates, academic certificates, income certificates, etc. for fraudulent purposes to obtain loans or job opportunities. Some users may also forge personal address certificates, occupation certificates, etc. to protect their private data for personal privacy protection or other reasons. In addition, some users may forge identity certificates or impersonate others to engage in illegal activities or evade supervision. In general, service providers need to accurately identify the credential images submitted by users in order to protect the rights and interests of all parties and the stable operation of the business.

[0029] At present, service providers usually arrange staff to carefully detect and identify the authenticity of credential images submitted by users. Some service providers have also begun to use deep learning algorithms to train models to detect the authenticity of credential images in RGB format. However, people have begun to use some auxiliary means (such as tampering boundary repair) to fine-tune the forged credential images, making the tampering traces in the forged credential images in RGB format unintuitive, in order to improve the tampering concealment of the forged credential images in RGB format, which easily leads to the inability of existing credential anti-counterfeiting detection solutions to accurately identify forged credential images in RGB format.

[0030] See also Figure 1 , which is a schematic diagram of an application scenario of a credential anti-counterfeiting detection solution provided in an embodiment of this specification.

[0031] like Figure 1 As shown, in the process of obtaining services, the user can use the user device 101 to send the first credential image 103 obtained by performing image acquisition on the target credential of the user to the service provider's device 102. When the service provider uses the device 102 to perform credential anti-counterfeiting detection on the first credential image 103, the first credential image 103 can be processed first to obtain the credential noise feature map or credential spectrum map as the second credential image.

[0032] A pre-trained credential anti-counterfeiting detection model 104 may also be deployed at the device 102, and the credential anti-counterfeiting detection model 104 may include: a multimodal feature encoder and a classifier connected in sequence, so that the credential anti-counterfeiting detection model 104 can be used to perform anti-counterfeiting detection processing on the first credential image 103 and the second credential image, and obtain a first anti-counterfeiting detection result with good accuracy for the first credential image 103 output by the credential anti-counterfeiting detection model 104.

[0033] See also Figure 2 , is a flow chart of a method for detecting credential anti-counterfeiting provided in an embodiment of this specification. The execution subject of the process may be a device for detecting credential anti-counterfeiting, or an application for detecting credential anti-counterfeiting carried by the device. Figure 2 The process shown is described in detail, and the credential anti-counterfeiting detection method may specifically include the following steps:

[0034] Step 202: Acquire a first credential image obtained by performing image acquisition on a target credential of a user.

[0035] In the embodiment of this specification, the first credential image of the user may be an image of the user credential that needs to be tested for anti-counterfeiting in the spatial domain, for example, an image in RGB color mode, a grayscale image, etc. In practical applications, there may be multiple types of target credentials, and there may also be multiple types of corresponding first credential images. For example, the first credential image may include, but is not limited to: identity credential images, education credential images, payment result credential images, asset credential images, real estate credential images, etc., without specific limitation.

[0036] Step 204: Process the first voucher image to obtain a second voucher image, where the second voucher image includes: any one of a voucher noise feature image and a voucher spectrum image.

[0037] In the embodiments of this specification, since the noise features between different regions in the forged voucher image often do not match when part of the image area in the voucher image is cut, moved, modified, or covered or filled with other images, the voucher noise feature map corresponding to the first voucher image can be used as the second voucher image to perform voucher anti-counterfeiting detection in combination with the second voucher image. The voucher noise feature map can refer to a feature map (Feature Map) obtained by performing noise feature extraction processing on the first voucher image.

[0038] Furthermore, since different regions in a forged voucher image usually undergo different compression rounds, and the voucher image in the frequency domain can reflect the difference in compression rounds, the voucher spectrum corresponding to the first voucher image can be used as the second voucher image to perform voucher anti-counterfeiting detection in combination with the second voucher image. The voucher spectrum can refer to a spectrum in the frequency domain obtained by performing frequency domain transformation processing on the first voucher image.

[0039] Step 206, using a credential anti-counterfeiting detection model to perform anti-counterfeiting detection processing on the first credential image and the second credential image, and obtain a first anti-counterfeiting detection result for the first credential image output by the credential anti-counterfeiting detection model; wherein the credential anti-counterfeiting detection model includes: a multimodal feature encoder and a classifier connected in sequence.

[0040] In the embodiment of the present specification, the credential anti-counterfeiting detection model may be a classification model pre-trained using a credential sample carrying label data reflecting whether the credential sample is a credential sample. Specifically, the multimodal feature encoder in the credential anti-counterfeiting detection model can perform feature extraction and feature fusion processing on the received first credential image and the second credential image, so as to send the generated credential image encoding data to the classifier, so that the classifier can obtain the first anti-counterfeiting detection result that can be used to reflect whether the first credential image is a credential image by classifying the credential image encoding data, which is convenient and quick.

[0041] Figure 2 According to the method, when anti-counterfeiting detection is performed on a first credential image of a user's target credential in a spatial domain, the first credential image can be processed to use the generated credential noise feature map or credential spectrum map as the second credential image. Since the credential noise feature map and the credential spectrum map can reflect the forgery traces in the first credential image, the credential anti-counterfeiting detection model can be used to collaboratively process multimodal credential images such as the first credential image and the second credential image, which is beneficial to improving the accuracy and effectiveness of the anti-counterfeiting detection result generated for the first credential image, reducing the risk of users using forged credential images, and protecting the rights and interests of all parties.

[0042] based on Figure 2 The method in this specification also provides some specific implementation plans of the method, which are described below.

[0043] For ease of understanding, the structure and working principle of the credential anti-counterfeiting detection model are explained here.

[0044] Optionally, the multimodal feature encoder may include: a first high-resolution network module, a second high-resolution network module and a feature fusion module; the first high-resolution network module and the second high-resolution network module may be connected to the feature fusion module respectively, and the feature fusion module may be connected to the classifier.

[0045] The first high-resolution network module may be used to perform feature extraction processing on the first voucher image to obtain feature data of the first voucher image at multiple scales.

[0046] The second high-resolution network module can be used to perform feature extraction processing on the second voucher image to obtain second voucher image feature data of multiple different scales.

[0047] The feature fusion module may be used to perform feature fusion processing on the plurality of first voucher image feature data of different scales and the plurality of second voucher image feature data of different scales to obtain third voucher image feature data.

[0048] The classifier may be used to perform classification processing based on the third credential image feature data to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

[0049] In the embodiments of this specification, the high-resolution network (HRNet) can run various workflows for processing images of different resolutions in parallel, and enhance the final feature representation by fusing the extracted image feature maps of different resolutions. This design enables the network to have the powerful power of deep learning while maintaining high resolution. Therefore, the high-resolution network can be used to build a multimodal feature encoder. Specifically, the multimodal feature encoder can include a first high-resolution network and a second high-resolution network to perform feature extraction processing on the first voucher image and the second voucher image, respectively, which is conducive to ensuring the accuracy of the extracted first voucher image feature data and the second voucher image feature data.

[0050] In the embodiments of the present specification, the credential anti-counterfeiting detection model may also include a feature fusion module to perform feature fusion processing on the first credential image feature data and the second credential image feature data, and input the third credential image feature data obtained by the feature fusion into a classifier, so that the classifier can determine the probability value of the first credential image belonging to a credible / forged credential image, and then obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

[0051] Optionally, the first high-resolution network module included in the multimodal feature encoder may be an RGB Stream module in a compressed artifact tracking network model; and the second high-resolution network module may be a DCT Stream module in a compressed artifact tracking network model.

[0052] The RGB Stream module may be used to perform feature extraction processing on the first voucher image to obtain feature data of the first voucher image at multiple different scales.

[0053] The DCT Stream module may be used to perform feature extraction processing on the second voucher image to obtain a plurality of second voucher image feature data of different scales.

[0054] In the embodiments of this specification, the compression artifact tracking network model (CAT-Net) is a neural network model suitable for detecting and locating the splicing area in the image. The compression artifact tracking network can generally include an RGB Stream module and a DCT Stream module built based on a high-resolution network (HRNet), wherein the RGB Stream module can perform feature extraction processing on the image in the spatial domain to learn the visual artifact features, and the DCT Stream module can perform feature extraction processing on the spectrum graph in the frequency domain to learn the compression artifact features, thereby jointly learning the image forgery features in multiple modalities such as the spatial domain and the frequency domain to accurately identify and segment the forged area in the image.

[0055] In practical applications, since the input data of the DCT Stream module is a spectrum graph, and both the spectrum graph and the noise feature graph are two-dimensional data, and the formats of the two can also be consistent, when the noise feature graph is input into the DCT Stream module, the DCT Stream module can perform feature extraction and analysis on the noise feature graph to learn the image tampering features reflected in the noise feature graph.

[0056] Based on this, the RGB Stream module and the DCT Stream module in the compressed artifact tracking network model (CAT-Net) can be used as the first high-resolution network module and the second high-resolution network module in the multimodal feature encoder, respectively, to perform feature extraction processing on the first voucher image and the second voucher image, which is beneficial to improve the accuracy of the extracted first voucher image feature data and the second voucher image feature data.

[0057] In the embodiments of this specification, there are many ways to implement the feature fusion module included in the multimodal feature encoder. For ease of understanding, this is explained below.

[0058] In one implementation, the feature fusion module included in the multimodal feature encoder may be a Fusion Stage module in a compression artifact tracking network model. The Fusion Stage module may be used to perform feature fusion processing on the first voucher image feature data of multiple different scales and the second voucher image feature data of multiple different scales to obtain the third voucher image feature data.

[0059] In the embodiment of the present specification, since the first high-resolution network module and the second high-resolution network module in the multimodal feature encoder can be implemented by the RGB Stream module and the DCT Stream module in the compressed artifact tracking network model (CAT-Net), and the Fusion Stage module included in the compressed artifact tracking network model (CAT-Net) has the function of performing feature fusion processing on each feature data output by the RGB Stream module and the DCT Stream module, the Fusion Stage module can be used as a feature fusion module in the multimodal feature encoder, so as to perform feature fusion processing on the first voucher image feature data and the second voucher image feature data output by the first high-resolution network module and the second high-resolution network module in the multimodal feature encoder. It can be understood that at this time, the structure of the multimodal feature encoder and the compressed artifact tracking network model can be consistent, which is not only conducive to simplifying the construction process of the voucher anti-counterfeiting detection model, but also conducive to ensuring the accuracy of the third voucher image feature data extracted by the multimodal feature encoder.

[0060] For ease of understanding, Figure 3 A schematic diagram of the structure of a credential anti-counterfeiting detection model provided in an embodiment of this specification, such as Figure 3 As shown, the multimodal feature encoder in the voucher anti-counterfeiting detection model may include: a first high-resolution network module 31, a second high-resolution network module 32 and a feature fusion module 33, and the above three modules may be: an RGB Stream module, a DCT Stream module and a Fusion Stage module in CAT-Net. At this time, each output layer of the first high-resolution network module 31 and each output layer of the second high-resolution network module 32 may be connected to the feature fusion module 33, and the output layer of the feature fusion module 33 may be connected to the classifier 34.

[0061] Implementation method two, the feature fusion module contained in the multimodal feature encoder may include: a first Transformer sub-module, a second Transformer sub-module, a Class Attention sub-module and a multi-scale feature fusion sub-module, and the multi-scale feature fusion sub-module may be a branch network in the Fusion Stage module in the compressed artifact tracking network model for performing feature fusion processing on the multi-scale feature map.

[0062] The output layers of the first high-resolution network module except the output layer for outputting the first voucher image feature data with the largest scale can be connected to the first Transformer sub-module, the output layers of the second high-resolution network module can be connected to the second Transformer sub-module, the first Transformer sub-module and the second Transformer sub-module can be connected to the Class Attention sub-module, the output layer for outputting the first voucher image feature data with the largest scale and the Class Attention sub-module can be connected to the multi-scale feature fusion sub-module, and the multi-scale feature fusion sub-module can also be connected to the classifier.

[0063] The first Transformer submodule may be used to perform feature fusion processing on the first voucher image feature data at other scales except the maximum scale to obtain fourth voucher image feature data.

[0064] The second Transformer submodule may be used to perform feature fusion processing on the second voucher image feature data of each scale to obtain fifth voucher image feature data.

[0065] The Class Attention submodule may be used to perform feature fusion processing on the fourth voucher image feature data and the fifth voucher image feature data to obtain sixth voucher image feature data.

[0066] The multi-scale feature fusion submodule may be used to perform feature fusion processing on the first voucher image feature data with the largest scale and the sixth voucher image feature data to obtain the third voucher image feature data.

[0067] In the embodiments of this specification, both the first Transformer submodule and the second Transformer submodule can be implemented using a Transformer model, or can be implemented by adaptively modifying the Transformer model according to actual needs, and no specific limitation is made to this. By using the first Transformer submodule to perform feature fusion processing on the first voucher image feature data of other scales except the maximum scale, fourth voucher image feature data with good accuracy can be obtained. And, by using the second Transformer submodule to perform feature fusion processing on the second voucher image feature data of each scale, fifth voucher image feature data with good accuracy can be obtained.

[0068] Class Attention (CA) is an attention mechanism used for visual Transformer in CaiT, which aims to extract information from a set of processed tiles. Therefore, in order to improve the feature fusion effect, the feature fusion module can also include a Class Attention submodule, so that the Class Attention submodule is used to first perform a feature fusion process on the fourth voucher image feature data and the fifth voucher image feature data output by the first Transformer submodule and the second Transformer submodule to obtain the sixth voucher image feature data.

[0069] Furthermore, since it is necessary to perform a feature fusion process on the first voucher image feature data and the sixth voucher image feature data with the largest scale in order to achieve fusion of all the first voucher image feature data and all the second voucher image feature data, the feature fusion module can also include a multi-scale feature fusion submodule, so as to generate the third voucher image feature data according to the first voucher image feature data and the sixth voucher image feature data. In practical applications, the multi-scale feature fusion submodule can be implemented by using a branch network in the FusionStage module in the compressed artifact tracking network model for feature fusion processing on the multi-scale feature map, which is convenient and fast. Of course, the multi-scale feature fusion submodule can also be implemented by other network structures (for example, multi-head attention modules, fully connected layers, etc.), and there is no specific limitation on this.

[0070] For ease of understanding, Figure 4 A schematic diagram of another credential anti-counterfeiting detection model provided in the embodiments of this specification, such as Figure 4 As shown, the multimodal feature encoder in the voucher anti-counterfeiting detection model may include: a first high-resolution network module 41, a second high-resolution network module 42 and a feature fusion module 43. The feature fusion module 43 may include: a first Transformer submodule 401, a second Transformer submodule 402, a Class Attention submodule 403 and a multi-scale feature fusion submodule 404. It is worth noting that the output layer of the first high-resolution network module 41 for outputting the first voucher image feature data with the largest scale and the output layer of the Class Attention submodule 403 can be connected to different network layers at the multi-scale feature fusion submodule 404, and of course, can also be connected to the same network layer at the multi-scale feature fusion submodule 404, and there is no specific limitation on this.

[0071] In the embodiments of this specification, there may be multiple ways to implement the classifier in the credential anti-counterfeiting detection model. For ease of understanding, this is explained below.

[0072] In implementation mode 1, the classifier may include: a first classification head, a second classification head and a third classification head.

[0073] The output layer of the feature fusion module can be connected to the first classification head, the output layer of the second high-resolution network module for outputting the second credential image feature data with the largest scale can be connected to the second classification head, and both the first classification head and the second classification head can be connected to the third classification head; wherein, the output layer of the feature fusion module can be the output layer of the Fusion Stage module or the output layer of the multi-scale feature fusion sub-module.

[0074] The first classification head may be used to perform classification processing on the third voucher image feature data to obtain a first classification probability value.

[0075] The second classification head may be used to perform classification processing on the second voucher image feature data with the largest scale to obtain a second classification probability value.

[0076] The third classification head may be used to perform classification processing on the first classification probability value and the second classification probability value to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

[0077] In the embodiments of this specification, the classification head may refer to the part of the network model that is specifically used to map the learned feature representation to the category prediction. It is usually located at the end of the network and immediately follows the feature extraction layer. Therefore, the classification head can be used to build the classifier in the credential anti-counterfeiting detection model.

[0078] Since the third credential image feature data is obtained by fusing all feature data extracted from the first credential image and the second credential image, the image information contained in the third credential image feature data is relatively comprehensive, so that a first classification head can be set to perform classification processing according to the third credential image feature data to obtain a first classification probability value of whether the first credential image belongs to a credible / forged credential image.

[0079] Since the second voucher image feature data with the largest scale outputted from the second high-resolution network module can better reflect the image forgery traces extracted from the voucher noise feature map or the voucher spectrum map, a second classification head can be set to perform classification processing according to the second voucher image feature data with the largest scale to obtain a second classification probability value of the first voucher image belonging to a credible / forged voucher image.

[0080] Furthermore, a third classification head may be set to classify the first classification probability value and the second classification probability value to obtain a first anti-counterfeiting detection result that reflects whether the first credential image belongs to a trusted credential image. This is not only conducive to making full use of the extracted credential image feature data, but also conducive to improving the accuracy of the generated first anti-counterfeiting detection result.

[0081] In a second implementation mode, the credential anti-counterfeiting detection model may further include: an edge perception module in a boundary guidance network model; and the classifier may further include: a fourth classification head.

[0082] The output layer of the first high-resolution network module for outputting the first credential image feature data with the largest scale can also be connected to the input layer of the edge perception module, the output layer of the edge perception module can be connected to the fourth classification head in the classifier, and the fourth classification head can also be connected to the third classification head.

[0083] The edge perception module may be used to perform feature extraction processing on the first voucher image feature data with the largest scale to obtain seventh voucher image feature data.

[0084] The fourth classification head may be used to perform classification processing on the seventh voucher image feature data to obtain a third classification probability value.

[0085] The third classification header can be specifically used to perform classification processing on the first classification probability value, the second classification probability value and the third classification probability value to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

[0086] In the embodiments of this specification, the boundary-guided network (BGNet) can improve the performance of camouflaged target detection through edge semantics. The edge-aware module (EAM) of the boundary-guided network can mine edge semantics related to the object from low-level features containing local edge details and high-level features containing global position information under object boundary supervision to guide feature learning, thereby enhancing boundary representation.

[0087] Based on this, an edge perception module (EAM) can be set in the credential anti-counterfeiting detection model. By utilizing the edge perception module to process the visual feature data of the credential image (for example, the first credential image feature data with the largest scale), abnormal edge traces existing in the forged credential image due to image tampering operations can be detected.

[0088] Furthermore, since the seventh credential image feature data output by the edge perception module can reflect traces of image forgery, a fourth classification head can also be set to perform classification processing on the seventh credential image feature data to obtain a third classification probability value that the first credential image belongs to a credible / forged credential image.

[0089] At this time, the third classification head can perform classification processing according to the first classification probability value, the second classification probability value and the third classification probability value to obtain a first anti-counterfeiting detection result that reflects whether the first credential image belongs to a trusted credential image. This is not only conducive to making full use of the extracted credential image feature data, but also conducive to improving the accuracy of the generated first anti-counterfeiting detection result.

[0090] In practical applications, the convolutional generalized average pooling layer (ConvGem) can capture the spatial correlation of global pixels, thereby reducing false detections caused by noise. Based on this, the first to fourth classification heads can be implemented using the convolutional generalized average pooling layer (ConvGem), which is beneficial to improving the accuracy of the first anti-counterfeiting detection result for the first credential image generated by the classifier. Of course, the above classification heads can also be implemented using a fully connected layer or other network structures, which are not specifically limited.

[0091] For ease of understanding, Figure 3 The specific structure of the classifier 34 is also shown in the certificate anti-counterfeiting detection model provided in , and it can be seen that the first classification head, the second classification head and the fourth classification head in the classifier 34 can be connected to the output layer of the feature fusion module 33, the output layer of the second high-resolution network module 32 for outputting the second certificate image feature data with the largest scale, and the output layer of the edge perception module 35. In addition, the output ends of the first classification head, the second classification head and the fourth classification head can also be connected to the third classification head, which will not be described in detail.

[0092] Figure 4 The specific structure of the classifier 44 is also shown in the certificate anti-counterfeiting detection model provided in , and it can be seen that the first classification head, the second classification head and the fourth classification head in the classifier 44 can be connected to the output layer of the feature fusion module 43, the output layer of the second high-resolution network module 42 for outputting the second certificate image feature data with the largest scale, and the output layer of the edge perception module 45. In addition, the output ends of the first classification head, the second classification head and the fourth classification head can also be connected to the third classification head, which will not be described in detail.

[0093] In practical applications, only the first classification head, the third classification head and the fourth classification head may be provided without providing the second classification head, which has good flexibility and is not specifically limited thereto.

[0094] Implementation method three, the credential anti-counterfeiting detection model may also include: a confidence decoder and a weighted pooling module in the TruFor model; the classifier may include: a forgery detector in the TruFor model.

[0095] Each output layer at the first high-resolution network module and each output layer at the second high-resolution network module can also be connected to the confidence decoder, the confidence decoder and the feature fusion module can be connected to the weighted pooling module, and the weighted pooling module can be connected to the forgery detector.

[0096] The confidence decoder may be used to perform feature extraction processing and feature fusion processing on the first voucher image feature data at each scale and the second voucher image feature data at each scale to obtain eighth voucher image feature data.

[0097] The weighted pooling module may be configured to determine a target weight required for weighted pooling of the third voucher image feature data based on the eighth voucher image feature data, so as to perform weighted pooling processing on the third voucher image feature data according to the target weight to obtain ninth voucher image feature data.

[0098] The forgery detector may be used to perform detection processing on the ninth credential image feature data to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

[0099] In the embodiments of this specification, TruFor is a model for detecting and locating image forgery. The TruFor model can detect forgery based on the visual features of the RGB image and the noise features extracted from the RGB image, and obtain a score representing the overall authenticity of the image from the forgery detector in the TruFor model with good accuracy.

[0100] In practical applications, since both the spectrum graph and the noise feature graph are two-dimensional data and their formats can be consistent, the spectrum graph can also be used to replace the noise feature graph required to be input into the TruFor model, so that the TruFor model can extract and fuse features of the spectrum graph and RGB image to learn the image tampering features in the spectrum graph and RGB image, thereby obtaining a good score that represents the overall authenticity of the image. Based on this, the TruFor architecture can also be used to build a credential anti-counterfeiting detection model.

[0101] Specifically, the first high-resolution network module and the second high-resolution network module in the multimodal feature encoder in the credential anti-counterfeiting detection model can be regarded as the encoder (Ecoder) in the TruFor model, and the feature fusion model in the multimodal feature encoder can be regarded as the anomaly decoder (Anomaly Decoder) in the TruFor model. And, since the functions required to be implemented by the classifier in the credential anti-counterfeiting detection model and the forgery detector (Forgery Detector) in the TruFor model can be consistent, the forgery detector in the TruFor model can be set as the classifier in the credential anti-counterfeiting detection model. In addition, it is also necessary to make the credential anti-counterfeiting detection model include key structures such as the confidence decoder (Confidence Decoder) and the weighted pooling module (Pooling) in the TruFor model, so that the constructed credential anti-counterfeiting detection model can be consistent with the TruFor model architecture, which is conducive to improving the accuracy of the credential anti-counterfeiting detection results generated by the constructed credential anti-counterfeiting detection model.

[0102] For ease of understanding, combined Figure 5 and Figure 6 The specific structure of the credential anti-counterfeiting detection model is explained.

[0103] like Figure 5 As shown, the multimodal feature encoder in the credential anti-counterfeiting detection model may include: a first high-resolution network module 51, a second high-resolution network module 52 and a feature fusion module 53, and the credential anti-counterfeiting detection model may also include: a confidence decoder 54 and a weighted pooling module 55 in the TruFor model. In addition, the classifier 56 of the credential anti-counterfeiting detection model may include a forgery detector in the TruFor model. Figure 5 The connection relationship between the various model structures is also shown in FIG.

[0104] like Figure 6 As shown, the multimodal feature encoder in the credential anti-counterfeiting detection model may include: a first high-resolution network module 61, a second high-resolution network module 62 and a feature fusion module 63, and the credential anti-counterfeiting detection model may also include: a confidence decoder 64 and a weighted pooling module 65 in the TruFor model. In addition, the classifier 66 of the credential anti-counterfeiting detection model may include a forgery detector in the TruFor model. Figure 6 The connection relationship between the various model structures is also shown in FIG.

[0105] In a fourth implementation, the classifier may further include: a fifth classification head, a sixth classification head, and a seventh classification head.

[0106] The output layer of the feature fusion module can also be connected to the fifth classification head, the output layer of the confidence decoder can also be connected to the sixth classification head, and the output layers of the fifth classification head, the sixth classification head and the forgery detector can all be connected to the seventh classification head; the output layer of the feature fusion module can be the output layer of the FusionStage module or the output layer of the multi-scale feature fusion sub-module.

[0107] The forgery detector may be specifically used to perform detection processing on the ninth credential image feature data to obtain a fourth classification probability value.

[0108] The fifth classification head may be used to perform classification processing on the third voucher image feature data to obtain a fifth classification probability value.

[0109] The sixth classification head may be used to perform classification processing on the eighth voucher image feature data to obtain a sixth classification probability value.

[0110] The seventh classification head may be used to perform classification processing on the fourth classification probability value, the fifth classification probability value, and the sixth classification probability value to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

[0111] In the embodiment of the present specification, since the forgery detector outputs a score representing the overall authenticity of the image, the score output by the forgery detector through detection and processing of the ninth credential image feature data can be directly used as the fourth classification probability value reflecting that the first credential image belongs to a forged / authentic credential image.

[0112] Since the output of the Fusion Stage module or the branch network for feature fusion processing for the multi-scale feature map at the Fusion Stage module is the segmentation result of the forged area in the image, when the output layer of the feature fusion module is the output layer of the Fusion Stage module, or the output layer of the branch network for feature fusion processing for the multi-scale feature map at the Fusion Stage module (i.e., the output layer of the multi-scale feature fusion submodule), the third credential image feature data output by the feature fusion module can reflect the forged area in the image. Therefore, in order to make full use of the data and further improve the credential anti-counterfeiting detection results, a fifth classification head can be set to classify the third credential image feature data, thereby obtaining a fifth classification probability value reflecting that the first credential image belongs to a forged / trusted credential image.

[0113] Since the confidence map output by the confidence decoder can reflect the possible false positive counterfeit areas in the credential image, and the real counterfeit areas and random anomalies can be distinguished by analyzing the confidence map, in order to make full use of the data and further improve the credential anti-counterfeiting detection results, a sixth classification head can be set to classify the eighth credential image feature data output by the confidence decoder, so as to obtain the sixth classification probability value reflecting that the first credential image belongs to a counterfeit / trusted credential image.

[0114] Subsequently, the seventh classification head can be used to classify the fourth classification probability value, the fifth classification probability value and the sixth classification probability value output by the forgery detector, which is beneficial to improving the accuracy of the first anti-counterfeiting detection result used to reflect whether the first credential image is a trusted credential image.

[0115] Implementation method five, the credential anti-counterfeiting detection model may also include: an edge perception module in a boundary guidance network model; the classifier may also include: an eighth classification head.

[0116] The output layer of the first high-resolution network module for outputting the first credential image feature data with the largest scale can also be connected to the input layer of the edge perception module, the output layer of the edge perception module can be connected to the eighth classification head in the classifier, and the eighth classification head can also be connected to the seventh classification head.

[0117] The edge perception module may be used to perform feature extraction processing on the first voucher image feature data with the largest scale to obtain tenth voucher image feature data.

[0118] The eighth classification head may be used to perform classification processing on the tenth voucher image feature data to obtain a seventh classification probability value.

[0119] The seventh classification header can be specifically used to perform classification processing on the fourth classification probability value, the fifth classification probability value, the sixth classification probability value and the seventh classification probability value to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

[0120] For ease of understanding, Figure 5 The specific structure and connection mode of the classifier 56 in the credential anti-counterfeiting detection model are also shown. It can be seen that the forgery detector, the fifth classification head, the sixth classification head and the eighth classification head in the classifier 56 can be respectively connected to the output layer of the weighted pooling module 55, the feature fusion module 53, the output layer of the confidence decoder 54 and the output layer of the edge perception module 57. The seventh classification head can be connected to each output end of the forgery detector, the fifth classification head, the sixth classification head and the eighth classification head, which will not be described in detail.

[0121] Figure 6 The specific structure and connection mode of the classifier 66 in the credential anti-counterfeiting detection model are also shown. It can be seen that the forgery detector, the fifth classification head, the sixth classification head and the eighth classification head in the classifier 66 can be respectively connected to the output layer of the weighted pooling module 65, the feature fusion module 63, the output layer of the confidence decoder 64 and the output layer of the edge perception module 67. The seventh classification head can be connected to each output end of the forgery detector, the fifth classification head, the sixth classification head and the eighth classification head, which will not be described in detail.

[0122] In the embodiment of the present specification, the processing of the first voucher image to obtain the second voucher image may include: performing frequency domain transformation processing on the first voucher image to obtain the voucher spectrum; wherein the frequency domain transformation processing includes discrete cosine transformation processing.

[0123] In the embodiment of the specification, since the user usually uses an image compression algorithm based on discrete cosine transform (DCT) in the process of forging a voucher image, and when different compression rounds are performed on different regions of the image, the discrete cosine transform coefficients of the image can often reflect the difference in such compression rounds, the first voucher image can be subjected to discrete cosine transform transformation to obtain discrete cosine transform coefficients as a voucher spectrum diagram. By combining the discrete cosine transform coefficients of the first voucher image for anti-counterfeiting detection processing, it is beneficial to improve the accuracy of the first anti-counterfeiting detection result of the generated first voucher image.

[0124] Of course, frequency domain transformation processing methods such as Fourier transform, fast Fourier transform (FFT), wavelet transform, frequency domain filtering, etc. can also be used to generate a voucher spectrum corresponding to the first voucher image, and there is no specific limitation on this.

[0125] In the embodiments of this specification, the credential anti-counterfeiting detection model may further include: a noise feature extraction model.

[0126] The noise feature extraction model can be any one of a Noiseprint extractor, a spatial enrichment model, and a Bayar convolution model; the noise feature extraction model can be connected to the second high-resolution network module. The noise feature extraction model is used to process the first voucher image to obtain the voucher noise feature map.

[0127] In the embodiments of this specification, the Noiseprint extractor can extract noise-sensitive fingerprints from RGB images. The Spatial Rich Model (SRM) can be used to suppress image content and highlight image tampering traces and noise. The Bayar Convolution model is usually used as a noise extractor to adaptively learn image tampering traces from image data. Therefore, any of the above three models can be used to extract the voucher noise feature map from the first voucher image, which is convenient and fast.

[0128] Optionally, the noise feature extraction model in the credential anti-counterfeiting detection model may include: a Noiseprint extractor, at least two models of a spatial enrichment model and a Bayar convolution model, and a multi-head attention module; the at least two models are connected to the multi-head attention module, and the multi-head attention module is also connected to the second high-resolution network module.

[0129] The at least two models may be used to process the first voucher image to obtain each initial voucher noise feature data.

[0130] The multi-head attention module can be used to perform feature fusion processing on each of the initial voucher noise feature data to obtain the voucher noise feature map.

[0131] In the embodiments of the present specification, multiple models can also be used to extract initial credential noise feature data of the first credential image respectively, and a multi-head attention module can be used to perform feature fusion processing on each initial credential noise feature data, so that the obtained credential noise feature map with better accuracy can be used as the second credential image, which is beneficial to improving the accuracy of the anti-counterfeiting detection result generated based on the second credential image.

[0132] At present, since the area where the key information in the credential image is located contains the most important credential information, users often tamper with the area where the key information in the credential image is located. However, the existing credential anti-counterfeiting detection scheme does not focus on analyzing and processing the area where the key information in the credential image is located, which can easily generate erroneous credential anti-counterfeiting detection results due to interference from other areas in the credential image, affecting the accuracy and effectiveness of the forged credential detection scheme.

[0133] In actual applications, since the area where the key information in the trusted credential is located may need to be located at a preset position, and the text in the area where the key information is located may also need to use a preset font, and the text in the area where the key information is located may also need to conform to a preset text alignment format, during credential anti-counterfeiting detection, the format detection strategy that needs to be executed on the area where the key information is located in the credential image can be summarized in advance based on expert experience, so as to generate the final credential anti-counterfeiting detection result based on the format detection result.

[0134] Based on this, Figure 2 The method may further include:

[0135] Determine location information of an area where key information in the first credential image is located.

[0136] Based on the position information of the area where the key information is located, it is detected whether the area where the key information is located is located at a preset position in the first credential image, and / or whether the text data in the area where the key information is located conforms to a preset text alignment format, and / or whether the text data in the area where the key information is located uses a preset font, to obtain a second anti-counterfeiting detection result for the first credential image.

[0137] A comprehensive anti-counterfeiting detection result for the first credential image is generated according to the first anti-counterfeiting detection result and the second anti-counterfeiting detection result.

[0138] In the embodiments of this specification, the first credential image may generally include an area where key information is located and an area where non-key information is located. Among them, the area where key information is located may include key information in the first credential image. In practical applications, the categories of key information contained in different types of credential images may often be different, and the areas where the same type of key information is located in different types of credential images may also be different, and this is not specifically limited.

[0139] For ease of understanding, an example is given to illustrate this. For example, assuming that the first credential image is an identity credential image, the area where the key information in the first credential image is located may be an area containing the user's face image, and an area containing the user's textual identity information; or, assuming that the first credential image is a payment result credential image, the area where the key information in the first credential image is located may be an area containing text or icons reflecting whether the payment is successful, and an area containing information such as the payment amount and the payee, without specific limitation. The area other than the area where the key information is located in the first credential image may belong to the area where the non-key information is located, which will not be elaborated.

[0140] In practical applications, the first voucher image can be input into a target detection model to obtain the location information of the area where the key information in the first voucher image is located, which is output by the target detection model. The target detection model can be a deep learning model for object detection, which can be used to predict the location and category of objects in an image. Based on this, the area where the key information in the first voucher image is located can be used as a detection object, and a trusted voucher image that is consistent with the voucher category to which the first voucher image belongs can be used as a training sample in advance, and the location information of the area where the key information of the trusted voucher image is located can be used as label data to train the target detection model, so that the target detection model can accurately identify the location information of the area where the key information in the first voucher image is located, which is convenient and quick.

[0141] Alternatively, the first voucher image may be subjected to correction processing and scale transformation processing to obtain a corrected voucher image of a target size; the target size is the size of a voucher template of the same voucher type as the target voucher. Thus, the preset position information of the area where the preset key information is located in the pre-set voucher template may be determined as the position information of the area where the key information is located in the corrected voucher image. Subsequently, it is necessary to perform various format detections related to the area where the key information is located on the corrected voucher image, so as to ensure the accuracy of the second anti-counterfeiting detection result generated for the first voucher image.

[0142] In the embodiments of the present specification, after detecting the area where the key information in the first credential image is located according to the preset format detection strategy, a second anti-counterfeiting detection result can be obtained to reflect whether the first credential image is a real / forged credential image. When the first anti-counterfeiting detection result and the second anti-counterfeiting detection result for the first credential image are combined to generate a comprehensive anti-counterfeiting detection result, when any anti-counterfeiting detection result reflects that the first credential image should belong to a forged credential image, the comprehensive anti-counterfeiting detection result can indicate that the first credential image should belong to a forged credential image. Alternatively, the forgery risk scores corresponding to the first anti-counterfeiting detection result and the second anti-counterfeiting detection result can be further determined, and a comprehensive forgery risk score can be generated based on each forgery risk score, so that when the comprehensive forgery risk score reaches a threshold, the comprehensive anti-counterfeiting detection result indicates that the first credential image should belong to a forged credential image, and no specific limitation is made to this.

[0143] Optionally, the detecting, based on the location information of the area where the key information is located, whether the area where the key information is located is located at a preset position in the first credential image may include:

[0144] Based on the position information of the area where the key information is located, determining whether a first distance between a left side of the area where the key information is located and a longitudinal center line of the first credential image is equal to a second distance between a right side of the area where the key information is located and the longitudinal center line; or

[0145] Based on the position information of the area where the key information is located, it is determined whether a third distance between a left side of the area where the key information is located and a left side of the first credential image is equal to a fourth distance between a right side of the area where the key information is located and a right side of the first credential image.

[0146] In the embodiments of the present specification, when the distances between the left and right sides of the area where the key information is located are equal to the corresponding left and right sides of the first credential image, or when the distances between the left and right sides of the area where the key information is located are equal to the longitudinal center line of the first credential image, it can be considered that the area where the key information is located is located at a position that is left-right symmetrical with respect to the longitudinal center line of the first credential image (i.e., a preset position). At this time, it can be considered that the possibility of a user tampering with the area where the key information is located is small, so that a second anti-counterfeiting detection result reflecting that the first credential image is not a forged credential image can be generated, or the detection results of other detection items can be combined to jointly generate the second anti-counterfeiting detection result, which is not specifically limited.

[0147] Optionally, the detecting, based on the position information of the area where the key information is located, whether the text data in the area where the key information is located conforms to a preset text alignment format may include:

[0148] Based on the position information of the area where the key information is located, the position information of each text data in the area where the key information is located is determined.

[0149] Based on the position information of the text data, determine whether the vertical coordinates of the bottom edges of the text data located in the same row are consistent, and / or determine whether the vertical coordinates of the top edges of the text data located in the same row are consistent, and / or determine whether the horizontal coordinates of the left edges of the text data located in the same column are consistent, and / or determine whether the horizontal coordinates of the right edges of the text data located in the same column are consistent.

[0150] In the embodiments of the present specification, the preset text alignment format that the text data in the area where the key information is located needs to comply with may be in-row alignment, in-column alignment, etc., and no specific limitation is made to this.

[0151] When detecting the preset text alignment format, the position information of each text data in the area where the key information is located can be determined, wherein the position information of each text data can be used to determine the range of the horizontal and vertical coordinates of the text data. Based on this, it can be determined whether the maximum or minimum value of the vertical coordinates of the text data in the same row is the same. If they are the same, it can be considered that the text data in the area where the key information is located is specifically aligned in the same row (preset text alignment format); similarly, it can be determined whether the maximum or minimum value of the horizontal coordinates of the text data in the same column is the same. If they are the same, it can be considered that the text data in the area where the key information is located is specifically aligned in the same column (preset text alignment format); at this time, it can be considered that the possibility of the user tampering with the area where the key information is located is small, therefore, a second anti-counterfeiting detection result reflecting that the first credential image is not a forged credential image can be generated, or, the detection results of other detection items can be combined to jointly generate the second anti-counterfeiting detection result, which is not specifically limited.

[0152] Optionally, the detecting, based on the location information of the area where the key information is located, whether the text data in the area where the key information is located adopts a preset font may include:

[0153] Based on the position information of the area where the key information is located, the credential sub-image in the area where the key information is located in the first credential image is input into a font detection model to obtain a detection result of a target font used by the text data in the area where the key information is located, which is output by the font detection model; wherein the font detection model is obtained by pre-training a classification model using image samples carrying label data, and the label data is used to reflect the type of font to which the text data in the image sample belongs.

[0154] Based on the detection result, it is determined whether the target font is consistent with the preset font. Or,

[0155] Based on the position information of the area where the key information is located, the voucher sub-image in the area where the key information is located in the first voucher image is input into a font feature extraction model to obtain target font feature data of the text data in the area where the key information is located extracted by the font feature extraction model; wherein the font feature extraction model comprises: an autoencoder model pre-trained using image samples containing text data of a preset font;

[0156] Determine whether the target font feature data is consistent with the reference font feature data of the preset font; wherein the reference font feature data is font feature data extracted using the font feature extraction model for a trusted credential image that is consistent with the credential category to which the first credential image belongs.

[0157] In the embodiments of this specification, the preset font used by the text data in the key information area can be set according to the actual situation. For example, the preset font can be Kaiti, Songti, Heiti, etc., and there is no specific limitation on this. By using the pre-trained font detection model or font feature extraction model, it is convenient and fast to identify whether the text data in the key information area uses the preset font, with good accuracy.

[0158] In practical applications, after determining that the text data in the area where the key information in the first credential image is located uses a preset font, the second anti-counterfeiting detection result reflecting that the first credential image is not a forged credential image can be directly generated, or the second anti-counterfeiting detection result can be generated together with the detection results of other detection items, without specific limitation.

[0159] Since the metadata information of an image may include shooting time, location, camera information, image modification information, etc., anti-counterfeiting detection may be performed on the image based on the metadata of the image to verify the authenticity of the image.

[0160] Based on this, Figure 2 The method may further include: obtaining an exchangeable image file of the first credential image, and determining whether the exchangeable image file contains a forged image keyword to obtain a target determination result.

[0161] Correspondingly, generating a comprehensive anti-counterfeiting detection result for the first credential image based on the first anti-counterfeiting detection result and the second anti-counterfeiting detection result may include: generating a comprehensive anti-counterfeiting detection result for the first credential image based on the first anti-counterfeiting detection result, the second anti-counterfeiting detection result and the target judgment result.

[0162] In the embodiments of the present specification, the exchangeable image file (EXIF information) can record the attribute information and shooting data of the digital photo; EXIF ​​can be attached to JPEG, TIFF, RIFF and other files to add the content of the digital camera shooting information and the index map or version information of the image processing software. Based on this, it is possible to pre-identify that after the user tampers with the credential image, the text reflecting the user's tampering with the credential image carried in the exchangeable image file of the forged credential image can be used as the forged image keyword. If the target judgment result indicates that the exchangeable image file of the first credential image contains the forged image keyword, it can be considered that the first credential image is likely to be a forged credential image. Therefore, by combining the target judgment result and the first anti-counterfeiting detection result and the second anti-counterfeiting detection result, a comprehensive anti-counterfeiting detection result for the first credential image is generated, which is conducive to improving the accuracy of the comprehensive anti-counterfeiting detection result.

[0163] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method. Figure 7 , is a schematic diagram of the structure of a credential anti-counterfeiting detection device provided in an embodiment of this specification. Figure 7 As shown, the credential anti-counterfeiting detection device 7 can be implemented as all or part of the electronic device through software, hardware or a combination of both. According to some embodiments, the credential anti-counterfeiting detection device 7 may include a first acquisition module 71, a second acquisition module 72, and an anti-counterfeiting detection module 73, wherein:

[0164] The first acquisition module 71 may be used to acquire a first credential image obtained by performing image acquisition on a target credential of a user.

[0165] The second acquisition module 72 may be used to process the first voucher image to obtain a second voucher image, where the second voucher image includes: any one of a voucher noise feature image and a voucher spectrum image.

[0166] The anti-counterfeiting detection module 73 can be used to perform anti-counterfeiting detection processing on the first credential image and the second credential image using a credential anti-counterfeiting detection model to obtain a first anti-counterfeiting detection result for the first credential image output by the credential anti-counterfeiting detection model; wherein the credential anti-counterfeiting detection model includes: a multimodal feature encoder and a classifier connected in sequence.

[0167] Optionally, the multimodal feature encoder may include: a first high-resolution network module, a second high-resolution network module and a feature fusion module; the first high-resolution network module and the second high-resolution network module are respectively connected to the feature fusion module, and the feature fusion module is connected to the classifier.

[0168] Among them, the first high-resolution network module can be used to perform feature extraction processing on the first voucher image to obtain a plurality of first voucher image feature data of different scales. The second high-resolution network module can be used to perform feature extraction processing on the second voucher image to obtain a plurality of second voucher image feature data of different scales. The feature fusion module can be used to perform feature fusion processing on the plurality of first voucher image feature data of different scales and the plurality of second voucher image feature data of different scales to obtain third voucher image feature data. The classifier can be used to perform classification processing based on the third voucher image feature data to obtain a first anti-counterfeiting detection result used to reflect whether the first voucher image is a credible voucher image.

[0169] Optionally, the first high-resolution network module may be an RGBStream module in a compression artifact tracking network model; the second high-resolution network module may be a DCT Stream module in a compression artifact tracking network model.

[0170] The RGB Stream module can be used to perform feature extraction processing on the first voucher image to obtain a plurality of first voucher image feature data of different scales. The DCT Stream module can be used to perform feature extraction processing on the second voucher image to obtain a plurality of second voucher image feature data of different scales.

[0171] Optionally, the feature fusion module may be a Fusion Stage module in a compression artifact tracking network model. The Fusion Stage module may be used to perform feature fusion processing on the first voucher image feature data of multiple different scales and the second voucher image feature data of multiple different scales to obtain third voucher image feature data.

[0172] Optionally, the feature fusion module may include a first Transformer submodule, a second Transformer submodule, a Class Attention submodule and a multi-scale feature fusion submodule, and the multi-scale feature fusion submodule may be a branch network in the Fusion Stage module in the compressed artifact tracking network model for performing feature fusion processing on the multi-scale feature map. All output layers of the first high-resolution network module except the output layer for outputting the first voucher image feature data with the largest scale may be connected to the first Transformer submodule, each output layer of the second high-resolution network module may be connected to the second Transformer submodule, the first Transformer submodule and the second Transformer submodule may be connected to the Class Attention submodule, the output layer for outputting the first voucher image feature data with the largest scale and the Class Attention submodule may be connected to the multi-scale feature fusion submodule, and the multi-scale feature fusion submodule is also connected to the classifier.

[0173] Among them, the first Transformer submodule can be used to perform feature fusion processing on the first voucher image feature data of other scales except the maximum scale to obtain the fourth voucher image feature data. The second Transformer submodule can be used to perform feature fusion processing on the second voucher image feature data of each scale to obtain the fifth voucher image feature data. The Class Attention submodule can be used to perform feature fusion processing on the fourth voucher image feature data and the fifth voucher image feature data to obtain the sixth voucher image feature data. The multi-scale feature fusion submodule can be used to perform feature fusion processing on the first voucher image feature data with the largest scale and the sixth voucher image feature data to obtain the third voucher image feature data.

[0174] Optionally, the classifier may include: a first classification head, a second classification head and a third classification head.

[0175] The output layer of the feature fusion module can be connected to the first classification head, the output layer of the second high-resolution network module for outputting the second credential image feature data with the largest scale can be connected to the second classification head, and both the first classification head and the second classification head can be connected to the third classification head; wherein, the output layer of the feature fusion module can be the output layer of the Fusion Stage module or the output layer of the multi-scale feature fusion sub-module.

[0176] The first classification head can be used to classify the third credential image feature data to obtain a first classification probability value. The second classification head can be used to classify the second credential image feature data with the largest scale to obtain a second classification probability value. The third classification head can be used to classify the first classification probability value and the second classification probability value to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

[0177] Optionally, the credential anti-counterfeiting detection model may further include: an edge perception module in a boundary guidance network model; the classifier may further include: a fourth classification head. The output layer of the first high-resolution network module for outputting the first credential image feature data with the largest scale may also be connected to the input layer of the edge perception module, the output layer of the edge perception module may be connected to the fourth classification head in the classifier, and the fourth classification head may also be connected to the third classification head.

[0178] The edge perception module can be used to perform feature extraction processing on the first voucher image feature data with the largest scale to obtain the seventh voucher image feature data. The fourth classification head can be used to perform classification processing on the seventh voucher image feature data to obtain a third classification probability value. The third classification head can be specifically used to perform classification processing on the first classification probability value, the second classification probability value and the third classification probability value to obtain a first anti-counterfeiting detection result reflecting whether the first voucher image is a credible voucher image.

[0179] Optionally, the credential anti-counterfeiting detection model may also include: a confidence decoder and a weighted pooling module in the TruFor model; the classifier may include: a forgery detector in the TruFor model.

[0180] Each output layer at the first high-resolution network module and each output layer at the second high-resolution network module can be connected to the confidence decoder, the confidence decoder and the feature fusion module can be connected to the weighted pooling module, and the weighted pooling module can be connected to the forgery detector.

[0181] Among them, the confidence decoder can be used to perform feature extraction processing and feature fusion processing on the first voucher image feature data of each scale and the second voucher image feature data of each scale to obtain eighth voucher image feature data. The weighted pooling module can be used to determine the target weight required for weighted pooling of the third voucher image feature data based on the eighth voucher image feature data, so as to perform weighted pooling processing on the third voucher image feature data according to the target weight to obtain ninth voucher image feature data. The forgery detector can be used to perform detection processing on the ninth voucher image feature data to obtain a first anti-counterfeiting detection result that reflects whether the first voucher image is a credible voucher image.

[0182] Optionally, the classifier may further include: a fifth classification head, a sixth classification head and a seventh classification head.

[0183] The output layer of the feature fusion module can also be connected to the fifth classification head, the output layer of the confidence decoder can also be connected to the sixth classification head, and the output layers of the fifth classification head, the sixth classification head and the forgery detector can all be connected to the seventh classification head; wherein, the output layer of the feature fusion module can be the output layer of the Fusion Stage module or the output layer of the multi-scale feature fusion sub-module.

[0184] The forgery detector can be used to perform detection processing on the ninth credential image feature data to obtain a fourth classification probability value. The fifth classification head can be used to perform classification processing on the third credential image feature data to obtain a fifth classification probability value. The sixth classification head can be used to perform classification processing on the eighth credential image feature data to obtain a sixth classification probability value. The seventh classification head can be used to perform classification processing on the fourth classification probability value, the fifth classification probability value and the sixth classification probability value to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

[0185] Optionally, the credential anti-counterfeiting detection model may further include: an edge perception module in a boundary guidance network model; the classifier may further include: an eighth classification head. The output layer of the first high-resolution network module for outputting the first credential image feature data with the largest scale may also be connected to the input layer of the edge perception module, the output layer of the edge perception module may be connected to the eighth classification head in the classifier, and the eighth classification head may also be connected to the seventh classification head.

[0186] Among them, the edge perception module can be used to perform feature extraction processing on the first credential image feature data with the largest scale to obtain the tenth credential image feature data. The eighth classification head can be used to perform classification processing on the tenth credential image feature data to obtain the seventh classification probability value. The seventh classification head can specifically be used to perform classification processing on the fourth classification probability value, the fifth classification probability value, the sixth classification probability value and the seventh classification probability value to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credential image.

[0187] Optionally, the second acquisition module may be configured to: perform frequency domain transformation processing on the first voucher image to obtain the voucher spectrum diagram; wherein the frequency domain transformation processing may include discrete cosine transformation processing.

[0188] Optionally, the credential anti-counterfeiting detection model may further include: a noise feature extraction model.

[0189] The noise feature extraction model can be any one of a Noiseprint extractor, a spatial enrichment model and a Bayar convolution model; the noise feature extraction model is connected to the second high-resolution network module; wherein the noise feature extraction model can be used to process the first voucher image to obtain the voucher noise feature map.

[0190] The noise feature extraction model may include: a Noiseprint extractor, at least two models of a spatial enrichment model and a Bayar convolution model, and a multi-head attention module; the at least two models may both be connected to the multi-head attention module, and the multi-head attention module may also be connected to the second high-resolution network module.

[0191] The at least two models can be used to process the first voucher image to obtain each initial voucher noise feature data. The multi-head attention module can be used to perform feature fusion processing on each initial voucher noise feature data to obtain the voucher noise feature map.

[0192] Optional, Figure 7 The device described in the above method may further include:

[0193] A location determination module, used to determine location information of an area where key information in the first credential image is located;

[0194] A format detection module is used to detect, based on the position information of the area where the key information is located, whether the area where the key information is located is located at a preset position in the first credential image, and / or to detect whether the text data in the area where the key information is located conforms to a preset text alignment format, and / or to detect whether the text data in the area where the key information is located uses a preset font, to obtain a second anti-counterfeiting detection result for the first credential image.

[0195] The comprehensive anti-counterfeiting detection result generating module is used to generate a comprehensive anti-counterfeiting detection result for the first credential image according to the first anti-counterfeiting detection result and the second anti-counterfeiting detection result.

[0196] Optionally, the format detection module can be specifically used to: based on the position information of the key information area, determine whether a first distance between the left side of the key information area and the longitudinal center line of the first voucher image is equal to a second distance between the right side of the key information area and the longitudinal center line. Or, based on the position information of the key information area, determine whether a third distance between the left side of the key information area and the left side of the first voucher image is equal to a fourth distance between the right side of the key information area and the right side of the first voucher image.

[0197] Optionally, the format detection module can be specifically used to: determine the position information of each text data in the area where the key information is located based on the position information of the area where the key information is located. According to the position information of the text data, determine whether the vertical coordinates of the bottom edges of the text data located in the same row are consistent, and / or determine whether the vertical coordinates of the top edges of the text data located in the same row are consistent, and / or determine whether the horizontal coordinates of the left edges of the text data located in the same column are consistent, and / or determine whether the horizontal coordinates of the right edges of the text data located in the same column are consistent.

[0198] Optionally, the format detection module can be specifically used to: based on the location information of the area where the key information is located, input the voucher sub-image in the area where the key information is located in the first voucher image into a font detection model, and obtain the detection result output by the font detection model for reflecting the target font used by the text data in the area where the key information is located; wherein the font detection model is obtained by pre-training a classification model using image samples carrying label data, and the label data is used to reflect the font type to which the text data in the image sample belongs. Based on the detection result, determine whether the target font is consistent with the preset font.

[0199] Optionally, the format detection module can be specifically used to: based on the location information of the area where the key information is located, input the voucher sub-image in the area where the key information is located in the first voucher image into the font feature extraction model to obtain the target font feature data of the text data in the area where the key information is located extracted by the font feature extraction model; wherein the font feature extraction model includes: an autoencoder model pre-trained using image samples containing text data of a preset font. Determine whether the target font feature data is consistent with the reference font feature data of the preset font; wherein the reference font feature data is the font feature data extracted by the font feature extraction model for a trusted voucher image that is consistent with the voucher category to which the first voucher image belongs.

[0200] Optional, Figure 7 The device described in the embodiment may further include: a third acquisition module, configured to acquire an exchangeable image file of the first credential image. A judgment module, configured to judge whether the exchangeable image file contains a forged image keyword to obtain a target judgment result. The comprehensive anti-counterfeiting detection result generation module may be specifically configured to generate a comprehensive anti-counterfeiting detection result for the first credential image based on the first anti-counterfeiting detection result, the second anti-counterfeiting detection result and the target judgment result.

[0201] The present specification also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above Figure 2 The specific execution process of the credential anti-counterfeiting detection method shown can be found in the specific description of the relevant embodiments of the credential anti-counterfeiting detection method, which will not be repeated here.

[0202] The present specification also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor as described above. Figure 2 The specific execution process of the credential anti-counterfeiting detection method shown can be found in the specific description of the relevant embodiments of the credential anti-counterfeiting detection method, which will not be repeated here.

[0203] The embodiments of this specification also provide Figure 8 The structural diagram of the electronic device shown in FIG. Figure 8 At the hardware level, the electronic device may include a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 2 The specific execution process of the credential anti-counterfeiting detection method shown can be found in the specific description of the relevant embodiments of the credential anti-counterfeiting detection method, which will not be repeated here.

[0204] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0205] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for computer-readable storage media, computer program products and Figure 8 As for the electronic device shown, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0206] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0207] The controller can be implemented in any appropriate manner, such as a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.

[0208] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0209] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0210] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0211] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0212] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0213] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0214] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0215] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0216] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0217] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0218] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0219] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0220] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0221] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A method for detecting counterfeiting of a certificate, the method include: Acquire a first credential image obtained by performing image acquisition on a target credential of a user; Processing the first voucher image to obtain a second voucher image, wherein the second voucher image includes: any one of a voucher noise feature image and a voucher spectrum image; Performing anti-counterfeiting detection processing on the first credential image and the second credential image using a credential anti-counterfeiting detection model to obtain a first anti-counterfeiting detection result for the first credential image output by the credential anti-counterfeiting detection model; Among them, the credential anti-counterfeiting detection model includes: a multimodal feature encoder and a classifier connected in sequence.

2. The method according to claim 1, wherein the multimodal feature encoder include: A first high-resolution network module, a second high-resolution network module, and a feature fusion module; The first high-resolution network module and the second high-resolution network module are respectively connected to the feature fusion module, and the feature fusion module is connected to the classifier; The first high-resolution network module is used to perform feature extraction processing on the first voucher image to obtain feature data of the first voucher image at multiple different scales; The second high-resolution network module is used to perform feature extraction processing on the second voucher image to obtain feature data of the second voucher image at multiple different scales; The feature fusion module is used to perform feature fusion processing on the plurality of first voucher image feature data of different scales and the plurality of second voucher image feature data of different scales to obtain third voucher image feature data; The classifier is used to perform classification processing based on the third credential image feature data to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

3. According to the method of claim 2, the first high-resolution network module is an RGB Stream module in a compression artifact tracking network model; the second high-resolution network module is a DCTStream module in the compression artifact tracking network model; The RGB Stream module is used to perform feature extraction processing on the first voucher image to obtain feature data of the first voucher image at multiple different scales; The DCT Stream module is used to perform feature extraction processing on the second voucher image to obtain a plurality of second voucher image feature data of different scales.

4. According to the method of claim 2, the feature fusion module is a Fusion Stage module in a compression artifact tracking network model; The Fusion Stage module is used to perform feature fusion processing on the plurality of first voucher image feature data of different scales and the plurality of second voucher image feature data of different scales to obtain the third voucher image feature data; or, The feature fusion module include: A first Transformer submodule, a second Transformer submodule, a ClassAttention submodule and a multi-scale feature fusion submodule, wherein the multi-scale feature fusion submodule is a branch network in the Fusion Stage module in the compression artifact tracking network model for performing feature fusion processing on the multi-scale feature map; All output layers of the first high-resolution network module except the output layer for outputting the first voucher image feature data with the largest scale are connected to the first Transformer submodule, all output layers of the second high-resolution network module are connected to the second Transformer submodule, the first Transformer submodule and the second Transformer submodule are connected to the Class Attention submodule, the output layer for outputting the first voucher image feature data with the largest scale and the Class Attention submodule are connected to the multi-scale feature fusion submodule, and the multi-scale feature fusion submodule is also connected to the classifier; The first Transformer submodule is used to perform feature fusion processing on the first voucher image feature data of other scales except the maximum scale to obtain fourth voucher image feature data; The second Transformer submodule is used to perform feature fusion processing on the second voucher image feature data of each scale to obtain fifth voucher image feature data; The Class Attention submodule is used to perform feature fusion processing on the fourth voucher image feature data and the fifth voucher image feature data to obtain sixth voucher image feature data; The multi-scale feature fusion submodule is used to perform feature fusion processing on the first voucher image feature data with the largest scale and the sixth voucher image feature data to obtain the third voucher image feature data.

5. The method according to claim 4, wherein the classifier include: A first classification head, a second classification head and a third classification head; The output layer of the feature fusion module is connected to the first classification head, the output layer of the second high-resolution network module for outputting the second voucher image feature data with the largest scale is connected to the second classification head, and the first classification head and the second classification head are both connected to the third classification head; wherein the output layer of the feature fusion module is the output layer of the Fusion Stage module or the output layer of the multi-scale feature fusion submodule; The first classification head is used to perform classification processing on the third voucher image feature data to obtain a first classification probability value; The second classification head is used to perform classification processing on the second voucher image feature data with the largest scale to obtain a second classification probability value; The third classification head is used to perform classification processing on the first classification probability value and the second classification probability value to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

6. The method according to claim 5, wherein the credential anti-counterfeiting detection model further comprises: include: The edge-aware module in the boundary-guided network model; The classifier further comprises: a fourth classification head; The output layer of the first high-resolution network module for outputting the first voucher image feature data with the largest scale is also connected to the input layer of the edge perception module, the output layer of the edge perception module is connected to the fourth classification head in the classifier, and the fourth classification head is also connected to the third classification head; The edge perception module is used to perform feature extraction processing on the first voucher image feature data with the largest scale to obtain the seventh voucher image feature data; The fourth classification head is used to perform classification processing on the seventh voucher image feature data to obtain a third classification probability value; The third classification header is specifically used to perform classification processing on the first classification probability value, the second classification probability value and the third classification probability value to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

7. The method according to claim 4, wherein the credential anti-counterfeiting detection model further comprises: include: The confidence decoder and weighted pooling module in the TruFor model; The classifier includes: a forgery detector in the TruFor model; Each output layer at the first high-resolution network module and each output layer at the second high-resolution network module are also connected to the confidence decoder, the confidence decoder and the feature fusion module are both connected to the weighted pooling module, and the weighted pooling module is connected to the forgery detector; The confidence decoder is used to perform feature extraction processing and feature fusion processing on the first voucher image feature data of each scale and the second voucher image feature data of each scale to obtain eighth voucher image feature data; The weighted pooling module is used to determine a target weight required to perform weighted pooling on the third voucher image feature data based on the eighth voucher image feature data, so as to perform weighted pooling processing on the third voucher image feature data according to the target weight to obtain ninth voucher image feature data; The forgery detector is used to perform detection processing on the ninth credential image feature data to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

8. The method according to claim 7, wherein the classifier further include: The fifth, sixth and seventh classification heads; The output layer of the feature fusion module is also connected to the fifth classification head, the output layer of the confidence decoder is also connected to the sixth classification head, and the output layers of the fifth classification head, the sixth classification head and the forgery detector are all connected to the seventh classification head; wherein the output layer of the feature fusion module is the output layer of the Fusion Stage module or the output layer of the multi-scale feature fusion submodule; The forgery detector is specifically used to perform detection processing on the ninth credential image feature data to obtain a fourth classification probability value; The fifth classification head is used to perform classification processing on the third voucher image feature data to obtain a fifth classification probability value; The sixth classification head is used to perform classification processing on the eighth voucher image feature data to obtain a sixth classification probability value; The seventh classification head is used to perform classification processing on the fourth classification probability value, the fifth classification probability value and the sixth classification probability value to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

9. The method according to claim 8, wherein the credential anti-counterfeiting detection model further comprises: include: The edge perception module in the boundary guidance network model; the classifier also includes: an eighth classification head; The output layer of the first high-resolution network module for outputting the first voucher image feature data with the largest scale is also connected to the input layer of the edge perception module, the output layer of the edge perception module is connected to the eighth classification head in the classifier, and the eighth classification head is also connected to the seventh classification head; The edge perception module is used to perform feature extraction processing on the first voucher image feature data with the largest scale to obtain the tenth voucher image feature data; The eighth classification head is used to perform classification processing on the tenth voucher image feature data to obtain a seventh classification probability value; The seventh classification header is specifically used to perform classification processing on the fourth classification probability value, the fifth classification probability value, the sixth classification probability value and the seventh classification probability value to obtain a first anti-counterfeiting detection result reflecting whether the first credential image is a credible credential image.

10. The method according to any one of claims 1 to 9, wherein the first voucher image is processed to obtain a second voucher image, include: Perform frequency domain transformation processing on the first voucher image to obtain the voucher spectrum diagram; wherein the frequency domain transformation processing includes discrete cosine transformation processing.

11. The method according to any one of claims 2 to 9, wherein the credential anti-counterfeiting detection model further comprises: include: Noise feature extraction model; The noise feature extraction model is any one of a Noiseprint extractor, a spatial enrichment model and a Bayar convolution model; the noise feature extraction model is connected to the second high-resolution network module; The noise feature extraction model is used to process the first voucher image to obtain the voucher noise feature map; or, The noise feature extraction model includes: a noiseprint extractor, at least two models of a spatial domain enrichment model and a Bayar convolution model, and a multi-head attention module; the at least two models are connected to the multi-head attention module, and the multi-head attention module is also connected to the second high-resolution network module; The at least two models are used to process the first voucher image to obtain each initial voucher noise feature data; The multi-head attention module is used to perform feature fusion processing on each of the initial voucher noise feature data to obtain the voucher noise feature map.

12. The method according to any one of claims 1 to 9, further comprising: include: Determining location information of an area where key information in the first credential image is located; Based on the position information of the area where the key information is located, detecting whether the area where the key information is located is located at a preset position in the first credential image, and / or detecting whether the text data in the area where the key information is located conforms to a preset text alignment format, and / or detecting whether the text data in the area where the key information is located uses a preset font, to obtain a second anti-counterfeiting detection result for the first credential image; A comprehensive anti-counterfeiting detection result for the first credential image is generated according to the first anti-counterfeiting detection result and the second anti-counterfeiting detection result.

13. The method according to claim 12, wherein based on the location information of the area where the key information is located, detecting whether the area where the key information is located is located at a preset position in the first credential image, include: Based on the position information of the area where the key information is located, determining whether a first distance between a left side of the area where the key information is located and a longitudinal center line of the first credential image is equal to a second distance between a right side of the area where the key information is located and the longitudinal center line; or Based on the position information of the area where the key information is located, it is determined whether a third distance between a left side of the area where the key information is located and a left side of the first credential image is equal to a fourth distance between a right side of the area where the key information is located and a right side of the first credential image.

14. The method according to claim 12, wherein the detecting, based on the position information of the area where the key information is located, whether the text data in the area where the key information is located conforms to a preset text alignment format, include: Based on the location information of the area where the key information is located, determining the location information of each text data in the area where the key information is located; Based on the position information of the text data, determine whether the vertical coordinates of the bottom edges of the text data located in the same row are consistent, and / or determine whether the vertical coordinates of the top edges of the text data located in the same row are consistent, and / or determine whether the horizontal coordinates of the left edges of the text data located in the same column are consistent, and / or determine whether the horizontal coordinates of the right edges of the text data located in the same column are consistent.

15. The method according to claim 12, wherein based on the location information of the area where the key information is located, detecting whether the text data in the area where the key information is located is in a preset font, include: Based on the position information of the area where the key information is located, the voucher sub-image in the area where the key information is located in the first voucher image is input into a font detection model to obtain a detection result output by the font detection model for reflecting the target font used by the text data in the area where the key information is located; wherein the font detection model is obtained by training a classification model in advance using image samples carrying label data, and the label data is used to reflect the type of font to which the text data in the image sample belongs; Based on the detection result, it is determined whether the target font is consistent with the preset font.

16. The method according to claim 12, wherein based on the location information of the area where the key information is located, detecting whether the text data in the area where the key information is located is in a preset font, include: Based on the position information of the area where the key information is located, the voucher sub-image in the area where the key information is located in the first voucher image is input into a font feature extraction model to obtain target font feature data of the text data in the area where the key information is located extracted by the font feature extraction model; wherein the font feature extraction model comprises: an autoencoder model pre-trained using image samples containing text data of a preset font; Determine whether the target font feature data is consistent with the reference font feature data of the preset font; wherein the reference font feature data is font feature data extracted using the font feature extraction model for a trusted credential image that is consistent with the credential category to which the first credential image belongs.

17. The method according to claim 12, further comprising: include: obtaining an exchangeable image file of the first credential image; Determine whether the exchangeable image file contains forged image keywords to obtain a target determination result; The step of generating a comprehensive anti-counterfeiting detection result for the first credential image according to the first anti-counterfeiting detection result and the second anti-counterfeiting detection result includes: A comprehensive anti-counterfeiting detection result for the first credential image is generated according to the first anti-counterfeiting detection result, the second anti-counterfeiting detection result and the target judgment result.

18. A certificate anti-counterfeiting detection device, include: A first acquisition module is used to acquire a first credential image obtained by performing image acquisition on a target credential of a user; A second acquisition module is used to process the first voucher image to obtain a second voucher image, where the second voucher image includes: any one of a voucher noise feature image and a voucher spectrum image; an anti-counterfeiting detection module, configured to perform anti-counterfeiting detection processing on the first credential image and the second credential image using a credential anti-counterfeiting detection model, and obtain a first anti-counterfeiting detection result for the first credential image output by the credential anti-counterfeiting detection model; Among them, the credential anti-counterfeiting detection model includes: a multimodal feature encoder and a classifier connected in sequence.

19. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 17.

20. An electronic device, include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method as claimed in any one of claims 1 to 17.

21. A computer program product having at least one instruction stored thereon, wherein when the at least one instruction is executed by a processor, the steps of the method according to any one of claims 1 to 17 are implemented.

Images