A license identification and forgery method based on multispectral images

Abstract

The present application belongs to the field of certificate anti-counterfeiting, and discloses a multi-country certificate authentication method and system based on multispectral images, which comprises: pre-screening based on difference images; constructing a characteristic model of the device according to the depth statistical characteristics of the image collected by the device to compensate the image; extracting and detecting ultraviolet anti-counterfeiting features by using a multi-scale texture analysis method; physically aligning the ultraviolet image with the white light image through affine transformation, performing multispectral feature fusion, and then performing fluorescent fiber detection after passing through a fluorescent enhancement network; and designing a multi-modal differential feature enhancement and detection network for the infrared image to detect complex anti-counterfeiting features under infrared light. The certificate authentication method and system based on multispectral images are superior to other existing certificate authentication methods in the technical fields of passport anti-counterfeiting feature enhancement, identity security verification, and unordered object detection.

Description

Technical Field

[0001] This invention belongs to, but is not limited to, the field of certificate anti-counterfeiting technology, and particularly relates to a multinational certificate authentication method and system based on multispectral images. Background Technology

[0002] With the steady progress of my country's economic development, in-depth exchanges and cooperation between my country and other countries in various fields have become more frequent, leading to an increase in inbound and outbound personnel for trade, tourism, and other purposes. To meet international service demands and improve personal information authentication systems, many important transportation facilities, such as railway stations and airports, have implemented passport-based identity authentication systems. Simultaneously, with the influx of foreign populations, hotel check-in and banking financial services also face the need to verify passport authenticity. Therefore, as a carrier of personal information and a legally valid document, the passport has become an indispensable part of the identity authentication process in public places. With the popularization of intelligent identification technology, to improve the speed of immigration processing and reduce the training costs of security personnel, customs and other ports of entry and exit, banks and other financial institutions, and hotels and other accommodations have implemented intelligent passport inspection processes. The overall passport inspection procedure is "receiving the document - verifying the person - checking the document - data entry - stamping and releasing." In the "verifying the document" process, passport authentication software uses passport authentication algorithms to detect the authenticity of various spectral images of the passport. The results of passport authentication software determine the validity and legality of identity documents for inbound and outbound travelers. A rigorous and precise passport authentication algorithm enhances the reliability of these results. The unique anti-counterfeiting features within a passport play a crucial role in the authentication software's ability to verify its authenticity: the algorithm identifies anti-counterfeiting features in specific areas of the passport, assesses the validity and completeness of all anti-counterfeiting features, and determines the passport's authenticity based on the identification results of all anti-counterfeiting features and the verification results of relevant personal information.

[0003] In practical applications, several problems exist in document authentication, resulting in poor effectiveness: First, the ultraviolet (UV) differences in document images acquired by different devices are too large, especially in the wavelength and intensity of UV light, leading to significant variations in UV images of the same document. This easily results in false positives. Cross-device image acquisition and UV feature detection is a key problem that this invention aims to solve. Second, wear and tear on documents can cause the UV image to appear darker overall, weakening the features of the UV detection area and leading to authentication errors. Third, passports contain irregularly distributed fluorescent fibers in UV light, with irregular lengths and distributions, posing a significant challenge to authentication.

[0004] Traditional anti-counterfeiting methods perform poorly in the aforementioned applications. Firstly, existing methods struggle to handle images captured by ultraviolet light of varying wavelengths and intensities across different devices, leading to occasional errors in cross-device image anti-counterfeiting. Secondly, existing traditional methods such as template matching and SIFT feature point detection are known to be less effective when passports exhibit localized wear, making anti-counterfeiting more challenging. Finally, traditional non-deep learning methods are ill-suited for detecting irregularly distributed fluorescent fibers, while deep learning methods demonstrate superior performance in this area.

[0005] Based on the above analysis, the urgent technical problems that need to be solved in the existing technology are:

[0006] The issue of pre-screening of common anti-counterfeiting features for certificates and licenses;

[0007] Problems such as difficulty in recognizing and accurately identifying counterfeit certificates in cross-device scenarios;

[0008] Traditional methods suffer from low detection efficiency and low accuracy when dealing with irregularly distributed patterns containing numerous interference elements.

[0009] Traditional technical solutions are not robust enough to handle the rich anti-counterfeiting features on passport infrared images, and often fail to detect counterfeits when faced with complex anti-counterfeiting features. Summary of the Invention

[0010] To address the problems existing in the prior art, this invention provides a method and system for authenticating multinational certificates based on multispectral images.

[0011] This invention is implemented as follows: a method for authenticating multinational certificates based on multispectral images, characterized in that the method specifically includes:

[0012] S1: Based on the pre-screening of difference images, detect the general anti-counterfeiting features of certificates and licenses. This is accomplished by calculating the absolute difference between the images, performing morphological operations on the difference images, and quantifying the image differences.

[0013] S2: Construct a characteristic model of the device based on the depth statistical features of the images acquired by the device, and then compensate the images to make their image features closer to those of standard devices;

[0014] S3: Extract and detect UV anti-counterfeiting features using multi-scale texture analysis (MSTA);

[0015] S4: The ultraviolet image is physically aligned with the white light image through affine transformation. The importance weights of the ultraviolet and white light image channels are automatically learned through the Squeeze-and-Excitation module to achieve dynamic fusion of multispectral features. At the same time, a three-layer CNN network is designed for fluorescence enhancement as the entry point of the detection model.

[0016] S5: A multimodal differential feature enhancement and detection network was designed for infrared images to detect complex anti-counterfeiting features under infrared light.

[0017] Further, in step S1, the infrared and ultraviolet images of the image are processed by the following algorithms: In the preprocessing stage, the image captured under white light is converted to grayscale; in terms of difference calculation, the absolute difference between the two images is calculated, the absolute difference is calculated, the difference image is then binarized to highlight significant differences, and finally morphological dilation is performed to connect adjacent difference points; in the difference quantification stage, the pre-screened image is mainly quantified using the infrared structural similarity index irSSIM and the ultraviolet difference pixel ratio uvDiffPercentage to measure whether the image has general anti-counterfeiting features. The calculation formulas for irSSIM and uvDiffPercentage are as follows:

[0018]

[0019]

[0020] in, , These are the mean values ​​of the ultraviolet image and the white light image, respectively. , These are the variances of the ultraviolet image and the white light image, respectively; σ is the covariance; and C1 and C2 are constants used to prevent the denominator from being zero.

[0021] After processing in S1, two types of passports can be initially identified: one type lacks common anti-counterfeiting features and is output as a fake passport, while the other type possesses common anti-counterfeiting features. For the latter type of passport, the multispectral image of the passport is further input into S2 for more precise anti-counterfeiting feature detection.

[0022] Furthermore, in step S2, multi-level feature transformation and statistical correction (MFTSC) is employed. First, multi-scale geometric analysis (MGA) is used to decompose the image into components of different frequencies and directions:

[0023]

[0024] in, These are the coefficients of scale s and direction d. Here, r is the corresponding basis function, and r is the residual.

[0025] The coefficient distribution of each sub-band is accurately modeled using the generalized Gaussian distribution (GGD):

[0026]

[0027] α controls the peakness of the distribution, σ controls the diffusion, and adaptive kernel density estimation is used instead of parameter distribution;

[0028] The parametric model for constructing the device characteristics is as follows, including spectral characteristics, nonlinear transformation, and noise characteristics:

[0029]

[0030] Where M is the color mixing matrix, r is the nonlinear response function, b is the offset, and η is the noise term;

[0031] For each subband, the mapping between the source device and the standard device is constructed using optimal transmission theory:

[0032]

[0033] Where F represents the cumulative distribution function, and Wasserstein distance minimization and regularized optimal transport are employed; considering the physical constraints of fluorescence characteristics, the fewer emission characteristics of the fluorescent material introduced, the better to ensure that the correction result conforms to physical laws.

[0034]

[0035] Where E(λ) represents the excitation spectrum, R(λ) represents the response characteristics of the fluorescent material, and S(λ) is the spectral sensitivity of the sensor;

[0036] Adaptive covariance structure cointegration is employed, and multivariate statistical analysis is used to correct the characteristic covariance structure among different devices.

[0037]

[0038] Where Σ represents the covariance matrix and μ represents the mean vector;

[0039] The original scene radiation is estimated using an iterative Bayesian framework:

[0040]

[0041]

[0042] Furthermore, step S3 is as follows:

[0043] (1) MSTA performs multi-scale decomposition on the image and creates an image pyramid;

[0044] (2) Use co-occurrence grayscale matrix (GLCM) to capture the spatial relationship between pixel pairs;

[0045] (3) A Gabor filter is used to simulate the selectivity of the human visual system to direction and frequency;

[0046] (4) Extract wavelet transform features to capture components of different frequencies in the fluorescent pattern;

[0047] (5) The LDIP algorithm using the local orientation mode pays more attention to the orientation characteristics of the pixel domain;

[0048] (6) MSTA fuses all the above features at multiple scales to generate a comprehensive fluorescent texture characterization.

[0049] Furthermore, in S4, the targets to be detected and distinguished are divided into three categories: fluorescent fibers, thin lines, and fluorescent text. First, sample images containing fluorescent lines are collected, and the diversity of the images is ensured: different angles, brightness, background complexity, length of fluorescent lines, etc.

[0050] A multispectral data fusion method is adopted to fuse ultraviolet and visible light images. The ORB or AKAZE algorithm is used to extract key points and feature descriptors from the two images. FLANN or BFMatcher is used for feature point matching, and high-quality matching pairs are selected. The least squares method is used to fit the transformation matrix, and the affine transformation matrix is ​​solved through the matching points. Finally, the ultraviolet image is aligned to the RGB image according to the transformation matrix. In terms of feature fusion, a channel attention mechanism is used for dynamic weighted fusion. The importance weights of ultraviolet and white light images are automatically learned through the Squeeze-and-Excitation module to achieve dynamic fusion of multispectral features.

[0051] The present invention also designs a 3-layer CNN network with RGB image as input and fluorescence-enhanced single-channel mask as output. The network is trained using pixel-level annotations of the fluorescence image. The enhanced image and the original image are stitched together to obtain the final image. Finally, the image after fluorescence feature enhancement is input into the fluorescence fiber target detection module to enhance the global contextual awareness of the fluorescence lines.

[0052] Furthermore, the S5 multimodal differential feature enhancement and detection network includes:

[0053] The dual-stream feature extraction module uses two branches to extract features for visible light image features and infrared image features respectively;

[0054] The feature difference module is used to calculate the multi-layer feature differences between visible light images and infrared images;

[0055] The geometric constraint module is used to utilize the fixed structure of the document as prior knowledge;

[0056] A multi-level supervision strategy is used to simultaneously utilize pixel-level, regional, and global supervision signals;

[0057] A self-calibrating sample generator is used to simulate ID card image pairs under different shooting conditions.

[0058] Furthermore, in the dual-stream feature extraction module, the visible light feature extraction branch uses a modified version of DenseNet as the backbone network, retaining shallow features while introducing a block attention mechanism to focus on different functional areas of the document, such as the common vanishing areas of portraits and the vanishing text areas above passports, etc., and finally outputs five feature maps at different scales: Fv1, Fv2, Fv3, Fv4, and Fv5; the infrared image feature extraction branch uses the same network structure as the visible light branch, but does not share weights, and adds a channel attention module after each convolutional block to enhance the unique texture expression of infrared, and finally outputs five feature maps at different scales: Fir1, Fir2, Fir3, Fir4, and Fir5; in this module, a spatial transformation network (STN) is used to automatically locate the document area and generate a document area mask for defining the region of interest for subsequent feature extraction.

[0059] Furthermore, the feature difference module employs a multi-layer differential feature calculation method to calculate the differences in the feature maps at each scale:

[0060]

[0061] Here, γ and β are learnable parameters. The first term captures the absolute difference, and the second term focuses on the features of the infrared disappearance region. Finally, it generates differential feature maps at 5 scales: D1, D2, D3, D4, and D5.

[0062] Furthermore, the geometric constraint module first establishes a document template library containing standard layouts for different types of documents, and uses a Defomable CNN to adapt to document distortion under different shooting angles.

[0063] Another objective of this invention is to provide a multi-national certificate authentication system based on multispectral images. This system includes a device characteristic compensation module, an ultraviolet anti-counterfeiting feature extraction and detection module, an ultraviolet fluorescent fiber detection module, and an infrared differential feature extraction and detection module.

[0064] The device feature compensation module is an algorithm compensation module designed for the significant differences in brightness and contrast of ultraviolet images acquired by different devices.

[0065] The ultraviolet anti-counterfeiting feature extraction and detection module and the ultraviolet fluorescent fiber detection module are detection modules for ultraviolet anti-counterfeiting features, respectively.

[0066] The infrared differential feature extraction and detection module is a detection module for the infrared anti-counterfeiting features of passports.

[0067] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0068] The pre-screening algorithm based on difference images proposed in this invention can effectively screen out passports with general anti-counterfeiting features by calculating the absolute difference between the images, performing morphological operations on the difference images, and quantifying the differences between the images, thereby greatly improving the efficiency of document authentication.

[0069] This invention can be used for document authentication in complex environments and supports cross-device algorithm compensation, effectively solving the problem of significant adjustments to algorithm modules required for device migration in similar algorithms. Considering that some authentication features in ultraviolet images may be affected by passport wear and accidental occlusion by frontline operators, the algorithm employs Multi-Scale Texture Analysis (MSTA). The passport ultraviolet image is decomposed at multiple scales to create an image pyramid. A co-occurrence grayscale matrix (GLCM) is used to capture the spatial relationships between pixel pairs. The Gabor filter, a miniature human visual system, is employed for its selectivity in direction and frequency. Wavelet transform features are extracted to capture different frequency components in the fluorescent pattern. The Local Directional Pattern (LDIP) algorithm is used to capture the directional structure in the fluorescent pattern. Finally, MSTA fuses all features at multiple scales to generate a comprehensive fluorescent texture representation. Compared to similar algorithms, this invention's algorithm still performs well in complex authentication scenarios.

[0070] This invention proposes an algorithm for detecting fluorescent fibers in passport ultraviolet images, which can provide strong technical support for the detection of such irregularly distributed fluorescent targets. For a long time, the detection of fluorescence on passport ultraviolet patterns has relied on traditional methods, which are inefficient and struggle to distinguish fluorescent fibers from similar patterns such as thin lines and fluorescent text. This invention employs a multispectral data fusion method, fusing ultraviolet and visible light images to enhance the contrast of fluorescent lines. Since pixel positions in ultraviolet and white light images may mismatch due to shooting angles, lens distortion, etc., affine transformations are needed to strictly align them, ensuring that the coordinates of the same physical point are consistent in both images. For feature fusion, a channel attention mechanism with dynamic weighted fusion is used, automatically learning the importance weights of ultraviolet and white light images through a Squeeze-and-Excitation module. This invention also designs a 3-layer CNN network, taking an RGB image as input as a single-channel mask after fluorescence enhancement. The enhanced image is then stitched together with the original image to obtain the final image. This lightweight fluorescence feature enhancement subnetwork is designed specifically to extract high-frequency information from the fluorescence channel and suppress background noise. Finally, the image enhanced with fluorescence features is input into the fluorescent fiber target detection module to enhance global contextual awareness of the fluorescent lines.

[0071] The infrared image disappearance pattern detection network provided by this invention can be used for counterfeit detection of various complex passport infrared feature patterns. For a long time, due to the diversity of passport infrared feature patterns, such as disappeared text information, disappeared facial information, and partially disappeared digital outlines, most methods have encountered problems such as inaccurate feature region localization and insufficient precision in feature region detection when facing these scenarios. This invention designs a multimodal differential feature enhancement network (MM-DFEN). Feature enhancement is one component, aiming to improve the accuracy and robustness of detection. The entire network is a complete infrared disappearance feature detection system, from feature extraction, difference calculation, constraint optimization to the final output segmentation mask, ultimately detecting the target in the infrared feature region.

[0072] The technological advancements of this invention are mainly reflected in the following aspects:

[0073] (1) For passport ultraviolet image authentication, a device response model based on a deep statistical model was introduced, and multi-scale texture analysis was adopted to decompose the image into multiple scales for feature extraction. The multi-dimensional anti-counterfeiting features of the ultraviolet image were combined for comprehensive analysis. A three-layer CNN fluorescence enhancement network was designed to extract high-frequency information of the fluorescence channel by using a multi-spectral feature fusion method. Finally, the image after fluorescence feature enhancement was input into the fluorescence fiber target detection module to enhance the global context perception of the fluorescence lines.

[0074] (2) Specifically, for passport infrared images, this invention designs a multimodal differential feature network, which includes a dual-stream feature extraction module, a multi-layer differential feature calculation module, a geometric constraint module, a multi-level decoder, a multi-level supervision module, etc. The entire network has complete infrared feature enhancement, extraction, and detection functions. Furthermore, thanks to its geometric constraint module, the network structure ensures that the passport image adapts to distortions caused by shooting from different angles. Compared to the potential failure of traditional image processing methods in specific scenarios, the solution of this invention has better robustness.

[0075] The solution mentioned in this invention has achieved significant technological advancements in areas such as pre-screening of general anti-counterfeiting features for certificates and licenses, compensation for anti-counterfeiting characteristics of equipment, and verification of certificates and licenses in complex scenarios involving irregular distribution of ultraviolet anti-counterfeiting features and infrared anti-counterfeiting features. These advancements not only improve the accuracy and reliability of this task but also provide new ideas and more effective technical means for combining traditional methods with deep learning methods to perform certificate and license verification.

[0076] (3) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0077] The expected benefits and commercial value of this technical solution are considerable. Firstly, the pre-screening algorithm based on difference images proposed in this invention can effectively filter passports with universal anti-counterfeiting features, thereby greatly improving the efficiency of document authentication. Simultaneously, this technical solution addresses the problem of excessive differences in image contrast, brightness, and color during document authentication across different devices. This significantly improves the effectiveness of image acquisition, reading, and authentication across devices due to variations in ultraviolet light intensity and wavelength, which can lead to significant adjustments to the algorithm framework. This technical solution enables a single authentication algorithm to be deployed on different devices, which will greatly attract airports, border control, customs, and various enterprises and institutions with document authentication needs to choose authentication devices based on this technical solution, rather than combining devices with different functions. Meanwhile, due to differences in hardware infrastructure between large and small document readers, the images captured by them often exhibit significant differences. The implementation of this technical solution facilitates the invention of new hardware platforms, requiring no major adjustments to the algorithm structure and greatly enhancing its universality. Secondly, this technical solution provides a solution for detecting irregularly distributed fluorescent fibers and various feature patterns under infrared light on documents, possessing significant potential and expected to be adopted by other industries, thereby generating sustained revenue. Furthermore, the patent protection of this technical solution will further ensure competitiveness and market share, bringing long-term commercial value and competitive advantage to enterprises. In conclusion, this technical solution has high expected returns and commercial value, and is expected to become an important technical and commercial standard for document collection systems at airports and border inspection ports.

[0078] (4) The technical solution of this invention fills a technical gap in the industry both domestically and internationally:

[0079] For a long time, the pre-screening of universal anti-counterfeiting features for certificates and documents has been an unresolved problem. The pre-screening algorithm based on difference images proposed in this invention can effectively screen passports with universal anti-counterfeiting features, which is of great significance for improving the efficiency of the entire authentication system. For a certificate authentication system, most current systems adjust the algorithm parameters and even the framework for different devices when authenticating multispectral images acquired across different devices. For example, when large document inspection equipment acquires images, the strong ultraviolet light results in higher contrast in the acquired ultraviolet images of certificates and documents, with differences in color, brightness, and other aspects. The method provided in this technical solution compensates for the device parameters based on a depth statistical model, thereby effectively solving the problems of difficult certificate recognition and inaccurate authentication across different devices. This effectively fills a technological gap in the industry both domestically and internationally. Furthermore, this invention has also achieved significant breakthroughs in fluorescent fiber detection and infrared region feature detection. Previously, fluorescent fiber detection mainly relied on traditional methods. However, these methods suffered from low efficiency and accuracy when dealing with irregularly distributed and interfering patterns. This invention employs deep learning and multispectral image data fusion to combine ultraviolet and visible light images, improving the contrast of fluorescent lines. Similarly, for infrared detection, traditional techniques are less robust to the rich anti-counterfeiting features in passport infrared images, often failing to detect complex features. The multimodal differential network designed in this invention highly integrates traditional image processing and deep learning methods, employing a multi-layered network structure to effectively handle the complex texture features of anti-counterfeiting areas in passport infrared images, providing a new technical solution and target for the development of infrared image authentication for documents.

[0080] Attached Figures / Tables

[0081] Figure 1 This is a flowchart of a multi-national certificate authentication method based on multispectral images provided in an embodiment of the present invention;

[0082] Figure 2 This is a structural diagram of a multi-national certificate authentication system based on multispectral images provided in an embodiment of the present invention;

[0083] Figure 3 This is a schematic diagram of a multispectral image of a Chinese citizen's passport provided in an embodiment of the present invention;

[0084] Figure 4 These are the infrared (left) and ultraviolet (right) images of Chinese citizen passports generated after passing through a pre-screening module based on common anti-counterfeiting features, as provided in this embodiment of the invention.

[0085] Figure 5This is a structural diagram of a pre-screening module based on universal anti-counterfeiting features provided in an embodiment of the present invention;

[0086] Figure 6 This is a schematic diagram of equipment characteristic model compensation provided in an embodiment of the present invention;

[0087] Figure 7 This is a diagram of the network structure for extracting ultraviolet anti-counterfeiting features of certificates provided in this embodiment of the invention;

[0088] Figure 8 This is a schematic diagram of the distribution of ultraviolet fluorescent fibers in the certificate provided in this embodiment of the invention;

[0089] Figure 9 This is a diagram of the ultraviolet fluorescence fiber detection network structure for certificates provided in this embodiment of the invention;

[0090] Figure 10 These are the input image, fused and enhanced image, and detection result image for fluorescent fiber detection provided in this embodiment of the invention;

[0091] Figure 11 This is a diagram of the multimodal differential feature enhancement network structure provided in this embodiment of the invention. Detailed Implementation

[0092] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0093] like Figure 1 As shown, this embodiment of the invention provides a method for authenticating multinational certificates based on multispectral images. The method specifically includes:

[0094] S1: Pre-screening based on difference images. This part mainly detects common anti-counterfeiting features of certificates, such as whether patterns disappear under infrared light or whether fluorescent patterns exist under ultraviolet light. This is mainly accomplished by calculating the absolute difference between the images, performing morphological operations on the difference images, and quantifying the image differences. Part S1 is the starting point of the entire technical solution and is used to initially screen certificates that do not have conventional anti-counterfeiting features.

[0095] S2: Based on the input image in S1, in order to compensate for the differences in the fluorescence characteristics of ultraviolet images in terms of image contrast and brightness caused by the different ultraviolet wavelengths and intensities when ultraviolet images are captured on different devices, the S2 part mainly constructs the characteristic model of the device based on the depth statistical characteristics of the images acquired by the device, so as to make its image characteristics closer to those of the standard device.

[0096] S3: Based on feature compensation in S2, S3 serves as the ultraviolet image feature extraction layer. Here, the multi-scale texture analysis (MSTA) method is mainly adopted. Texture is a visual feature that spans different spatial frequencies. Fluorescent patterns often exhibit different texture characteristics at multiple scales. MSTA combines statistical features, structural features, spectral features, model features, etc., and is very suitable for the detection of passports, which are prone to fading of some texture features due to wear and tear over the years.

[0097] S4: Detection of irregular fluorescent fiber targets in ultraviolet images of certificates. This invention employs a multispectral fusion method, physically aligning the ultraviolet image with the white light image through affine transformation. The Squeeze-and-Excitation module automatically learns the importance weights of the ultraviolet and white light image channels, achieving dynamic fusion of multispectral features. A three-layer CNN network is also designed for fluorescence enhancement as the entry point for the detection model.

[0098] S5: In response to the characteristic that some text or patterns on certificates disappear under infrared light, this invention combines traditional image processing and deep learning algorithm frameworks to design a multimodal differential feature enhancement and detection network, which significantly improves the anti-counterfeiting feature recognition capability.

[0099] S1 firstly requires identifying whether the input infrared image has anti-counterfeiting features that disappear under infrared light, and whether the input ultraviolet image has fluorescent anti-counterfeiting patterns under ultraviolet light.

[0100] The following algorithms are applied to the infrared and ultraviolet images of the image: In the preprocessing stage, the image captured under white light is converted to grayscale to reduce the influence of color on the difference calculation; in terms of difference calculation, the absolute difference between the two images is calculated, and then the difference image is binarized to highlight significant differences. Finally, morphological dilation is performed to connect adjacent difference points and reduce the influence of noise; in the difference quantification stage, the pre-screened images are mainly quantified using the infrared structural similarity index irSSIM and the ultraviolet difference pixel ratio uvDiffPercentage to measure whether the image has general anti-counterfeiting features. The calculation formulas for irSSIM and uvDiffPercentage are as follows:

[0101]

[0102]

[0103] in, , These are the mean values ​​of the ultraviolet image and the white light image, respectively. , These are the variances of the ultraviolet image and the white light image, respectively; σ is the covariance; and C1 and C2 are constants used to prevent the denominator from being zero.

[0104] After processing in step S1, two types of passports can be initially identified: one type lacks common anti-counterfeiting features and is output as a fake passport; the other type possesses common anti-counterfeiting features. For this type of passport, the multispectral image of the passport is further input into step S2 for more precise anti-counterfeiting feature detection.

[0105] Specifically, S2 employs fluorescence image correction using a depth statistical model. It is assumed that an image I can be represented as a composite function of device features and content features:

[0106]

[0107] Where S is the scene radiation, Represents content feature functions, The present invention aims to find a mapping function g such that: (This represents a device characteristic function.)

[0108]

[0109] in This represents the characteristic function of the standard device, so that device compensation can be achieved by the algorithm regardless of the device, thereby realizing the fundamental cross-device anti-counterfeiting feature area detection.

[0110] To obtain the mapping function g, this invention needs to solve the following: 1. Separating device features and content features in the image; 2. Constructing a compensation mapping from the source device to the standard device. The most basic goal of constructing the compensation mapping is to compensate for a series of differences in color, contrast, noise, and other factors caused by different devices.

[0111] From a statistical perspective, this can be expressed as solving for a conditional probability distribution:

[0112]

[0113] Where S is a latent variable representing device-independent scene radiation.

[0114] This approach primarily employs Multi-Level Feature Transform and Statistical Correction (MFTSC). Firstly, Multi-Scale Geometric Analysis (MGA) is used to decompose the image into components of different frequencies and directions.

[0115]

[0116] in, These are the coefficients of scale s and direction d. is the corresponding basis function, and r is the residual. Here, non-subsampled contour wave transform or curve wave transform is used to replace the traditional wavelet transform, which better captures curve features and direction features.

[0117] The coefficient distribution of each sub-band is accurately modeled using the generalized Gaussian distribution (GGD):

[0118]

[0119] In this model, α controls the peak level of the distribution, σ controls the diffusion, and adaptive kernel density estimation is used instead of parameter distribution to more accurately describe the non-Gaussian distribution of fluorescence properties.

[0120] The parametric model for constructing the device characteristics is as follows, including spectral characteristics, nonlinear transformation, and noise characteristics:

[0121]

[0122] Where M is the color mixing matrix, r is the nonlinear response function, b is the offset, and η is the noise term. A multi-segment distributed model is used here to replace simple gamma correction, which more accurately characterizes the nonlinear response of the device.

[0123] For each subband, the mapping between the source device and the standard device is constructed using optimal transmission theory:

[0124]

[0125] Where F represents the cumulative distribution function, the Wasserstein distance minimization and regularized optimal transmission are used to solve the overfitting problem in traditional histogram matching.

[0126] To address the physical constraints of fluorescence characteristics, the fewer emission properties of fluorescent materials introduced, the better the correction results conform to physical laws.

[0127]

[0128] Where E(λ) represents the excitation spectrum, R(λ) represents the response characteristics of the fluorescent material, and S(λ) is the spectral sensitivity of the sensor. The less physical-inspired spectrum used and the more local color uniformity regularization, the better.

[0129] Adaptive covariance structure cointegration is employed, and multivariate statistical analysis is used to correct the characteristic covariance structure among different devices.

[0130]

[0131] Here, Σ represents the covariance matrix and μ represents the mean vector. Robust covariance estimation and sparse precision matrix estimation are applied to improve robustness to outliers.

[0132] The original scene radiation is estimated using an iterative Bayesian framework:

[0133]

[0134]

[0135] The convergence and computational efficiency are improved by combining variational Bayesian approximation with Markov chain Monte Carlo (MCMC) sampling.

[0136] The extraction and analysis of texture features in S3 is a crucial step in the anti-counterfeiting of certificates. Texture is a visual feature that spans different spatial frequencies. The fluorescent patterns of certificates often exhibit different texture characteristics at multiple scales: 1. At the macro scale, the distribution and shape of the fluorescent pattern of the passport as a whole have obvious texture characteristics. 2. At the meso scale, the stripes and patterns inside the fluorescent area of ​​the passport have unique texture characteristics. 3. From the micro perspective, the fine texture and particle characteristics of the fluorescent material also contain obvious texture information.

[0137] Document anti-counterfeiting differs from general anti-counterfeiting measures, and its algorithm design must be grounded in reality, conforming to actual situations and needs. In practical document anti-counterfeiting work, passports often experience wear and tear due to years of use, leading to the fading of some texture features, especially on a macro scale. If passport anti-counterfeiting measures are based solely on a single texture feature, the fading caused by wear can easily lead to erroneous conclusions. Furthermore, at frontline anti-counterfeiting sites such as border crossings, the high volume of people and the heavy workload of frontline staff can easily result in situations where key anti-counterfeiting features are partially obscured. In such cases, judging passport authenticity based solely on a single anti-counterfeiting feature can easily lead to incorrect conclusions.

[0138] To address the specific challenges of passport anti-counterfeiting, this invention employs a multi-scale texture analysis (MSTA) method. First, MSTA decomposes the image at multiple scales, creating an image pyramid. Second, it uses a co-occurrence grayscale matrix (GLCM) to capture the spatial relationships between pixel pairs, which is highly suitable for describing the texture regularity and directionality of fluorescent regions. Third, it uses a Gabor filter to simulate the human visual system's selectivity for direction and frequency, making it well-suited for detecting directional structures in fluorescent images. Fourth, wavelet transform provides localized analysis of frequency and space; here, wavelet transform features are extracted to capture components of different frequencies in the fluorescent pattern. Fifth, it employs the Local Directional Pattern (LDIP) algorithm, which focuses more on the directional characteristics of pixel domains, making it highly suitable for capturing the directional structure of fluorophores. Finally, MSTA fuses all the above features at multiple scales to generate a comprehensive fluorescent texture representation.

[0139] In S4, the passport ultraviolet image contains many unevenly distributed fluorescent lines, which this invention refers to as fluorescent fibers. For example... Figure 8 As shown, fluorescent fibers are thin, elongated fluorescent lines with irregular distribution, varying lengths, and even inconsistent brightness. Traditional target detection algorithms, such as the Viola Jones detector, HOG detector, and part-based variable model (DPM) detector, suffer from problems such as inaccurate target localization, missed detections, and misidentification when detecting this type of target. Therefore, this invention employs a deep learning approach to detect fluorescent fibers.

[0140] This invention categorizes the targets to be detected and distinguished into three types: fluorescent fibers, thin lines, and fluorescent text. The primary target for identification is fluorescent fibers; the inclusion of thin lines and fluorescent text aims to prevent detection confusion due to their similarity to fluorescent fibers.

[0141] First, sample images containing fluorescent lines must be collected, ensuring image diversity: different angles, brightness, background complexity, length of fluorescent lines, etc. When processing fluorescent fibers in passport UV images, thin lines in the background and some fluorescent text can interfere with the detection of fluorescent fibers, so these parts must be distinguished.

[0142] This invention employs a multispectral data fusion method to fuse ultraviolet (UV) and visible light images, enhancing the contrast of fluorescent lines. Since pixel positions in UV and white light images may mismatch due to differences in shooting angle, lens distortion, rotation, scaling, or sensor position, affine transformation is required to strictly align them, ensuring that the coordinates of the same physical point are consistent in both images. Specifically, ORB or AKAZE algorithms are used to extract key points and feature descriptors from both images, and FLANN or BFMatcher is used for feature point matching to select high-quality matching pairs. The least squares method is used to fit the transformation matrix, and the affine transformation matrix is ​​solved using the matching points. Finally, the UV image is aligned to the RGB image according to the transformation matrix. For feature fusion, a channel attention mechanism with dynamic weighted fusion is employed. The Squeeze-and-Excitation module automatically learns the importance weights of the UV and white light images, achieving dynamic fusion of multispectral features. In detail, the channel attention mechanism enhances useful features and suppresses noise by automatically learning the importance weights of different channels (such as ultraviolet and white light channels). Its detailed steps mainly include: First, concatenating the aligned ultraviolet image (1 channel) and white light image (3 channels) along the channel dimension to form a 4-channel input tensor; Second, compressing the two-dimensional spatial features of each channel into a scalar through global average pooling to represent the global importance of that channel; Third, learning the nonlinear relationship between channels through a fully connected layer to generate a weight vector; Fourth, multiplying the learned weight vector s with the original input channels for weighted fusion.

[0143] This invention also designs a 3-layer CNN network, taking an RGB image as input and outputting a fluorescence-enhanced single-channel mask. The network is trained using pixel-level annotations of the fluorescence image. The enhanced image and the original image are then stitched together to obtain the final image. A lightweight fluorescence feature enhancement sub-network is designed specifically to extract high-frequency information from the fluorescence channel and suppress background noise. Finally, the fluorescence-enhanced image is input into the fluorescence fiber target detection module to enhance global contextual awareness of the fluorescent lines.

[0144] As described in S5, due to the special materials used in its manufacture, certain special patterns or text on the certificate will partially disappear when photographed under infrared light, becoming its unique anti-counterfeiting features. However, the disappearance of these features is not entirely regular; some features, such as facial patterns, will disappear completely, while the upper half of some text patterns will disappear. This complexity makes traditional algorithms difficult to handle, posing a significant challenge to the detection of the anti-counterfeiting features of this invention.

[0145] This invention designs a multimodal differential feature enhancement network that highly integrates traditional image processing and deep learning methods. Specifically, it designs a feature difference module to calculate multi-layer feature differences between visible light and infrared images; a geometric constraint module that utilizes the fixed structure of the document as prior knowledge; introduces a multi-level supervision strategy, simultaneously using pixel-level, regional, and global supervision signals; and designs a self-calibrating sample generator to simulate document image pairs under different shooting conditions.

[0146] This invention employs a dual-stream feature extraction module, using two branches for feature extraction: one for visible light images and the other for infrared images. The visible light feature extraction branch uses a modified version of DenseNet as its backbone network, preserving shallow features while introducing a block-based attention mechanism to focus on different functional regions of the document, such as the common vanishing image region and the text vanishing region above a passport. It outputs five feature maps at different scales: Fv1, Fv2, Fv3, Fv4, and Fv5. The infrared image feature extraction branch uses the same network structure as the visible light branch, but without sharing weights. A channel attention module is added after each convolutional block to enhance the unique texture representation of infrared images. It also outputs five feature maps at different scales: Fir1, Fir2, Fir3, Fir4, and Fir5. This module uses a Spatial Transformation Network (STN) to automatically locate document regions and generate document region masks for defining regions of interest in subsequent feature extraction.

[0147] This invention employs a multi-layer differential feature calculation module to calculate the feature differences between infrared and white light images. This module incorporates an adaptive feature alignment layer to eliminate the differences in the statistical distributions of the two branches. A multi-layer differential feature calculation method is used to calculate the differences in the feature maps at each scale:

[0148]

[0149] Here, γ and β are learnable parameters; the first term captures the absolute difference, and the second term focuses on the features of the infrared vanishing region. Finally, differential feature maps at five scales are generated: D1, D2, D3, D4, and D5.

[0150] To ensure that ID card images adapt to document distortion from different shooting angles, this invention employs a geometric constraint module to constrain and correct the physical structure of the image. First, an ID card template library is established, containing standard layouts for different types of ID cards. Then, a Defomable CNN is used to adapt to document distortion under different shooting angles.

[0151] To further enhance the expressive power of differential features, a Feature Pyramid Network (FPN) structure was designed, fusing multi-scale features from both bottom-up and top-down perspectives. Treaty connections were used to combine shallow detail features with deep semantic features, forming an enhanced multi-scale feature representation.

[0152]

[0153] This invention employs a dilated spatial pyramid pooling (ASPP) module to capture multi-scale context and uses dilated convolutions with different dilation rates to expand the receptive field, thereby enhancing the global contextual understanding of features.

[0154]

[0155] Simultaneously, to minimize noise and preserve characteristic galaxies, a self-attention mechanism is designed to filter noisy features, and a channel attention gating mechanism is introduced to suppress irrelevant channels. Finally, residual connections are used to preserve the original feature information.

[0156]

[0157] To reconstruct the vanishing region mask from the enhanced features, a multi-level decoder is required. This invention employs progressive upsampling, using dedicated convolutions to gradually restore feature resolution. At each upsampling stage, skip connection features of the corresponding scale are fused, and residual modules are used to improve gradient flow. Intermediate prediction results are generated at each decoding stage: {M1, M2, M3, M4, M5}, and an attention-weighted mechanism is used to fuse multi-scale predictions.

[0158]

[0159] The weight coefficients are dynamically generated using the science department's attention network. Edge-guided convolutions are used to refine the segmentation boundaries, and a Conditional Random Field (CRF) post-processing method is introduced to improve boundary accuracy. Boundary consistency constraints are applied.

[0160]

[0161] To achieve effective supervised training, this invention implements end-to-end multi-level supervised training, including pixel-level, region-level, and global-level supervision. Pixel-level supervision uses binary cross-entropy loss to supervise the final prediction mask.

[0162]

[0163] Applying Dice loss to address class imbalance issues:

[0164]

[0165] And a weighted combination was used:

[0166]

[0167] Regional-level supervision calculates regional-level loss based on the functional areas of the document, while assigning different weights to different anti-counterfeiting feature areas, focusing on key anti-counterfeiting areas. The regional weighted loss function is:

[0168]

[0169] Global-level supervision introduces an adversarial training mechanism. The discriminator D evaluates the authenticity of the segmentation results, and an adversarial loss is introduced.

[0170]

[0171] The global consistency loss function is:

[0172]

[0173] By integrating multi-level monitoring signals and using adaptive weights to adjust the sharing of different loss terms, the overall loss function is:

[0174]

[0175] like Figure 2 As shown in the figure, the present invention provides a multi-national certificate authentication system based on multispectral images. The system includes a device characteristic compensation module, an ultraviolet anti-counterfeiting feature extraction and detection module, an ultraviolet fluorescent fiber detection module, and an infrared differential feature extraction and detection module.

[0176] The device feature compensation module is an algorithm compensation module designed for different devices, such as large document examination equipment and small mini equipment, where the ultraviolet images they collect have significant differences in brightness and contrast.

[0177] The ultraviolet anti-counterfeiting feature extraction and detection module and the ultraviolet fluorescent fiber detection module are detection modules for ultraviolet anti-counterfeiting features, respectively.

[0178] The infrared differential feature extraction and detection module is a detection module for the infrared anti-counterfeiting features of passports.

[0179] Example 1: Passport Authenticity Verification at Ports of Entry

[0180] Passports are used to verify identity during border crossings. Passport authenticity verification is a crucial step in border control. Historically, passport verification has relied primarily on the experience of frontline inspectors for visual inspection, a method that is inefficient and requires extensive practical experience. The technical solution of this invention is applicable to this scenario.

[0181] 1. A pre-screening module based on common certificate anti-counterfeiting features initially screens out certificates that do not have common certificate anti-counterfeiting features.

[0182] 2. Equipment feature compensation: Compensate for the equipment characteristics of the counterfeit detection equipment used at the port to ensure that the collected images are consistent with those of the standard equipment.

[0183] 3. Perform multi-scale texture analysis on the collected passport ultraviolet images, extract statistical features, structural features, spectral features, model features, etc. of the passport ultraviolet images, and integrate the multi-scale texture features for comprehensive judgment.

[0184] 4. The acquired passport ultraviolet images are subjected to affine transformation and multispectral fusion using the fluorescent fiber enhancement detection module designed in this invention to detect irregularly distributed fluorescent fibers.

[0185] 5. A multimodal differential feature enhancement and detection network is used to extract infrared features from the acquired passport infrared images and perform detection.

[0186] 6. Based on a comprehensive judgment combining the results of ultraviolet image authentication and infrared image authentication, give the overall authentication result.

[0187] Example 2: Multispectral images were acquired in a laboratory environment using a large document examination instrument and a small document reader, and then the documents were authenticated according to the technical solution of the present invention.

[0188] In a laboratory environment, various types of multispectral acquisition devices for documents, such as large document examiners and small document readers, can be used. The multispectral images acquired by these different devices have significant differences in contrast, brightness, and other aspects, posing a considerable challenge to the technical solution of this invention.

[0189] 1. A pre-screening module based on common certificate anti-counterfeiting features initially filters out images that do not have common certificate anti-counterfeiting features.

[0190] 2. The multispectral images acquired by large document examination equipment and small certificate reading equipment have significant differences in brightness and contrast. Based on the depth statistical characteristics of the images acquired by the equipment, a characteristic model of the equipment is constructed to compensate for the images and make its image characteristics closer to those of standard equipment.

[0191] 3. Perform multi-scale texture analysis on images acquired by large document examination equipment and small certificate reader, extract statistical features, structural features, spectral features, model features, etc. of passport ultraviolet images, and integrate multi-scale texture features for comprehensive judgment.

[0192] 4. The fluorescent fiber enhancement detection module designed in this invention is used to perform affine transformation and multispectral fusion on the ultraviolet images collected by large document examination equipment and small certificate reader to detect irregularly distributed fluorescent fibers.

[0193] 5. For infrared images acquired by large document examination equipment and small certificate readers, a multimodal differential feature enhancement and detection network is used to extract infrared features from the acquired passport infrared images and perform detection.

[0194] 6. Based on a comprehensive judgment combining the results of ultraviolet image authentication and infrared image authentication, give the overall authentication result.

[0195] These two case studies demonstrate the broad applicability of multispectral image-based document authentication methods and systems across various scenarios, from port access to immigration management. This approach effectively improves the efficiency and accuracy of document authentication. Through algorithms such as device characteristic compensation, multi-scale texture ultraviolet feature extraction, ultraviolet fluorescence detection, and multimodal differential feature enhancement networks, it can more accurately identify the authenticity of passports from different countries and regions, significantly enhancing the security of national immigration management.

[0196] The document authentication method and system based on multispectral images provided in this invention include the following steps:

[0197] Specifically, for the input infrared image, this invention generally needs to identify whether it has an anti-counterfeiting feature pattern that disappears under infrared light. For the input ultraviolet image, this invention needs to identify whether it has a fluorescent anti-counterfeiting pattern under ultraviolet light.

[0198] This invention performs the following algorithmic processing on infrared and ultraviolet light images respectively: In the preprocessing stage, the image captured under white light is converted to grayscale to reduce the influence of color on the difference calculation; in terms of difference calculation, this invention mainly calculates the absolute difference between the two images, performs an absolute difference operation on the two images, then binarizes the difference image to highlight significant differences, and finally performs morphological dilation to connect adjacent difference points and reduce the influence of noise; in the difference quantification stage, the difference percentage is mainly used for quantification, which refers to the percentage of non-zero pixels in the difference image after thresholding out of the total number of pixels.

[0199] Figure 3 The example demonstrates the input multispectral image of an embodiment of the present invention. The present invention applies the aforementioned pre-screening algorithm based on general anti-counterfeiting features to both the infrared and ultraviolet images of the image. Experimental results in section S1 are shown below. Figure 4 As shown, white areas (bright spots) represent locations where there is a significant difference between two images, while black areas indicate that the two images are essentially the same or differ less than a set threshold at these locations. Figure 4 As can be seen from this invention, the white areas in the infrared image are concentrated in the left face area, the middle area where the text disappears, and the right area where the portrait disappears. In the ultraviolet image, the white areas are concentrated in the central fluorescent pattern and the fluorescent lines covering the entire area. The pre-screened images are primarily quantified using the infrared structural similarity index irSSIM and the ultraviolet difference pixel ratio uvDiffPercentage to measure whether the image possesses universal anti-counterfeiting features. The calculation formulas for irSSIM and uvDiffPercentage are as follows:

[0200]

[0201]

[0202] in, , These are the mean values ​​of the ultraviolet image and the white light image, respectively. , These are the variances of the ultraviolet image and the white light image, respectively; σ is the covariance; and C1 and C2 are constants used to prevent the denominator from being zero.

[0203] To verify the effectiveness of the S1 method, this paper designed a positive-negative comparison experiment. First, a positive sample group was set up: 500 multispectral images of genuine passports from multiple countries (such as China, Cambodia, and Armenia); a negative sample group was set up: 500 high-quality counterfeit documents, including some printed copies lacking spectral characteristics. First, white light images were... Convert to grayscale, then calculate the infrared difference images separately. and UV difference image Morphological dilation was performed using a 3×3 structuring element to connect discrete difference points. Then, the irSSIM and uvDiffPercentage indices were calculated for each pair of images, while a pre-screening threshold was set. and .

[0204] The pre-screening results of S1 based on general anti-counterfeiting features are shown in Table 1. As can be seen from the table, genuine certificates exhibit significant pattern disappearance under infrared light, resulting in an extremely low structural similarity index (irSSIM) between them and white light images. Simultaneously, the fluorescent pattern excited by ultraviolet light maintains a high uvDiffPercentage. Through the pre-screening of the S1 module, the system successfully intercepted 98.8% of obviously counterfeit samples, allowing subsequent S2-S5 modules to process only suspected samples with general anti-counterfeiting features, significantly reducing the system's computational load.

[0205] After processing in step S1, two types of passports can be initially identified: one type lacks common anti-counterfeiting features and is output as a fake passport; the other type possesses common anti-counterfeiting features. For this type of passport, the multispectral image of the passport is further input into step S2 for more precise anti-counterfeiting feature detection.

[0206] Specifically, S2 employs fluorescence image correction using a depth statistical model. As described in Embodiment 2 of this invention, the invention uses a large document examination device and a small certificate reader to acquire multispectral images. The fluorescence characteristics captured by the two devices under ultraviolet light excitation show significant differences: the large document examination device exhibits a bright green fluorescence with clear boundaries and rich texture features, while the small certificate reader, as the standard device, captures images with a pale green fluorescence, low contrast, blurred boundaries, and insufficient detail. The objective of this invention is to compensate for the device characteristics of the small certificate reader, making its acquired ultraviolet images approach those of the standard device.

[0207] First, this invention represents the image formation process of two devices based on a theoretical model:

[0208]

[0209] Where A represents a large-scale document examination device, B represents a small-scale document reader, and S represents the actual fluorescent radiation characteristic of a passport. Indicates fluorescence content characteristics, and These represent the imaging characteristics of the two devices, respectively.

[0210] The goal of this invention is to find a mapping function g such that:

[0211]

[0212] That is to say:

[0213]

[0214] First, the image acquired by device B undergoes multi-scale geometric decomposition. The decomposition results in low-frequency residuals and high-frequency components at different scales and orientations. Next, this invention performs statistical characteristic analysis on each sub-band and estimates the color response characteristics of device B, applying an optimal transport mapping for each sub-band. The image is reconstructed using the mapped coefficients, and color correction is applied. Then, physical constraints on fluorescence are applied to ensure that the corrected image conforms to fluorescence physical properties. Finally, variational Bayesian methods are used for iterative optimization.

[0215] like Figure 6As shown, from left to right, the images are those acquired by device A, those acquired by device B, and the compensated image of device B. It can be seen that the fluorescence color of the compensated image of device B changes from pale yellow-green to bright green, the fluorescence boundary is clearer, and the details and textures are restored. At the same time, the SSIM increases from 0.65 to 0.92, and the IOU of the fluorescence region increases from 0.70 to 0.95.

[0216] S3 specifically, such as Figure 7 As shown, this invention employs a multi-scale texture analysis (MSTA) method. First, MSTA decomposes the image at multiple scales to create an image pyramid. Second, a co-occurrence grayscale matrix (GLCM) is used to capture the spatial relationships between pixel pairs, which is well-suited for describing the texture regularity and directionality of fluorescent regions. Third, a Gabor filter is used to simulate the selectivity of the human visual system for direction and frequency, making it suitable for detecting directional structures in fluorescent images. Fourth, wavelet transform provides localized analysis of frequency and space; here, wavelet transform features are extracted to capture components of different frequencies in the fluorescent pattern. Fifth, the Local Directional Pattern (LDIP) algorithm is used, which pays more attention to the directional characteristics of pixel domains and is well-suited for capturing the directional structure of fluorophores. Finally, MSTA fuses all the above features at multiple scales to generate a comprehensive fluorescent texture representation.

[0217] S4 specifically, such as Figure 7 As shown, passport ultraviolet images contain many unevenly distributed fluorescent lines, which this invention refers to as fluorescent fibers. This invention categorizes the targets to be detected and distinguished into three types: fluorescent fibers, thin lines, and fluorescent text. The primary target for identification is fluorescent fibers. The inclusion of thin lines and fluorescent text as categories aims to prevent detection confusion due to their similarity to fluorescent fibers.

[0218] like Figure 8As shown, this module employs a multispectral data fusion method to fuse ultraviolet (UV) and visible light images, enhancing the contrast of fluorescence lines. Since pixel positions in UV and white light images may mismatch due to differences in shooting angle, lens distortion, rotation, scaling, or sensor position, affine transformation is used to strictly align them, ensuring that the coordinates of the same physical point are consistent in both images. Specifically, ORB or AKAZE algorithms are used to extract key points and feature descriptors from both images, and FLANN or BFMatcher is used for feature point matching to select high-quality matching pairs. The least squares method is used to fit the transformation matrix, and the affine transformation matrix is ​​solved using the matching points. Finally, the UV image is aligned to the RGB image according to the transformation matrix. For feature fusion, a channel attention mechanism with dynamic weighted fusion is employed. The Squeeze-and-Excitation module automatically learns the importance weights of the UV and white light images, achieving dynamic fusion of multispectral features. In detail, the channel attention mechanism enhances useful features and suppresses noise by automatically learning the importance weights of different channels (such as ultraviolet and white light channels). Its detailed steps mainly include: First, concatenating the aligned ultraviolet image (1 channel) and white light image (3 channels) along the channel dimension to form a 4-channel input tensor; Second, compressing the two-dimensional spatial features of each channel into a scalar through global average pooling to represent the global importance of that channel; Third, learning the nonlinear relationship between channels through a fully connected layer to generate a weight vector; Fourth, multiplying the learned weight vector s with the original input channels for weighted fusion.

[0219] This invention also designs a 3-layer CNN network, taking an RGB image as input and outputting a fluorescence-enhanced single-channel mask. The network is trained using pixel-level annotations of the fluorescence image. The enhanced image and the original image are then stitched together to obtain the final image. A lightweight fluorescence feature enhancement sub-network is designed specifically to extract high-frequency information from the fluorescence channel and suppress background noise. Finally, the fluorescence-enhanced image is input into the fluorescence fiber target detection module to enhance global contextual awareness of the fluorescent lines.

[0220] like Figure 9 The image shown is the input UV image for this module. It can be seen from the input image that the fluorescent fibers are unevenly distributed, the lines are thin, and the contrast with the background is not strong enough. After affine transformation and multispectral feature fusion, the input UV image enters a 3-layer CNN network for fluorescence enhancement. Figure 9 This is the output image after fluorescence enhancement using a 3-layer CNN network. It can be seen that the thickness of the fluorescent lines is improved, and the contrast with the surrounding background is also significantly enhanced. Finally, the image after fluorescence feature enhancement is input into the fluorescent fiber target detection module to enhance global contextual awareness of the fluorescent lines. Figure 10The results of fluorescent fiber detection are shown. As can be seen from the figure, even in extreme cases where there is fluorescent text interference in the background and the fiber length is less than 20 pixels, the fused and enhanced CNN network can still accurately mark the fiber position (with confidence scores all above 0.85), effectively avoiding the situation of misjudging slender fluorescent text as fibers.

[0221] Specifically, S5 employs a multimodal differential feature enhancement network to detect the disappearance of patterns in specific infrared regions. For example... Figure 11 As shown, the network uses dual-stream feature extraction. The visible light branch focuses on the overall structure and content of the ID card, while the infrared branch enhances the texture features of the remaining visible areas through channel attention. The differential feature calculation module calculates the following for the national emblem region:

[0222]

[0223] For portrait regions with significant differences, the β parameter is automatically increased to enhance the response to completely vanished regions. For vanished text regions, an adaptive feature alignment layer is used to handle subtle partial vanishing. The geometric constraint module calls the passport standard template library to verify the spatial distribution of vanished regions. An FPN structure is used to fuse shallow texture features to deep semantic information. The ASPP module pays special attention to the partial vanishing phenomenon of text regions, and a self-attention mechanism filters out noise from document edges and background. The multi-level decoding module generates prediction masks at five scales to accurately locate vanished regions, and edge-guided convolution processes the fine boundaries of text regions. Finally, an accurate infrared vanished region mask is output.

[0224] The experimental setup for the multimodal differential feature enhancement network proposed in S5 during training and testing is as follows: The input passport visible light and infrared images are resized to 512×512. During training, enhancement strategies such as random flipping, rotation, and boundary cropping are employed to prevent overfitting of the training images. The backbone network parameters are initialized using the Swin-B pre-trained model, while the remaining parameters are initialized to PyTorch's default settings. The network is trained using the Adam optimizer with a batch size of 8 and an initial learning rate of 5e-5. The learning rate is divided by 10 every 100 epochs, and the model converges within 150 epochs. Experiments are conducted on a Linux server with an NVIDIA 5090 32GB GPU, using PyTorch version 2.8.0, Python version 3.12, Ubuntu version 22.04, and CUDA version 12.8.

[0225] The multimodal differential feature enhancement network used in this invention achieves an accuracy of over 96% in recognizing character-level vanishing patterns and 94.5% in recognizing special vanishing patterns of decorative groups. Furthermore, it can adapt to the differentiated features of passports from different countries, representing a significant improvement over traditional methods.

[0226] In summary, compared to existing methods, the feature compensation based on device characteristics proposed in this invention is the first feature compensation module for document authentication devices to employ a deep statistical model. Furthermore, the ultraviolet fluorescence feature extraction based on multiple texture features integrates various texture features, resulting in stronger accuracy and reliability. This invention is the first to apply affine transformation and multispectral fusion to fluorescent fiber detection and uses a 3-layer CNN structure to enhance fluorescence features. For infrared anti-counterfeiting feature detection, this invention employs a multimodal differential feature enhancement and detection network, significantly improving the reliability of infrared anti-counterfeiting feature detection and facilitating the practical application of document authentication devices.

[0227] Table 1 shows the experimental results of the pre-screening module based on general anti-counterfeiting features provided in the embodiments of the present invention.

[0228] Table 1

[0229] Sample categories Total sample size irSSIM average uvDiffPercentage (average) Pre-screening pass rate Real passport 500 0.12 12.4% 98.6% Forged passport 500 0.88 0.8% 1.2%

[0230] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0231] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for authenticating multinational certificates based on multispectral images, characterized in that, The method includes: S1: Based on the pre-screening of difference images, detect the general anti-counterfeiting features of certificates and licenses. This is accomplished by calculating the absolute difference between the images, performing morphological operations on the difference images, and quantifying the image differences. S2: Construct a characteristic model of the device based on the depth statistical features of the images acquired by the device, and then compensate the images to make their image features closer to those of standard devices; S3: Extract and detect UV anti-counterfeiting features using multi-scale texture analysis (MSTA); S4: The ultraviolet image is physically aligned with the white light image through affine transformation. The importance weights of the ultraviolet and white light image channels are automatically learned through the Squeeze-and-Excitation module to achieve dynamic fusion of multispectral features. At the same time, a three-layer CNN network is designed for fluorescence enhancement as the entry point of the detection model. S5: A multimodal differential feature enhancement and detection network was designed for infrared images to detect complex anti-counterfeiting features under infrared light.

2. The multi-national certificate authentication method based on multispectral images as described in claim 1, characterized in that, S1 performs the following algorithm processing on the infrared and ultraviolet light images of the image respectively: In the preprocessing stage, the image taken under white light is grayscale processed; in terms of difference calculation, the absolute difference between the two images is calculated, the absolute difference between the two images is calculated, then the difference image is binarized to highlight the significant difference, and finally morphological dilation is performed to connect adjacent difference points. In the difference quantification stage, the pre-screened images are mainly quantified using the infrared structural similarity index irSSIM and the ultraviolet difference pixel ratio uvDiffPercentage to measure whether the images have general anti-counterfeiting features. The calculation formulas for irSSIM and uvDiffPercentage are as follows: irSSIM=(2m uv m bi +C1)(2σ+C2)) / ((μ uv 2 +m bi 2 +C1)(s uv 2 +s bi 2 +C2)); uvDiffPercentage = (Number of non-zero pixels / Total number of pixels) × 100%; Where, μ uv μ bi These are the mean values ​​of the ultraviolet image and the white light image, respectively, σ uv , σ bi These are the variances of the ultraviolet image and the white light image, respectively; σ is the covariance; and C1 and C2 are constants to prevent the denominator from being 0. After processing in S1, two types of passports can be initially identified: one type lacks common anti-counterfeiting features and is output as a fake passport, while the other type possesses common anti-counterfeiting features. For the latter type of passport, the multispectral image of the passport is further input into S2 for more precise anti-counterfeiting feature detection.

3. The multi-national certificate authentication method based on multispectral images as described in claim 1, characterized in that, The S2 method employs multi-level feature transformation and statistical correction (MFTSC), first using multi-scale geometric analysis (MGA) to decompose the image into components of different frequencies and directions: ； in, These are the coefficients of scale s and direction d. Here, r is the corresponding basis function, and r is the residual. The coefficient distribution of each sub-band is accurately modeled using the generalized Gaussian distribution (GGD): ； α controls the peakness of the distribution, σ controls the diffusion, and adaptive kernel density estimation is used instead of parameter distribution; The parametric model for constructing the device characteristics is as follows, including spectral characteristics, nonlinear transformation, and noise characteristics: ； Where M is the color mixing matrix, r is the nonlinear response function, b is the offset, and η is the noise term; For each subband, the mapping between the source device and the standard device is constructed using optimal transmission theory: ； Where F represents the cumulative distribution function, and Wasserstein distance minimization and regularized optimal transport are employed; considering the physical constraints of fluorescence characteristics, the fewer emission characteristics of the fluorescent material introduced, the better to ensure that the correction result conforms to physical laws. ； Where E(λ) represents the excitation spectrum, R(λ) represents the response characteristics of the fluorescent material, and S(λ) is the spectral sensitivity of the sensor; Adaptive covariance structure cointegration is employed, and multivariate statistical analysis is used to correct the characteristic covariance structure among different devices. ； Where Σ represents the covariance matrix and μ represents the mean vector; The original scene radiation is estimated using an iterative Bayesian framework: ； 。 4. The multi-national certificate authentication method based on multispectral images as described in claim 1, characterized in that, The steps in S3 are as follows: (1) MSTA performs multi-scale decomposition on the image and creates an image pyramid; (2) Use co-occurrence grayscale matrix (GLCM) to capture the spatial relationship between pixel pairs; (3) A Gabor filter is used to simulate the selectivity of the human visual system to direction and frequency; (4) Extract wavelet transform features to capture components of different frequencies in the fluorescent pattern; (5) The LDIP algorithm using the local orientation mode pays more attention to the orientation characteristics of the pixel domain; (6) MSTA fuses all the above features at multiple scales to generate a comprehensive fluorescent texture characterization.

5. The multi-national certificate authentication method based on multispectral images as described in claim 1, characterized in that, In step S4, the targets to be detected and distinguished are divided into three categories: fluorescent fibers, thin lines, and fluorescent text. First, sample images containing fluorescent lines are collected, and the diversity of the images is ensured: different angles, brightness, background complexity, length of fluorescent lines, etc. A multispectral data fusion method is adopted to fuse ultraviolet and visible light images. The ORB or AKAZE algorithm is used to extract key points and feature descriptors from the two images. FLANN or BFMatcher is used for feature point matching, and high-quality matching pairs are selected. The least squares method is used to fit the transformation matrix, and the affine transformation matrix is ​​solved through the matching points. Finally, the ultraviolet image is aligned to the RGB image according to the transformation matrix. In terms of feature fusion, a channel attention mechanism is used for dynamic weighted fusion. The importance weights of ultraviolet and white light images are automatically learned through the Squeeze-and-Excitation module to achieve dynamic fusion of multispectral features. The present invention also designs a 3-layer CNN network with RGB image as input and fluorescence-enhanced single-channel mask as output. The network is trained using pixel-level annotations of the fluorescence image. The enhanced image and the original image are stitched together to obtain the final image. Finally, the image after fluorescence feature enhancement is input into the fluorescence fiber target detection module to enhance the global contextual awareness of the fluorescence lines.

6. The multi-national certificate authentication method based on multispectral images as described in claim 1, characterized in that, The S5 multimodal differential feature enhancement and detection network includes: The dual-stream feature extraction module uses two branches to extract features for visible light image features and infrared image features respectively; The feature difference module is used to calculate the multi-layer feature differences between visible light images and infrared images; The geometric constraint module is used to utilize the fixed structure of the document as prior knowledge; A multi-level supervision strategy is used to simultaneously utilize pixel-level, regional, and global supervision signals; A self-calibrating sample generator is used to simulate ID card image pairs under different shooting conditions.

7. The multi-national certificate authentication method based on multispectral images as described in claim 1, characterized in that, The dual-stream feature extraction module uses a modified version of DenseNet as the backbone network for the visible light feature extraction branch, preserving shallow features while introducing a block attention mechanism to focus on different functional areas of the document, such as the common vanishing image area and the vanishing text area above the passport. It outputs five feature maps at different scales: Fv1, Fv2, Fv3, Fv4, and Fv5. The infrared image feature extraction branch uses the same network structure as the visible light branch, but does not share weights. A channel attention module is added after each convolutional block to enhance the unique texture representation of infrared images. It also outputs five feature maps at different scales: Fir1, Fir2, Fir3, Fir4, and Fir5. This module uses a Spatial Transformation Network (STN) to automatically locate document regions and generate document region masks for defining the regions of interest in subsequent feature extraction.

8. The multi-national certificate authentication method based on multispectral images as described in claim 1, characterized in that, The feature difference module employs a multi-layer differential feature calculation method to calculate the differences in the feature map at each scale: ； Here, γ and β are learnable parameters. The first term captures the absolute difference, and the second term focuses on the features of the infrared disappearance region. Finally, it generates differential feature maps at 5 scales: D1, D2, D3, D4, and D5.

9. The multi-national certificate authentication method based on multispectral images as described in claim 1, characterized in that, The geometric constraint module first establishes a document template library containing standard layouts for different types of documents, and then uses a Defomable CNN to adapt to document distortion under different shooting angles.

10. A multi-national certificate authentication system based on multispectral images as described in claims 1-9, characterized in that, The system includes a device characteristic compensation module, an ultraviolet anti-counterfeiting feature extraction and detection module, an ultraviolet fluorescent fiber detection module, and an infrared differential feature extraction and detection module. The device feature compensation module is an algorithm compensation module designed for the significant differences in brightness and contrast of ultraviolet images acquired by different devices. The ultraviolet anti-counterfeiting feature extraction and detection module and the ultraviolet fluorescent fiber detection module are detection modules for ultraviolet anti-counterfeiting features, respectively. The infrared differential feature extraction and detection module is a detection module for the infrared anti-counterfeiting features of passports.

Images